Computational approaches in historical linguistics have made great progress during the past two decades. As of now, it is much more common to propose subgroupings based on phylogenetic analyses than on traditional considerations using shared innovations. We have also seen a drastic increase in openly available datasets that share cognate judgments for various language families. Thanks to new standardization efforts providing facilitated access to several dozen comparative wordlists, it seems about time to work on on improved benchmarks of manually annotated cognates in computational historical linguistics. In this study, a first effort of this kind is undertaken, by presenting Lexibench, a preliminary gold standard for computational historical linguistics. Lexibench builds on the Lexibank repository to extract 63 multilingual wordlists, all manually annotated for cognacy, that can be used to assess the quality of cognate detection and phylogenetic reconstruction methods in computational historical linguistics.
Continue reading
Tag Archives: CLDF
Using CLDFBench and PyLexibank on Windows
Using tools such as CLDFBench and PyLexibank, datasets can be converted into Cross-Linguistic Data Formats (CLDF), offering a standardized and interoperable representation of linguistic data. While these tools are powerful, lifting datasets to CLDF can present unique challenges for Windows users due to idiosyncrasies in the Windows operating system. Although CLDFBench and PyLexibank are compatible with Windows, certain workarounds may be necessary to address system-specific issues. This guide aims to demonstrate how CLDFBench and PyLexibank can be effectively installed and used on a Windows computer to lift a dataset to CLDF.
Retrieving and Analyzing Taste Colexifications from Lexibank
Colexifications have enjoyed a considerable amount of popularity in the recent years. However, there are still many semantic domains, where not much research on colexification patterns has been carried out so far. Here we show, how the recently published Lexibank repository can be queried to yield colexification data on taste colexifications which can in turn be easily plotted on geographic maps.
Creating a CLDF Wordlist from Heath et al.’s Dogon Comparative Wordlist
The Dogon and Bangime Linguistics project (https://dogonlanguages.org) offers a large comparative spreadsheet in which translational equivalents for a huge number of concepts are translated into various Dogon languages. Due to its enormous size, no attempts have been made so far to integrate the spreadsheet with the lexical resources that were compiled as part of the CLDF initiative in order to populate the Lexibank repository. Here, we report a first attempt to circumvent the problems resulting from the size of the spreadsheet by convert not all but parts of the spreadsheet to CLDF Wordlist standards, which allows us to integrate parts of the data with other resources in Lexibank.
Creating a Standardized Comparative Wordlist of Newari Varieties
Newari is one of the few ancient Sino-Tibetan languages attested in written texts. Since previous studies on the phylogeny of Sino-Tibetan did not take Newari data into account, we felt it is important to close this gap by providing an up-to-date comparative wordlist of Newari varieties. This wordlist has now been finalized in a first version that has additionally been standardized following the recommendations of the Cross-Linguistic Data Formats initiative.
Continue reading
Creating Custom Commands in CLDF: From Lexibank to Nexus Files
Thanks to the CLDFBench Python package, CLDF datasets compiled with CLDFbench have a rich command-line utility that can easily be expanded by custom commands (Forkel et al. 2018, Forkel and List 2020) . Taking as an example the creation of a Nexus-file for phylogenetic analysis from an existing Lexibank Wordlist (List et al. 2022) , this tutorial will guide you through the necessary steps for writing a script that can be used as a CLDFBench command. You will then be able to apply the same workflow to other scripts you may want to use for a certain repository.
Querying Datasets with Cognates in the Lexibank Repository
Recently, I was asked by a colleague how one could query only those datasets in the Lexibank repository which come with cognate sets annotated by humans. While I first thought this could be done in a very straightforward way, I figeured out, when trying it myself, that the code still needs some workarounds. As a result, I thought it is best to share the solution I came up with in a blog post in order to make it accessible to colleagues who might be interested in inspecting and using the data provided by the Lexibank repository in more detail.
PyEDICTOR: A Small Python Package that Integrates LingPy, EDICTOR, and CLDF
With the introduction of CLDF as a major format for data exchange, there is an increased need in handy solutions for the conversion of CLDF to the formats required by computer-assisted tools like LingPy and EDICTOR, which allow to preprocess data automatically or to curate data by adding detailed annotations. With the publication PyEDICTOR, there is now a very lightweight software package that is supposed to provide first solutions for a successful integration of CLDF with these existing tools for computer-assisted language comparison.
How to Compute Colexifications with CL Toolkit (How to do X in Linguistics 10)
Colleagues often ask us how they could receive more detailed information on specific languages and colexifications in the CLICS database. With the publication of the CL Toolkit package, which allows to merge several CLDF datasets on the fly, carrying out analyses on certain parts of the data underlying the CLICS database is now much easier than before. In order to illustrate this, this tutorial shows how colexifications for a selected number of languages can be computed from two distinct datasets that are included in CLICS (Version 3).
Converting Streitberg’s Gothic Dictionary to a CLDF Wordlist on a Windows System
I recently converted the Gothic dictionary written by Wilhelm Streitberg to a CLDF wordlist. Since I was using Windows, I had some difficulties during the conversion progress, which Unix system users may not have to deal with. I thought it would be useful to share my experience here and point out that users of Windows operating systems should be aware of certain aspects when converting data to CLDF.
Converting the Vietic Dataset by Sidwell and Alves from 2021 to CLDF
A few days ago, Sidwell and Alves submitted a very nice dataset on Vietic languages to Zenodo (10.5281/zenodo.5263194). When inspecting the data, I realized that this dataset could be easily converted to our CLDF formats in our new Lexibank standards. Since both authors explicitly invited for discussions of the data and the testing of the results, I thought it would even be better to quickly illustrate the CLDF conversion in a blog post, as this may enable colleagues to do the same with their datasets in the future.
How to Share Data and Code when Submitting Papers to a Journal: Transparent Data (How to do X in Linguistics 8)
When sharing data and code when submitting papers to a journal, you need to make sure that the reviewers can test and inspect your data as conveniently as possible. In between reviews, you should also be maximally transparent on any changes that have been made to the data or the code underlying your study. When using data that was published elsewhere, this means you should pay specific attention to the versions you have used and make sure they are readily accessible.
Computer-Assisted Comparison of Gelong and Hlai using Cross-Linguistic Data Formats
In this study, we discuss the sparsely studied Gelong language of Hainan Island and its affiliation to the Hlai languages. Our work is based on Andy Chin’s article “The Gelong Language in the Multilingual Hub of Hainan”. We extracted Chin’s data and processed it with the help of various computer-assisted methods in order to make it more accessible, machine-readable, and comparable with other datasets.
A multilingual body part concept list
In the summer of 2018, I set out to collect data for my master’s thesis (Tjuka 2019). The goal was to elicit body part terms that can also refer to object and landscape features. This was BC (before COVID-19), so I was able to meet with informants living in Berlin at the time and conduct my urban fieldwork study. The informants who participated in the study spoke one of 13 different languages (e.g., Wolof, Vietnamese, Czech). As a first task, I elicited 28 body part terms to get a sense of the naming patterns in each language. This blog post provides background information and introduces the resulting multilingual body part concept list.
Data Gathering in Times of a Pandemic: Upcycling Constenla Umaña’s Data on the Chibchan, Lencan and Misumalpam Language Families
While searching for the topic of a small research project about the linguistic history of South America, I realized that a lot of data that is crucial for assessing central arguments is not openly available, but new data is difficult to come by these days. And when it is, it is not usually presented in data format that allows for easy reuse. Guided by these thoughts, I decided to turn towards the upcycling of previously published data (also called retro-standardization, see for example Geisler et al. (forthcoming) on the upcycling of the TPPSR dataset, https://tppsr.clld.org). The dataset I chose was previously published by Adolfo Constenla Umaña (2005). In this article, the author investigated comparatively the long-claimed genealogical relationship of three families of Central and South America, Chibchan, Lencan and Misumalpam (Lehmann 1920).