Category Archives: Dataset

Creating a CLDF Wordlist from Heath et al.’s Dogon Comparative Wordlist

The Dogon and Bangime Linguistics project (https://dogonlanguages.org) offers a large comparative spreadsheet in which translational equivalents for a huge number of concepts are translated into various Dogon languages.  Due to its enormous size, no attempts have been made so far to integrate the spreadsheet with the lexical resources that were compiled as part of the CLDF initiative in order to populate the Lexibank repository. Here, we report a first attempt to circumvent the problems resulting from the size of the spreadsheet by convert not all but parts of the spreadsheet to CLDF Wordlist standards, which allows us to integrate parts of the data with other resources in Lexibank.

Continue reading

Creating a Standardized Comparative Wordlist of Newari Varieties

Newari is one of the few ancient Sino-Tibetan languages attested in written texts. Since previous studies on the phylogeny of Sino-Tibetan did not take Newari data into account, we felt it is important to close this gap by providing an up-to-date comparative wordlist of Newari varieties. This wordlist has now been finalized in a first version that has additionally been standardized following the recommendations of the Cross-Linguistic Data Formats initiative.
Continue reading

A New Dataset with Phonological Reconstructions in CLDF

Data in historical linguistics is typically presented in non-machine-readable formats, such as text-based supplementary material or even handwritten manuscripts. Many annotations and important facts are given in prose or remain within linguists’ heads. Those problems make it difficult for non-experts in the specific field to understand the data, and to reproduce and replicate the results, and also limits the exposure that linguists receive for their hard work. Similar to previous blog posts on retro-standardizing data, we present the digitization of a dataset that includes phonological reconstructions. By representing this kind of data in CLDF, we can apply a variety of computer-assisted methods to assess the quality of the reconstructions.

Converting Streitberg’s Gothic Dictionary to a CLDF Wordlist on a Windows System

I recently converted the Gothic dictionary written by Wilhelm Streitberg to a CLDF wordlist. Since I was using Windows, I had some difficulties during the conversion progress, which Unix system users may not have to deal with. I thought it would be useful to share my experience here and point out that users of Windows operating systems should be aware of certain aspects when converting data to CLDF.

Continue reading

Converting the Vietic Dataset by Sidwell and Alwes from 2021 to CLDF

A few days ago, Sidwell and Alwes submitted a very nice dataset on Vietic languages to Zenodo (10.5281/zenodo.5263194). When inspecting the data, I realized that this dataset could be easily converted to our CLDF formats in our new Lexibank standards. Since both authors explicitly invited for discussions of the data and the testing of the results, I thought it would even be better to quickly illustrate the CLDF conversion in a blog post, as this may enable colleagues to do the same with their datasets in the future.

Continue reading

Computer-Assisted Comparison of Gelong and Hlai using Cross-Linguistic Data Formats

In this study, we discuss the sparsely studied Gelong language of Hainan Island and its affiliation to the Hlai languages. Our work is based on Andy Chin’s article “The Gelong Language in the Multilingual Hub of Hainan”. We extracted Chin’s data and processed it with the help of various computer-assisted methods in order to make it more accessible, machine-readable, and comparable with other datasets.

Continue reading

A multilingual body part concept list

In the summer of 2018, I set out to collect data for my master’s thesis (Tjuka 2019). The goal was to elicit body part terms that can also refer to object and landscape features. This was BC (before COVID-19), so I was able to meet with informants living in Berlin at the time and conduct my urban fieldwork study. The informants who participated in the study spoke one of 13 different languages (e.g., Wolof, Vietnamese, Czech). As a first task, I elicited 28 body part terms to get a sense of the naming patterns in each language. This blog post provides background information and introduces the resulting multilingual body part concept list.

Continue reading

Data Gathering in Times of a Pandemic: Upcycling Constenla Umaña’s Data on the Chibchan, Lencan and Misumalpam Language Families

While searching for the topic of a small research project about the linguistic history of South America, I realized that a lot of data that is crucial for assessing central arguments is not openly available, but new data is difficult to come by these days. And when it is, it is not usually presented in data format that allows for easy reuse. Guided by these thoughts, I decided to turn towards the upcycling of previously published data (also called retro-standardization, see for example Geisler et al. (forthcoming) on the upcycling of the TPPSR dataset, https://tppsr.clld.org). The dataset I chose was previously published by Adolfo Constenla Umaña (2005). In this article, the author investigated comparatively the long-claimed genealogical relationship of three families of Central and South America, Chibchan, Lencan and Misumalpam (Lehmann 1920).

Continue reading

Mapping Multi-SimLex to Concepticon

Multi-SimLex (https://multisimlex.com) is a multilingual resource which provides user ratings for word pairs translated into different languages. The data is important for the evaluation of methods that derive word embeddings from large corpora. While it is on the one hand desirable to link such a large dataset to Concepticon, it is difficult to do so in concrete, given that the datasets represents word similarity ratings withouth any clear reference to concepts. In this post, I will show how the data can nevertheless be linked to Concepticon, and how the original Multi-SimLex data can be represented without losing any information in the form of a Concepticon Concept List.

Continue reading

Computing colexification statistics for individual languages in CLICS

In the last two weeks we had a renewed interest in colexifications, especially in the third generation of the “Database of Cross-Linguistic Colexifications” (Rzymski, Tresoldi, et al., 2020). The attention was due to two different and independent requests in few days. For those unfamiliar, the concept of “colexification” (François, 2008) refers to instances in which a language uses the same lexeme to express more than one comparable concept (e.g., Russian де́рево, which can mean both “tree” and “wood”). The CLICS project, first developed by List et al. (2014), is an offspring of the transparent approaches to standardization, aggregation, and curation of linguistic data that have been promoted within the CLDF framework (Forkel et al., 2018). It uses standardized lexical databases to identify “colexification networks”.

Continue reading

Towards a refined wordlist of German in the Intercontinental Dictionary Series

For a long time, I have been wondering about the origin of the German wordlist in the Intercontinental Dictionary Series (Key and Comrie 2016. Not only are many of the words given as translations for the large concept list of 1310 items very archaic variants, which are no longer in use, we also find many annoying problems, such as unusual spellings (consequently avoiding the letter “ß”, which is still in use, even if some people think differently), wrong translations, and, of course, no phonetic transcriptions. Already during my doctoral studies, I therefore started to work on a refined list, but I soon had so many other things on my plate, that I never really managed to finish this work. Recently, however, I realized that my previous work which I had done years ago was far more complete than I had thought, and I had even added information on potential borrowings, extracted from Kluge’s (2002) etymological dictionary. Given that this list can come in handy in various ways, I decided to finish the work and publish the list officially in a very first version.

Continue reading

Annotating Rhyme Judgments for a Complex Corpus of Manuscript Sources: Making Sense of the «Cang Jie pian 蒼頡篇»

Establishing a standardized annotation framework for communicating rhyme judgments identified in historical texts will both ease the use of computational tools for rhyme analysis, and hopefully inspire greater collaboration amongst scholars interested in historical linguistics. The framework we have proposed (List, Hill & Foster 2019), was designed with simplicity, exhaustiveness, and flexibility in mind (p. 30), with the intension of eventual inclusion in the Cross-Linguistic Data Formats initiative (https://cldf.clld.org). Further testing of the framework is desired to demonstrate its utility and identify areas requiring refinement. This study presents such a test on rhyming in the Cang Jie pian 蒼頡篇, an ancient Chinese scribal treatise only recently reconstructed from a complex corpus of surviving manuscript fragments. In a follow-up study, the proposal will be formally evaluated by providing code to test the annotations.

Continue reading

A list of 171 body part concepts

The body of most human beings consist of similar parts such as a head, arms, legs, and so on. Many body parts also occur in animals. The shapes and functions of body parts are universal across cultures, but speakers of various languages choose to categorize the body differently. For example, Vietnamese has a single word (tay) for the concepts HAND and ARM. The universality of the human body and its categorization into different parts have attracted attention across research areas such as lexical typology and cognitive science. Therefore, I present a comprehensive list of human and animal body part terms based on German which were mapped to the concepts in the Concepticon (List et al. 2020). The list is intended for investigations on cross-linugistic naming patterns of body parts.

Continue reading

A model of distinctive features for computer-assisted language comparison

This post introduces a model of segmental/distinctive features for the symbolic representation of sounds, covering almost 600 segments from CLTS (List et al., 2019) mapped to unique sets of bivalent features. It is being designed as an alternative input to vectors of presence/absence built from BIPA descriptors, analogous to other feature matrices like the one by Phoible (Moran & McCloy, 2019). While still under development, it can already be used both for training models of machine learning and statistics, notably decision trees, and for bootstrapping language- and process-specific models, aided by an “universal” and concise reference. The complete matrix is available on Zenodo. Asupporting Python library, distfeat, is available on PyPI.

Continue reading

Concept Similarity in STARLING

STARLING is a software package, originally created by Sergej A. Starostin, which is designed for historical linguists who want to build their own etymological dictionaries. It is not only a database system that allows its users to set up a very straightforward relational database structure, but also a package full of surprises, since it contains many methods that are supposed to automate specific tasks in historical linguistics. These range from phylogenetic tree reconstruction via the preliminary identification of sound correspondences up to the comparison of elicitation glosses for their semantic similarity. While phylogenetic reconstruction and sound correspondences are now quite successfully handled in alternative software packages, I thought it would be interesting to discuss the routine for assessing concept similarity in more detail, since it offers interesting possibilities for those who practice historical language comparison.

Continue reading