While the core of the Concepticon project (https://concepticon.clld.org, List et al. 2019) are the numerous conceptlists which are constantly being added by the growing list of contributors, we have already from the beginning of the project, with the first version (List et al. 2016) tried to collect various kinds of concept metadata for all our concept sets.
Linguists can never solely concentrate on language. This is especially true for field linguists who document a previously unknown language. Unveiling the charm of an undocumented language requires the researcher to explore as much as possible about the people, the society, and of course, the nature around them.
As we have seen in previous instances of this blog post series, there are many parallels but also many differences between the evolutionary branches of biology and of linguistics. In the following, I will present a comparison of the causes due to which two related inheritable entities (e.g. two words or two genes of different languages or species) may differ from each other, or two unrelated ones resemble each other. The linguistic categories presented here can also be found in List (2016) whereas the biological categories are largely based on Koonin (2005).
Besides new algorithms and tools that facilitate established workflows, one change prompted by computer-assisted approaches to language comparison is a distinct relationship between scientists and their data. A critical part of our work, and perhaps the one with the most lasting impact, is to promote an approach in which the data life-cycle is not constrained within the limits of planning and publishing a study. Data are organized and planned for reuse in investigations perhaps not even considered during collection, with the output of one project becoming the input of another.
As was mentioned in the introduction to this series of blogposts, both species and languages are often presented in a tree model. In biology, trees of each individual gene are created in order to account for horizontal transmission and other processes in which the history of a gene differs from the general history of its genome. From the sum of these trees, the species trees are then derived, a method called gene tree reconciliation (Nakhleh 2013). In linguistics on the other hand, phylogenetic trees normally are built on cognate sets of related words, from which the most likely tree of the languages is calculated. A closer equivalent however would be to describe the history of each individual word or word form, including regular sound change, irregular changes to its form, semantic changes, borrowings, and processes of word formation, and to derive the language tree based on the sum of the word histories (Gray, Greenhill, and Ross 2007, 15). Unlike its biological equivalent, this is normally done manually.
Having seen how we can obtain a simple scorer derived from the feature system in CLTS (List et al. 2019) in last month’s post, what is missing now, in order to use the scorer for alignment analyses, is an alignment function which can take the scorer as an argument. If one does not have higher ambitions with respect to the alignment function itself, this step can be achieved in a very straightforward way with help of LingPy’s (List et al. 2018)
sw_align() method. As can be seen from the documentation, this method takes as input two sequences (i.e., lists of sounds), along with a scoring function. Obviously, all we need to do now is to create our specific scorer based on the CLTS features, and then pass this scoring function along with our sequences to the function.
In the past, people have repeatedly asked me how they could use their own scoring functions in combination with LingPy’s alignment algorithms. Their major concern was that the sound-class-based scoring systems we use in LingPy might fail to reflect true phonetic similarity of sounds, specifically also because they are not informed by classical ideas about distinctive features in phonology. As described in detail in List (2014), LingPy converts sounds in phonetic transcription to an internal alphabet of less than 30 letters, to which the alignment algorithms are then applied in a second stage.
During the 24th International Conference of Historical Linguistics, I was asked by a colleague whether I would know a good way to align and scores sentences available in form of phonetic transcriptions. While it is clear that one can roughly compare the difference between sequences rather easily by aligning them, and calculating, for example, the edit distance between them, it is clear that the task of sentence alignment could be done in a somewhat more subtle way.
One of the crucial steps in creating a database of lexical cognates is the selection of concepts one wants to use for a given study. While many scholars use the classical Swadesh list of 200 items) (Swadesh 1952) for this purpose, or the combined list of 207 items, in which the former has been merged with Swadesh’s updated list of 100 items (Swadesh 1955), and which is often mistakenly attributed to Swadesh himself, although the first official reference seems to be Comrie (1977), it is useful to give the selection of concepts some more thought initially.
Rooting of phylogenetic trees is an important task, not only in evolutionary biology, but also in historical linguistics. So far, however, different rooting methods have not yet been sufficiently tested on linguistics data. Given that a new method for the automatic rooting of phylogenetic trees has been presented recently in biology, it seemed to be a good occasion to test in detail how well this new method works in comparison with alternative methods.
Evolutionary biology and historical linguistics share a long history of scientific exchange, reflected both not only in the sharing and transfer of metaphors but more recently also in the transfer of methods. Already Charles Darwin claimed that both species and languages evolve in tree-like patterns, and linguists, like Wilhelm Meyer-Lübke, used terms like “sprachliche[n] Biologie” (‘linguistic biology’) when referring to the history of languages (Meyer-Lübke 1890, x). While the discipline of historical-comparative linguistics allowed biologists to adopt evolutionary thought against religious dogma in the late 19th century (Wells 1987: 54), it was biological applications which opened up the possibility of large quantitative studies in linguistics based on computational approaches (Geisler and List 2013, 111).
One of the major efforts behind our recently published paper on the origin and spread of the Sino-Tibetan languages (Sagart et al. 2019) was the creation of a database of lexical cognates which was used to run the phylogenetic analyses. The creation of this database started about four years ago, when I joined the Centre des Recherches Linguistiques sur l’Asie Oriental in Paris as a research fellow in January 2015, and Guillaume Jacques and Laurent Sagart approached me with the idea of making a phylogenetic study of Sino-Tibetan languages. In December 2017, almost three years after having started, our database consisted of 180 concepts translated into 50 different languages. Since creating the database was not directly straightforward from the beginning, with quite a few situations in which we realized we had to re-arrange the data or the procedure, I thought it might be useful to share our experience in a series of blog posts, as it might be interesting for scholars who wish to create their own database.
Mapping a given concept list to Concepticon can be done in a straight-forward way, even if automatic mappings need manual refinement. But what can we do when having to deal with larger datasets, say, a dictionary, from which we want to extract specific concepts, such as, for example, the ones in the classical Swadesh list of 100 items (Swadesh 1955)?
A major problem for data reuse in computer-assisted historical linguistics, especially when employing data collected with no computational workflows in mind, is linking datasets in terms of the meanings of the words (or, technically, “forms”) they carry. Just as linking languages across different datasets is not as straightforward as one might naively assume, demanding a complex reference catalog such as Glottolog, linking the concepts used in a wordlist (a “concept list”) to our Concepticon project might well be the most intensive task in preparing a dataset for cross-linguistic studies.
Having illustrated how a quick correspondence pattern analysis can be done with help of readily formatted data and the EDICTOR tool alone, it is now time to show how we can use the LingRex package in order to carry out a full-fledged correspondence pattern analysis. While EDICTOR uses a simple algorithm that is based on sorting the patterns, the Python algorithm for correspondence pattern detection, which is described in detail in List (2019), uses a greedy approach inspired by the Welsh-Powell algorithm for graph coloring (Welsh and Powell 1967), in order to cluster all alignment sites in the data into clusters which are compatible with each other.