The CLICS database in its current format makes direct use of the data assembled by the Concepticon project in order to aggregate lexical data from different sources. At the same time, the CLICS database itself can be seen as an interesting conceptlist, providing information on concept polysemy and semantic similarity.

For this reason, I added a CLICS conceptlist reflecting the results of the first version of the CLICS database (List et al. 2014) to one of the earlier versions of Concepticon. This concept list, called List 2014 1280 offers different data on each of the Concepticon Concept Sets which were reflected in the first version of CLICS, as described in the following table.

Column	Description
Frequency	The number of languages in which a word for a given concept is attested.
Degree	The number of concepts with which another concept shares a colexification.
WeightedDegree	The sum of the number of different language families in which any colexification of the given concept is attested.
Rank	The rank, with respect to the degree of a given concept.
CommunityID	The identifier for the Infomap community that was computed for the data.
CommunityLabel	The central concept in the respective community, i.e., the concept with the highest degree.

Although CLICS has since then been updated two times (List et al. 2018, Rzymski et al. 2020), the corresponding concept lists have not yet been added for Concepticon. Since the creation of these concept lists involves some code, I decided to illustrate how one can produce the conceptlists from the CLICS datasets with some lines of Python code.

Code and Data Requirements

The code can be used for both the data underlying CLICS2 and the data underlying CLICS3. In both cases, the data is available in form of a file in GML-format, a rather flexible format for graphs, which contains the results of all calculations in form of annotations to the nodes and edges of the respective graphs. In both cases, the file that we will need is called infomap-3-families.gml. The file name reflects that the communities in the data were computed with help fo the Infomap algorithm (Rosvall and Bergstrom 2007) and that the threshold was set to 3 language families in order to accept a given colexification.

The data for CLICS2 can be found in the folder output/graphs/infomap-3-families.gml.zip at https://github.com/clics/clics2, and for CLICS3 it can be found in the main folder of the CLICS3 GitHub repository at https://github.com/clics/clics3 or in the CodeOcean capsule (https://codeocean.com/capsule/7201165/tree/v2, see under results/graphs) accompanying the study.

In the following, I will assume that you have downloaded the files and placed them in the same folder in which you also place the script to run the code. To make it easier to distinguish both files, I also assume that you rename them in clics2.gml and clics3.gml, respectively.

There are a couple of code requirements, such as LingPy (List et al. 2019), python-igraph (Csárdi and Nepusz 2006), and networkx (Hagberg 2009), which can be most easily installed with pip (see the installation instructions we provide for the CLICS3 package for details).

Before we start with the little script, we have to import the libraries we need.

import networkx as nx
import igraph
from lingpy.convert.graph import igraph2networkx

Note that we need igraph merely for loading the data in the GML format here, since the networkx package has a very strict requirement that all data be coded in ASCII. However, since this illustration uses networkx to iterate over the data and compute the missing data points, we will use lingpy‘s conversion function to convert the data to networkx in a second run.

We now add a little custom function that allows us to run the script with two different configurations, one for CLICS2, and one for CLICS3.

from sys import argv
if '2' in argv[1:]: 
    clics = 'clics3' 
    ref = 'List-2018' 
else: 
    clics = 'clics3' 
    ref = 'Rzymski-2020'

This line will change the file we load (if we pass 2 as an argument, we will load clics2.gml, otherwise clics3.gml), and the name which we give to the identifiers in Concepticon and the file (since the authors differ for both lists).

Loading the graph

As mentioned before, we need to use the igraph package to load the graph. While we could also compute the relevant data points with this package alone, it is faster for myself to convert the graph to a networkx graph object, since I know the relevant functions better. In order to do so, we use the function as it is provided by LingPy. Given that LingPy expects a specific input format from the igraph graph, we need to give each node the attribute name, which is missing when loading the graph directly, but all in all this is a very small workaround.

_G = igraph.read(clics+'.gml')
for node in _G.vs:
    node['name'] = node['label']
G = igraph2networkx(_G)

Computing the degrees

One of the simplest way to compute polysemy scores from CLICS data is to compute the degree of each concept, i.e., the number of links it has to other concepts with respect to colexifications. In addition, we can also compute the weighted degree, which counts, how often a given colexification for a given concept occurs. Here, we have two possibilities: we can compute how many languages show a given colexification, or how many families.

deg = nx.degree(G)
fdeg = nx.degree(G, weight='FamilyWeight')
ldeg = nx.degree(G, weight='LanguageWeight')

Writing the data to file

We can now write the data to file. In order to do so, we create a simple list, in which we already insert the header.

table = [[
        'ID',
        'ENGLISH',
        'CONCEPTICON_ID',
        'CONCEPTICON_GLOSS',
        'FAMILY_FREQUENCY',
        'LANGUAGE_FREQUENCY',
        'WORD_FREQUENCY',
        'RANK',
        'COMMUNITY',
        'CENTRAL_CONCEPT',
        'DEGREE',
        'WEIGHTED_FAMILY_DEGREE',
        'WEIGHTED_LANGUAGE_DEGREE'
        ]]

While we have already computed the degrees, we will compute the rank on the fly, by sorting our graph according to the (unweighted) degree. The community refers to the identifier of the Infomap community which was computed for the respective study, and the central concept refers to the central concept in the respective community, measured by its (unweighted) degree. The information for both the community identifier and the central concept are both available in the graph we just loaded, as are the information on the frequency of the word (with respect to the attestation in the CLICS database, in form of words, languages, and as reflected in language families).

Filling the concept list with data is now straightforward. We just loop across the network, which we sort at the same time, and then add the relevant information to the table.

for i, (node, data) in enumerate(sorted(
        G.nodes(data=True), 
        key=lambda x: deg[x[0]],
        reverse=True,
        )):
    table += [[
        '{0}-{1}-{2}'.format(ref, len(G), i+1),
        data['Gloss'],
        data['ConcepticonId'],
        data['Gloss'],
        str(int(data['FamilyFrequency'])),
        str(int(data['LanguageFrequency'])),
        str(int(data['WordFrequency'])),
        str(i+1),
        data['infomap'],
        data['CentralConcept'],
        str(int(deg[node])),
        str(int(fdeg[node])),
        str(int(ldeg[node])),
        ]]

Writing data to file

Once this is done, it is also only a three-liner to write the data to file (and it could be a oneliner, but I have no ambitions with respect to brevity here):

with open('{0}-{1}.tsv'.format(ref, len(G)), 'w') as f:
    for line in table:
        f.write('\t'.join(line)+'\n')

Submitting the data to Concepticon

In this form, little more information is required to make a pull request to the Concepticon at https://github.com/concepticon/concepticon-data and add the two new concept lists. The only things missing are

• the description of the concept list in conceptlists.tsv,
• the references for the two studies (they are available in BibTex from EvoBib),
• the metadata file in JSON format (which can be automatically created).

More information on how this can be done, can be found in the last months blog by Annika Tjuka (Tjuka 2020).

References

Gábor Csárdi and Tamás Nepusz (2006): The igraph software package for complex network research. InterJournal. Complex Systems .1695. .

Hagberg, Aric (2009): NetworkX. High productivity software for complex networks. http://networkx.lanl.gov/index.html.

List, Johann-Mattis and Greenhill, Simon and Tresoldi, Tiago and Forkel, Robert (2019): LingPy. A Python library for quantitative tasks in historical linguistics. Version 2.6.5. Max Planck Institute for the Science of Human History. Jena: http://lingpy.org.

List, Johann-Mattis and Prokić, Jelena (2014): A benchmark database of phonetic alignments in historical linguistics and dialectology. In: Proceedings of the Ninth International Conference on Language Resources and Evaluation. 288-294.

List, Johann-Mattis and Greenhill, Simon J. and Anderson, Cormac and Mayer, Thomas and Tresoldi, Tiago and Forkel, Robert (2018): CLICS². An improved database of cross-linguistic colexifications assembling lexical data with help of cross-linguistic data formats. Linguistic Typology 22.2. 277-306.

Rosvall, M. and Bergstrom, C. T. (2008): Maps of random walks on complex networks reveal community structure. Proceedings of the National Academy of Sciences 105.4. 1118-1123.

Rzymski, Christoph and Tiago Tresoldi and Simon Greenhill and Mei-Shin Wu and Nathanael E. Schweikhard and Maria Koptjevskaja-Tamm and Volker Gast and Timotheus A. Bodt and Abbie Hantgan and Gereon A. Kaiping and Sophie Chang and Yunfan Lai and Natalia Morozova and Heini Arjava and Nataliia Hübler and Ezequiel Koile and Steve Pepper and Mariann Proos and Briana Van Epps and Ingrid Blanco and Carolin Hundt and Sergei Monakhov and Kristina Pianykh and Sallona Ramesh and Russell D. Gray and Robert Forkel and List, Johann-Mattis (2020): The Database of Cross-Linguistic Colexifications, reproducible analysis of cross- linguistic polysemies. Scientific Data 7.13. 1-12.

Tjuka, Annika (2020): Adding concept lists to Concepticon: A guide for beginners. Computer-Assisted Language Comparison in Practice 3.1. URL: https://calc.hypotheses.org/2225

---

---