Author Archives: Kellen Parker van Dam

Dialect Identification from Automatically Transcribed Speech

This study investigates whether automatic speech recognition (ASR) output can support rapid dialect identification in Khiamniungan, an under-documented Tibeto-Burman language spoken across Northeast India and Myanmar. Using transcripts produced by an existing wav2vec 2.0 ASR model, we train two text-based dialect identification systems on 83.93 minutes of speech from six speakers representing three dialects: Thang, Nokhu, and Peshu. The systems use either regularized logistic regression over syllable, syllable-pair, and phoneme n-gram features, or character n-gram language models scored by perplexity. Under file-grouped cross-validation, both models correctly classify 37 of 38 held-out files, achieving 0.974 accuracy and 0.975 macro F1. The models also recover the same dialect relationships, grouping Thang and Nokhu against Peshu. On two previously unseen broadcasts, including one by a speaker whose home variety differs from the variety spoken on air, both systems correctly identify Thang from approximately half a minute of automatic transcription. These results show that ASR transcripts can provide sufficient phonological and lexical information for efficient dialect identification without additional neural modeling, offering a lightweight tool for organizing and comparing documentary language archives.

Continue reading →

Digitizing Legacy Lexical Data of Muishaung for Computer-Assisted Language Comparison

This study describes the process of digitizing legacy materials into a computer-readable format for the purposes of computational typology and computer-assisted historical reconstruction. It presents a comparative wordlist that is made available in the formats recommended by the Cross-Linguistic Data Formats initiative.

Continue reading →

Preparing Acoustic Pitch Data for Computational Analysis and Presentation

Abstract

Pitch plays an important role in many linguistic systems. It is the primary set of features which determine vowel quality distinctions as well as forming the basis for intonation and contrastive tone systems. Unfortunately, much of the literature has relied on approaches to presenting and analysing pitch data that can result in a lack of data transparency, reproducibility, and analytical robustness. These issues are easily solved through the selection of a more appropriate scale for pitch values. This study presents the issues with using raw pitch data as Hertz values some historical efforts to resolve these issues, and two more appropriate solutions than some of the more widely used systems, with a way to easily calculate these alternative systems in a short Python script.
Continue reading →

Implementing Fuzzy Spelling Search in Dictionaries of Under-Described Languages Lacking Standard Orthographies

Non-standard orthographies are common in the world of under-described language documentation. Whether they are semi-conventionalised community spellings, orthographies partially adopted from missionary works, or hastily transcribed texts representing as-yet uncertain phonologies, there is a need to be able to work through lexical data in a way which can accommodate and respond to such non-standard transcriptions. Here, a few options are considered, with a solution for fuzzy string matching based on attested variations is presented.

Continue reading →