Skip to content

The official implementation of the method discussed in the paper Improving Spoken Language Identification with Map-Mix(work accepted at ICASSP-2023)

Notifications You must be signed in to change notification settings

skit-ai/Map-Mix

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

15 Commits
 
 
 
 
 
 

Repository files navigation

Map-Mix

The official implementation of the method discussed in the paper Improving Spoken Language Identification with Map-Mix(accepted at ICASSP 2023).

Abstract: The pre-trained multi-lingual XLSR model generalizes well for language identification after fine-tuning on unseen languages. However, the performance significantly degrades when the languages are not very distinct from each other, for example, in the case of dialects. Low resource dialect classification remains a challenging problem to solve. We present a new data augmentation method that leverages model training dynamics of individual data points to improve sampling for latent mixup. The method works well in low-resource settings where generalization is paramount. Our datamaps-based mixup technique, which we call Map-Mix improves weighted F1 scores by 2% compared to the random mixup baseline and results in a significantly well-calibrated model.

Table of Content

Introduction

Requirements and Installation

Each baseline methods mentioned in the paper(ie: pretrained-baselines, mixup-baselines, datamap-mixup, map-mix) are added to different branches of this repository.

Download the LRE-2017 Dataset and update the dataset path in config.json file.

git clone https://github.com/skit-ai/Map-Mix.git
cd Map-Mix

# checkout to the required method's branch
git checkout baselines
pip install -r requirements.txt

Training and Evaluation

The training and testing script for all the baselines and methods are same. Edit the config.json file for the respective branch with the training hyperparameters and dataset path.

# Train the model
python train.py
# Evaluate the model and generate test prediction csv files
python test.py
# Get the evaluation metrics
# add the csv path to the test_metrics.py file
python test_metrics.py

Results



Licence

Nil

Citation

@misc{https://doi.org/10.48550/arxiv.2302.08229,
  doi = {10.48550/ARXIV.2302.08229},
  url = {https://arxiv.org/abs/2302.08229},
  author = {Rajaa, Shangeth and Anandan, Kriti and Dalmia, Swaraj and Gupta, Tarun and Chng, Eng Siong},
  keywords = {Machine Learning (cs.LG), Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
  title = {Improving Spoken Language Identification with Map-Mix},
  publisher = {arXiv},
  year = {2023},
  copyright = {arXiv.org perpetual, non-exclusive license}
}

References

  1. mixup: Beyond Empirical Risk Minimization
  2. Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics
  3. Exploiting Class Similarity for Machine Learning with Confidence Labels and Projective Loss Functions
  4. S3PRL Github Repository
  5. Huggingface's transformers package

About

The official implementation of the method discussed in the paper Improving Spoken Language Identification with Map-Mix(work accepted at ICASSP-2023)

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published