v0.3.1

What's Changed

• Update README.md by @xhluca in #60
• Make tqdm optional cause it interferes with thread monkey patching in gevent by @ddorian in #61
• Update setup.py by @xhluca in #62

New Contributors

• @ddorian made their first contribution in #61

Full Changelog: v0.3.0...v0.3.1

dl-translate v0.3.0

New

• NLLB is now available, allowing you to translate up to 200 languages. It is also much faster.
• Notebook demo for nllb

Changed

• New minimum version of transformers (4.30+) and PyTorch (2.0+)
• Modify behavior of _resolve_lang_codes
• Moved certain hidden functions to utils

Fixed

• dlt.utils: Changed default for available_languages, get_lang_code_map, available_codes. Previously was incorrectly set to mbart50.

Docs

• Improved Readme with examples for nllb
• Removed mentions of readthedocs.io
• Change language reference to table (previously bullet points)

dl-translate v0.2.6

Update readme with link to new docs

dl-translate v0.2.5

Correct name of citation.cff file

dl-translate v0.2.4

Update readme to link to latest zenodo DOI

dl-translate v0.2.3

Add citations.cff and update readme with badges

dl-translate v0.2.2.post1

Enable Zenodo Archiving

This will allow this repo to obtain a DOI. No changes to this repo was made

dl-translate v0.2.2

Change languages available in dlt.lang

Changed

• Docs: Available languages now include "Khmer" (which maps to central khmer)

Fixed

• dlt.lang will now have all the languages corresponding to m2m100 instead of mbart50

dl-translate v0.2.1

Fix dlt.TranslationModel.load_obj

Added

• New tests for saving and loading.

Fixed

• dlt.TranslationModel.load_obj: Will now work without having to explicitly give the model family.

dl-translate v0.2.0

Add m2m100 as the new default model to support 100 languages

Added

• dlt.lang.m2m100 module: Now has variables for over 100 languages, also auto-complete ready. Example: dlt.lang.m2m100.ENGLISH.
• dlt.utils.available_languages, dlt.utils.available_codes: Now supports argument "m2m100"
• Available languages for each model family
• Script and template to generate available languages

Changed

• [BREAKING] dlt.lang.TranslationModel: A new model parameter called model_family in the initialization function. Either "mbart50" or "m2m100". By default, it will be inferred based on model_or_path. Needs to be explicitly set if model_or_path is a path.
• [BREAKING] Default model changed to m2m100
• Docs and readme about mbart50 were reframed to take into account the new model
• dlt.TranslationModel.translate: Improved docstring to be more general.
• Tests pertaining to m2m100
• scripts/generate_langs.py: Renamed, mechanism now changed to loading from json files
• docs/index.md: Expand the "Usage" and "Advanced" sections
• README.md: Add acknowledgement about m2m100, significantly trim "Advanced" section, make "Usage" more concise

Fixed

• dlt.TranslationModel.available_codes() was returning the languages instead of the codes. It will now correctly return the code.

Removed

• Output type hints for TranslationModel.get_transformers_model and TranslationModel.get_tokenizer
• [BREAKING] dlt.TranslationModel.bart_model and dlt.TranslationModel.tokenizer are no longer available to be used directly. Please use dlt.TranslationModel.get_transformers_model and dlt.TranslationModel.get_tokenizer instead.