We performed some interesting multilingual training with OpenNMT-py.
This first experiment includes 5 languages, 20 pairs (English, French, Italian, German, Spanish).
We used a “medium transformer” (6x768) and here are some example results:
Newstest13 (DE to FR):
Single pair model: 30.55
Google T: 28.25
Multilingual model: 30.40
Newstest19 (DE to FR):
Single pair model: 35.21
Google T: 32.18
Multilingual model: 34.60
Pivot with SOA engines DE-EN/EN-FR: 34.12
Newstest14 (FR to EN):
Single pair model: 41.3
Google T: 38.79
Multilingual model: 39.0
It performs quite well, it’s always above the pivot via EN when EN is not in the pair.
Next step is to try with more languages.