OpenNMT-py Tutorial

Tutorials
1

Hi Colleagues!

The notebooks of this OpenNMT-py tutorial are now updated for v. 3. They mainly cover: data processing, training an NMT model with OpenNMT-py, translation, and evaluation. There are also references to some advanced topics.

I hope this helps.

Kind regards,
Yasmin

9 Likes
2
3

Hi Yasmin,
Another great tutorial that you’ve prepared. Thank you for that – I will surely use it in my classes.
I am trying it out on Kaggle and on our own multi-GPU machine. The MT-Preparation works perfect in Kaggle, but on my own machine I get an error with the filter.py script:
ValueError: Specified \n as separator or delimiter. This forces the python engine which does not accept a line terminator. Hence it is not allowed to use the line terminator as separator. Any idea what could cause this?

4

Apparently a pandas version issue – if I downgrade my pandas to 1.3.3 it worked.
pip install pandas==1.3.3

5

Thanks, Vincent!

I am aware of this issue, but I could not replicate it either on my Mac or Google Colab. One solution is to replace sep="\n" with sep="\0" as suggested here.

Kind regards,
Yasmin

6

Hi Yasmin,

Sorry for the beginner question but is it possible to fine tune & translate OpenNMT models without having to clone the repo?

I’ve read the tutorial and saw that you used commands like

!onmt_train -config config.yaml
!onmt_translate -model models/model.fren_step_3000.pt -src UN.en-fr.fr-filtered.fr.subword.test -output UN.en.translated -gpu 0 -min_length 1

But my shell would not recognise onmt_train and onmt_translate commands?

7 
pip3 install OpenNMT-py
1 Like