Completely new to NMT


#1

Hey there,
like I said in the topic I´m completely new to NMT/OpenNMT and I hope to get some advices here.
I started with this tutorial:

I´m done and I´ve tested the quality of the translator, but it´s not very good yet. Since I have no idea how to continue I hope you can give me some clue how to go on from now? I´ve just found some more tutorials for advanced programmers. Maybe there are some tutorials for newbies, too?
Also I didn´t unterstand a lot in this tutorial. Can you tell me some source from where to learn about this from zero?

Thanks a lot


(Guillaume Klein) #2

Hi,

First I suggest you to read the following material to understand what NMT is and what are the limitations:

If you want to get serious about NMT, it is important to learn how the model works in order to understand its visible behavior (i.e. good or bad translation).

Regarding the tutorial applied on the WMT15 dataset, I identified 2 key points for improvements:

  1. Use a larger model
  2. Apply a subword tokenization

For 1., the tutorial used the default settings which define a relatively small neural network. Usually, people train a larger one with at least these training options:

th train.lua [...] -encoder_type brnn -layers 4 -rnn_size 800

You should also consider training for more epochs than the default (see the training options).

For 2., search online and in the documentation for the BPE tokenization. It’s a technique used to split words which increases the model performance. It is also critical to apply the same tokenization on the data you used for training the system.

I think you have already a lot to lean from the above. Good luck!