I do not know what tokenization is enabled at this point. I’m kinda running OpenNMT-py as a black box, hoping it’ll get some baseline results. I didn’t take it through the perl tokenization script - perhaps that’s one huge mistake.
Model options - I just used the usual source target, 30 epochs, 5 layers (enc + dec).
How do I enable this. In the source I can find an MLP and some PositionalEncoding - reading code is turning out to be cumbersome - is there some documentation?