Fix_word_vecs option stored in the model?

[Etienne38 (Etienne Monneret)]

I just added a note in my previous post here:

Early stopping : a fake solution?

Since de beginning of this year I'm discovering NMT, with the practice of OpenNMT. Quickly, I was puzzled by the strange behaviour of the training. The scenario is this one: - in the first 2 ou 3 epochs, I get a very interesting model, already doing quite good draft translations. It lets me hope for a wonderful evolution in next epochs.. but, - for several epochs, the model is very slowly improving. It lets me think it will take a lot of time to handle sentences in detail, and produce good trans…

In fact, it seems that, in the second part of my test, the embeddings are not really changing. I certainly have inverted 2 files.

Are the fix_word_vecs options stored in the model, thus restored when no more defined on the command line ?

[guillaumekln (Guillaume Klein)]

You are right, embeddings remain fixed after a retraining. This does not seem like a good behavior (unless you are using -continue of course). I will look into it.

[Etienne38 (Etienne Monneret)]

Perhaps these 2 fix_word_vecs options may be defined with 0/1 explicit values on the command line, rather than set/omitted with implicit values when omitted.

[guillaumekln (Guillaume Klein)]

https://github.com/OpenNMT/OpenNMT/commit/4efad84a7ad8c1700b6617a531aa0369247dd98b

When -continue is not used, -fix_word_vecs_enc and -fix_word_vecs_dec are correctly applied.

[guillaumekln (Guillaume Klein)]

Perhaps these 2 fix_word_vecs options may be defined with 0/1 explicit values on the command line, rather than set/omitted with implicit values when omitted.

FYI, this approach is now on master.