Wrong word UNK replacement

Hi! I trained translation model to grammatical error correction in English. Source is the sentence with error (or even correct sentence) and target is corrected sentence. I trained a model without using bpe. The problem is that, when I make translations with unk word replacement, the unknown word is replaced by the wrong word (specifically, is the word or punctuation sign which follows the unknown word).
hyp: These days, my idea of a safe place to keep money is a Full full o’Nuts coffee can.
trg: These days, my idea of a safe place to keep money is a Chock full o’Nuts coffee can.

hyp: Carved, stepping, – you name it, he did it.
trg: Carving, stepping, cuttys – you name it, he did it.

The script I use to train and translate is the following:

th tools/tokenize.lua
-nparallel 8
-joiner_annotate true
-segment_numbers true < $set > ${set}.tok

th preprocess.lua
-train_tgt training.tok.${trg_}
-valid_src dev.tok.${src_}
-valid_tgt dev.tok.${trg_}
-save_data preprocessed
-sort true
-report_progress_every 100000 \

th train.lua -data preprocessed-train.t7
-rnn_size 128
-encoder_type rnn
-rnn_type LSTM
-end_epoch 60
-max_batch_size 50
-save_model models/
-layers 2
-dropout 0.3
-optim adam
-learning_rate 0.0002
-learning_rate_decay 1.0
-src_word_vec_size 128
-tgt_word_vec_size 128
-gpuid 1

th translate.lua
-src ${test_}
-detokenize_output true
-tok_tgt_joiner_annotate true
-output models/pred.tok.${f}.txt
-model /models/${f}
-tok_tgt_case_feature true
-gpuid 1

Have you any idea what I’m doing wrong?
Thanks in advance


I remember there were some issues with the alignment being offset by one. This has been fixed in OpenNMT-tf for example.

Did you try with BPE also? It should at least mitigate this issue.

using bpe there are no unknown word but in my case, bpe creates strange words. Is there some possibility to make translations in opennmt-tf using model trained in lua?


You may need more data and/or a bigger model.

No, this is not possible.

Ok, thanks.

hi @Sasanita
Did you figure out some solution to this. Actually I am facing this issue for Names of person

Hi @ajitesh3
Finally I use Opennmt-tf and it works fine.

@Sasanita Was it able to solve the proper noun(names of person) problem? Like subword BPE is creating new and weird words for original name? Any idea how to correct this thing in Opennmt py ? and what different you do in tensorflow?
Do you know anyway we can prevent translation of a word(names here), i.e I want to keep them as it is dont want them to translate