Pinned Thread of Feature Requests (v0.6 is next release)
Modeling
Data sets / Models
Advanced
Extras
There are several applications where the user can provide implicit feedback about the quality of a translation (though they may not provide the ground truth translation) through their behavior E.g: accept or reject the translation, click or not click etc. Think of a webapp which uses a translation service as a backend.
This paper Learning to Search Better Than Your Teacher provides a way to start from some reference translation and improve it through exploration while providing localy optimality guarantees.
In essence it is middle ground between pure RL and fully supervised learning.
@srush I think it’ll be a good addition to the OpenNMT toolkit and we could use an RNN as the Oracle in this case. What do you feel?
There is another thread about some of this. I think we will get to it, but it is pretty low-pri unless someone takes an interest in implementing.
I think it’s an important problem since the language that we use is changing constantly. Getting human generated translations will be difficult at scale.
I can help implementing it. How do I create a proposal-what process do you follow?
Wabbit are you in our gitter channel? https://gitter.im/OpenNMT/openmt maybe a good place to discuss.
First thing is to figure out a general purpose way to add RL training to the core code, likely will have to discuss with @guillaumekln and others.
I think direct output the original unk word to the results is needed.
Can you be more specific?
in my old translate system(Moses), unknown words are copied verbatim to the output.
in openNMT, when I translate a sentence, often give the wrong unk word in the results,especially if the first vocabulary is unk.
for example:
src sentences:
bitterness fed on the man who had made the world laugh .
the bitterness is unk word, will got wrong result:
fed on on . 的 人
the bitterness has been lost in result.
@guillaumekln give me explain, but I think It is very important for a practical system.
It’s certainly due to the attn layer that makes a bad word alignment between the source and the translation. It’s one of the reasons why I asked for more explanations on its functioning, and the way to tune it, here:
Yup, I get it now. This is not a “simple” problem, we actually have to look into research for doing this better. We will try to find something reasonable to implement.
I have very limited human translated data and have been reading about RL. Would like to work on it ! Please suggest some good reading materials !
Also, is there a way to fine tune OpenNMT’s model ? To limit backpropagation while retraining?
The RL feature is quite significant for NLP nowadays. I wonder if anyone works on it now?