Exposure bias during training

Research
1

Hi,

Is exposure bias handled in OpenNMT? If yes, how?

Is the model exposed to its own predictions during training?

2

Hello @Shruti, are you refering to scheduled sampling?

http://papers.nips.cc/paper/5956-scheduled-sampling-for-sequence-prediction-with-recurrent-neural-networks.pdf

If yes, it is implemented in this PR:

For MT it does not gives any good result, but @homink just validated that it works well for ASR training, so I will merge the PR soon.

3

Hi @jean.senellart and @homink,

Yes, this is what I was trying to ask. Thanks a lot for pointing me to this.

Regarding scheduled sampling for ASR, have you worked well for you with the Pyramidal encoder?

4

yes it does. we are still doing more training, but we will publish some intermediate results soon.

1 Like