Exposure bias during training


Is exposure bias handled in OpenNMT? If yes, how?

Is the model exposed to its own predictions during training?

Hello @Shruti, are you refering to scheduled sampling?


If yes, it is implemented in this PR:

For MT it does not gives any good result, but @homink just validated that it works well for ASR training, so I will merge the PR soon.

Hi @jean.senellart and @homink,

Yes, this is what I was trying to ask. Thanks a lot for pointing me to this.

Regarding scheduled sampling for ASR, have you worked well for you with the Pyramidal encoder?

yes it does. we are still doing more training, but we will publish some intermediate results soon.

1 Like