Exposure bias during training

(Shruti P) #1


Is exposure bias handled in OpenNMT? If yes, how?

Is the model exposed to its own predictions during training?

(jean.senellart) #2

Hello @Shruti, are you refering to scheduled sampling?

If yes, it is implemented in this PR:

For MT it does not gives any good result, but @homink just validated that it works well for ASR training, so I will merge the PR soon.

(Shruti P) #3

Hi @jean.senellart and @homink,

Yes, this is what I was trying to ask. Thanks a lot for pointing me to this.

Regarding scheduled sampling for ASR, have you worked well for you with the Pyramidal encoder?

(jean.senellart) #4

yes it does. we are still doing more training, but we will publish some intermediate results soon.