Exposure bias during training


(Shruti P) #1

Hi,

Is exposure bias handled in OpenNMT? If yes, how?

Is the model exposed to its own predictions during training?


(jean.senellart) #2

Hello @Shruti, are you refering to scheduled sampling?

If yes, it is implemented in this PR:

For MT it does not gives any good result, but @homink just validated that it works well for ASR training, so I will merge the PR soon.


(Shruti P) #3

Hi @jean.senellart and @homink,

Yes, this is what I was trying to ask. Thanks a lot for pointing me to this.

Regarding scheduled sampling for ASR, have you worked well for you with the Pyramidal encoder?


(jean.senellart) #4

yes it does. we are still doing more training, but we will publish some intermediate results soon.