Finding perplexity score for existing model

(Alexander Chebykin) #1

I’ve trained quite a few models with OpenNMT. In the process I’ve used BLEU as validation metric calculated after each epoch. Now I’d like to get the perplexity values for these existing models.
However, I can’t seem to find how to do it. There is a scorer, but it doesn’t allow to measure perplexity.
So, could anyone please point me to such an instrument if it exists, or to a place in code which I can modify?

(Guillaume Klein) #2


You can run translate.lua on your validation data and provide both -src and -tgt options. At the end of the translation look for GOLD PPL in the logs.

(Alexander Chebykin) #3

Thanks for the info and for the swift reply.