I’ve trained quite a few models with OpenNMT. In the process I’ve used BLEU as validation metric calculated after each epoch. Now I’d like to get the perplexity values for these existing models.
However, I can’t seem to find how to do it. There is a scorer, but it doesn’t allow to measure perplexity.
So, could anyone please point me to such an instrument if it exists, or to a place in code which I can modify?
You can run
translate.lua on your validation data and provide both
-tgt options. At the end of the translation look for
GOLD PPL in the logs.
Thanks for the info and for the swift reply.