Adding perplexity or negative log-likelihood to CTranslate

ctranslate

(Devin G Bost) #1

What would be required to add to CTranslate the prediction score that’s typically output by OpenNMT when run from the translate.lua script?


(Guillaume Klein) #2

Not much.

Would it be just part of the API or also the standard output? If the latter, what should be the format?


(Devin G Bost) #3

It would be helpful if there could be a parameter that we could pass that would give the output in the format provided by translate.lua, which is like:

[-4.37] How can I help you today?

where the score is in the square brackets.
I’d be happy to make the change on a fork and submit a pull request if someone can point me in the right direction.


(Guillaume Klein) #4

It’s about passing the highest score from include/onmt/Translator.hxx to cli/translate.cc. It is either end_score[b] or max_score[b]:

It could be done by extending the TranslationResult class.

Then you may need to call another Translator's method from translate.cc to access a TranslationResult object.