Adding perplexity or negative log-likelihood to CTranslate

What would be required to add to CTranslate the prediction score that’s typically output by OpenNMT when run from the translate.lua script?

Not much.

Would it be just part of the API or also the standard output? If the latter, what should be the format?

1 Like

It would be helpful if there could be a parameter that we could pass that would give the output in the format provided by translate.lua, which is like:

[-4.37] How can I help you today?

where the score is in the square brackets.
I’d be happy to make the change on a fork and submit a pull request if someone can point me in the right direction.

It’s about passing the highest score from include/onmt/Translator.hxx to cli/ It is either end_score[b] or max_score[b]:

It could be done by extending the TranslationResult class.

Then you may need to call another Translator's method from to access a TranslationResult object.

1 Like