Predict score with saved_model

Support
1

Hello,

I’ve been searching in the doc and the forum, but I can’t seem to find a way to get the equivalent of the output from infer with --predictions_file when using a saved_model with this script:
https://github.com/OpenNMT/OpenNMT-tf/blob/master/examples/serving/python/ende_client.py

I’m interested in both the predict score and having n_beam >1.

2

Hi,

  • You can find the scores in outputs["log_probs"]
  • The example retrieves the best predictions tokens[0] but you can also look for other prediction tokens[i]. If there are no other predictions, you might need to re-export the model with n_best > 1.
1 Like