Predict score with saved_model


I’ve been searching in the doc and the forum, but I can’t seem to find a way to get the equivalent of the output from infer with --predictions_file when using a saved_model with this script:

I’m interested in both the predict score and having n_beam >1.


  • You can find the scores in outputs["log_probs"]
  • The example retrieves the best predictions tokens[0] but you can also look for other prediction tokens[i]. If there are no other predictions, you might need to re-export the model with n_best > 1.
1 Like