Calculate MRR and Recall

I would like to calculate MRR and Recall of the performance of my model.
To calculate those, I need not only 1 prediction per input, but multiple predictions, sorted by score.
Should I use n_best>1 to get those?
Is there anyway to export those to a file I can use to calculate?

Duplicate of Github Issue, discussion happening here: