Evaluate translation hypotheses

Support
1

Hello everyone,

Please, I want to know if there is any tool or module to scor translation hypotheses withe the trained neural network.
The idea is to give a source sentence (e) and a translation (f), the system must evaluate the quality of this translation a give me a score representing the quality of the f according to e (P(f|e)).

Thanks a lot.
best regards,
Ameur

2

Hello,

The translation script already outputs this information in PRED SCORE.

See this thread to know what is precisely reported:

1 Like
3

at the decoding step yes I know it. What I want is to score a translation that I give, not the one outputted by the decoder.

4

Oh, I read it too quickly. So you should set the -tgt option with the file containing your translations and look for GOLD SCORE in the translation logs.

1 Like
5

Thank you very much the -tgt option is what I wanted.

[10/20/17 16:53:12 INFO] SENT 3: القرار ألف وخمسمئة وسبع ألفان وثلاث	(source)
[10/20/17 16:53:12 INFO] GOLD 3: resolution one thousand five hundred and seven two thousand and three (reference)	
[10/20/17 16:53:12 INFO] GOLD SCORE: -14.72	(reference score)
[10/20/17 16:53:12 INFO] PRED 3: resolution two thousand and five two thousand and seven (output)
[10/20/17 16:53:12 INFO] PRED SCORE: -7.04	(output score)

I want to know now if it is possible to use the option “-save_attention” to save the alignment between the source and the reference translation (gold) ?

thank you in advance
Ameur

6

This is not possible at this time and requires some code changes.

1 Like
7

Hi,
Is there a way to output the scores to a file?
Thanks