Ctranslate2 - evaluation attention

[SamuelLacombe]

Hello,

I have a use case where I would need the option “return_attention” from the function “score_batch”, I’m pretty sure in the backend of “score_batch”, “batch_translation” is called? If so, how hard would it be to have the option of the attentions?

My use case is this:

1. I will train a model with just my data
2. Translate my text with a good translator such as DeepL (doesn’t offer custom training)
3. Use the score_batch to check if there are any tokens which my model is sure (with the predict score) it’s “wrong”
4. If the attention would be present, I could grab the source tokens and translate it with my model and then replace the token/word in the original DeepL Translation.

DeepL is just an example… but I guess you figure out the concept.

Best regards,
Samuel

[guillaumekln (Guillaume Klein)]

Hi,

“score_batch” does not run a batch translation, but we can consider returning attention vectors in a future version.

However, you should remember that Transformer attention may not work as alignment by default:

How to find that particular word from original text translated to particular word(s)?

The translation method can return the target->source attention weights with the flag return_attention. For each target token, you could select the highest attention weight as the aligned source token. However, Transformer models have multiple attention heads so there is no guarantee the returned attention can be interpreted as alignments. Most training frameworks have an option to supervise the training of one or several attention heads. For example: OpenNMT-py: https://opennmt.net/OpenNMT-p…

[SamuelLacombe]

Thank you Guillaume, I’m aware that attentions are not intended for alignment and can be completly off. I’m actually experiencing with it to find the best way to use the attentions to create a really reliable alignment. I believe taking the highest is not enough. I’m trying a combination of horizontally and vertically at the same time.