Visualize attention

valentinmace · August 26, 2019, 3:08pm

Hi,

Is there a simple way visualize attention at inference time ? (OpenNMT-tf)

Thanks in advance

guillaumekln · August 27, 2019, 9:17am

Hi,

Here is the function that prints the inference output:

github.com

OpenNMT/OpenNMT-tf/blob/v1.24.0/opennmt/utils/misc.py#L31-L60


def format_translation_output(sentence,
                              score=None,
                              token_level_scores=None,
                              attention=None,
                              alignment_type=None):
  """Formats a translation output with possibly scores, alignments, etc., e.g:


  1.123214 ||| Hello world ||| 0.30907777 0.030488174 ||| 0-0 1-1


  Args:
    sentence: The translation to output.
    score: If set, attach the score.
    token_level_scores: If set, attach the token level scores.
    attention: The attention vector.
    alignment_type: The type of alignments to format (can be: "hard").
  """
  if score is not None:
    sentence = "%f ||| %s" % (score, sentence)
  if token_level_scores is not None:
    scores_str = " ".join("%f" % s for s in token_level_scores)

This file has been truncated. show original

You could change it to also dump the attention values.

anderleich · August 27, 2021, 12:49pm

Hi,

How about the Pytorch version? Does it have a way to see attention values?