About Opennmt-tf 1.25.3 output of exported models

duzeyao · February 12, 2020, 4:04am

Hi, I am using Opennmt-tf 1.25.3 to do a translate task.

The model (a standard transformer) is trained. Exported model serving is great. Thanks for the awesome project.

Now I want to do a minor (but seems not that easy) modification. I want the exported model to provide the output of last encoder layer, too. What are changes of code I should make to to achieve this?

Thank you very much!

guillaumekln · February 12, 2020, 8:47am

Hi,

Here is the output dictionary that is returned by the exported model:

github.com

OpenNMT/OpenNMT-tf/blob/v1.25.3/opennmt/models/sequence_to_sequence.py#L306


if decoding_noise:
  sampled_length -= 1  # Ignore </s>
  target_tokens, sampled_length = _add_noise(
      target_tokens,
      sampled_length,
      decoding_noise,
      params.get("decoding_subword_token", "￭"))
  sampled_length += 1
  alignment = None  # Invalidate alignments.


predictions = {
    "tokens": target_tokens,
    "length": sampled_length,
    "log_probs": log_probs
}
if alignment is not None:
  predictions["alignment"] = alignment


num_hypotheses = params.get("num_hypotheses", 1)
if num_hypotheses > 0:
  if num_hypotheses > beam_width:

You can add other tensors in it. In particular, you can find encoder_outputs higher up in the same function.

Once the code is updated, you should re-export your model.

duzeyao · February 12, 2020, 9:11am

Thank you! I solved the problem with your help, really appreciated.