Export model with inference config

Support
1

I would like to export several models from a given checkpoint in order to run experiments.

For example, a default one using traditional beam search, a noisy beam one, a sampling one etc.
Since I am only interested in the export operation, training configurations are irrelevant. I only provide those parameters, the vocabulary files and model directory like so:

model_dir: /work/model

data:
  source_words_vocabulary: /work/shared.bpe
  target_words_vocabulary: /work/shared.bpe

params:
  decoding_subword_token: ▁
  decoding_noise:
    - dropout: 0.1
    - replacement: [0.1, <unk> ]
    - permutation: 3

I use this conf file and my checkpoint to export a model, then call that model to infer. I am using ONMT as a library if it makes any difference. However, when inspecting the translations I don’t see any permutations or noising, in fact it appears that vanilla beam search ran instead. Is this a known issue, or am I making a mistake somewhere?

2

Hi,

Can you give more details on how you export the models? Did you also try running inference with the checkpoint and not the exported model?

3

Hello,

here is the snippet for the export call:

config = load_config([config_path])
model = load_model(config["model_dir"])
runner = Runner(model, config, auto_config=auto_config)
return runner.export(checkpoint_path=checkpoint_path, export_dir_base=export_dir_base).decode("utf-8")

I will now try inference from the checkpoint and let you know if that works!

4

So even using to onmt-main infer tool and a checkpoint rather than an exported model has the same effect. The output seems perfect even when using dropout: 0.9 in the yml file.

Here is the new inference snippet:

onmt-main infer \
       --config /config_noisy_beam.yml \
       --features_file /test.tok \
       --predictions_file /test.translated \
       --checkpoint_path /best_checkpoint/model.ckpt-1120000

The yml file is as in the previous message but with - dropout: 0.9

5

What OpenNMT-tf version are you using?

onmt-main -v
6

OpenNMT-tf 1.17.1

7

Oh, you need to update to latest. The noise feature was added in 1.23.0.

8

Ah I see. Thanks for your help!