OpenNMT Forum

Export model with inference config

I would like to export several models from a given checkpoint in order to run experiments.

For example, a default one using traditional beam search, a noisy beam one, a sampling one etc.
Since I am only interested in the export operation, training configurations are irrelevant. I only provide those parameters, the vocabulary files and model directory like so:

model_dir: /work/model

data:
  source_words_vocabulary: /work/shared.bpe
  target_words_vocabulary: /work/shared.bpe

params:
  decoding_subword_token: ▁
  decoding_noise:
    - dropout: 0.1
    - replacement: [0.1, <unk> ]
    - permutation: 3

I use this conf file and my checkpoint to export a model, then call that model to infer. I am using ONMT as a library if it makes any difference. However, when inspecting the translations I don’t see any permutations or noising, in fact it appears that vanilla beam search ran instead. Is this a known issue, or am I making a mistake somewhere?

Hi,

Can you give more details on how you export the models? Did you also try running inference with the checkpoint and not the exported model?

Hello,

here is the snippet for the export call:

config = load_config([config_path])
model = load_model(config["model_dir"])
runner = Runner(model, config, auto_config=auto_config)
return runner.export(checkpoint_path=checkpoint_path, export_dir_base=export_dir_base).decode("utf-8")

I will now try inference from the checkpoint and let you know if that works!

So even using to onmt-main infer tool and a checkpoint rather than an exported model has the same effect. The output seems perfect even when using dropout: 0.9 in the yml file.

Here is the new inference snippet:

onmt-main infer \
       --config /config_noisy_beam.yml \
       --features_file /test.tok \
       --predictions_file /test.translated \
       --checkpoint_path /best_checkpoint/model.ckpt-1120000

The yml file is as in the previous message but with - dropout: 0.9

What OpenNMT-tf version are you using?

onmt-main -v

OpenNMT-tf 1.17.1

Oh, you need to update to latest. The noise feature was added in 1.23.0.

Ah I see. Thanks for your help!