OpenNMT-py REST server ensemble decoding

Support
1

Hi all,

I am trying to use ensemble decoding using the OpenNMT-py REST server, but I am not sure how to pass the list of my models in config.json. My config file looks something like this:

"models_root": "./models/dev",
    "models": [
        {
            "id": 1,
            "models_root": "./models/dev",
            "model": "model1.pt",
            "load": true,
            "opt": {
                "batch_size": 1,
                "replace_unk": true,
                "verbose": true,
                "n_best": 5,
                "model": "model1.pt model2.pt"
            }
        }
    ]
}

How do I specify my list of models in config.json? When I pass a list to “model” I get an error as it expects a single model and when I pass a list to opt.model, nothing happens, and it just uses the single model passed to “model”.

Appreciate any help!

2

For anyone who’s wondering, after digging into the Ensemble PR (https://github.com/OpenNMT/OpenNMT-py/pull/732/files), I realized that the above config file should be updated as follows:

"models_root": "./models/dev",
    "models": [
        {
            "id": 1,
            "models_root": "./models/dev",
            "models": ["model1.pt", "model2.pt"]
            "load": true,
            "opt": {
                "batch_size": 1,
                "replace_unk": true,
                "verbose": true,
                "n_best": 5,
            }
        }
    ]
}

Essentially model names should be passed as a list to “models” not “model”.

1 Like