Hosting a OpenNMT-tf model on OpenNMT-py server

Support
1

Hi, I want to host a OpenNMT-tf model on the OpenNMT-py server. I converted the model with the ct2 converter and obtained two files: model.bin and shared_vocabulary.txt.
Is there an example of the config.json for non-Pytorch models?
Or a tutorial on how to run non-Pytorch ct2 models on OpenNMT-py server?

1 Like
2

Once it’s CT2 form, it doesn’t matter if it was from TF, Marian, Huggingface. The settings will be something like

{
    "models_root": "./available_models",
    "models": [{
            "id": 100,
            "ct2_model" : "model_name/",
            "model": "model_name/",
            "device": "cuda",
            "name": "XX-XX",
            "timeout": 30,
            "on_timeout": "unload",
            "load": false,
            "opt": {
                "gpu": 1,
                "max_length":400,
                "min_length":1,
                "tgt_prefix": true,
                "beam_size": 5,
                "batch_size":64
            }
        }
}
2 Likes
3

Thanks for the example. It was helpful.