Inferencing with translate.py

[helloworld]

Hi all, currently I am trying to inference a model that I trained with opennmt-py using the big translate.py script. Anyone knows if I am able to replace the arguments in the command line with manual input arguments? Or any advice would be appreciated

[ymoslem (Yasmin Moslem)]

Hello!

Have you tried CTranslate2?

All the best,
Yasmin

[helloworld]

Hey

I have tried but my model was not trained with a transformer, therefore I think I am not able to used it? I may be wrong, can you advice?

[ymoslem (Yasmin Moslem)]

CTranslate2 works for Transformer models only. Could you please elaborate on what you are trying to achieve?

[helloworld]

Understood that because I did not use a transformer model I am not able to utilised CTRANSLATE2. Currently what I am trying to do achieve is that I want to use fastAPI as my service to get an inference to the model that I have trained.

[francoishernandez (François Hernandez)]

What you’re looking for is probably the REST server, server.py, not translate.py.

Simple OpenNMT-py REST server

OpenNMT-py Translation Server – Update [2018-09-13] I made a simple REST server to use OpenNMT-py models. I] How it works? The idea behind the translation server is to make a entry point for translation with multiple models. A configuration file (default: ./available_models/conf.json) will describes such models, with the path of the checkpoint files along with other parameters. Configuration: models_root: (opt) folder containing model checkpoints, [default: ./available_models] models: lis…

[helloworld]

Hey I have tried this, but faced some error with the conf.json file. Is it required to have all the keys in the conf.json file such as the tokenizer?