Simple OpenNMT-py REST server


(Pltrdy) #1

OpenNMT-py Translation Server – Update [2018-09-13]

I made a simple REST server to use OpenNMT-py models.

I] How it works?

The idea behind the translation server is to make a entry point for translation with multiple models.
A configuration file (default: ./available_models/conf.json) will describes such models, with the path of the checkpoint files along with other parameters.


  • models_root: (opt) folder containing model checkpoints, [default: ./available_models]
  • models: list of objects such as :
    • id: (opt) manually assign an id (int), [default: value from counter]
    • name: (opt) assing a name (str)
    • model: (required) path to checkpoint file i.e. *.pt
    • timeout: (opt) interval (seconds) before unloading, reset at each translation using the model
    • load: (opt) whether to load the model at start [default: False]
    • on_timeout: (opt) what to do on timeout: unload removes everything; to_cpu transfer the model to RAM (from GPU memory) this is faster to reload but takes RAM.
    • opt: (opt) dict of translation options (see ./
    • tokenizer: (opt) set tokenizer options (if any), such as:
      • type: (str) value in {sentencepiece, pyonmttok}.
      • model: (str) path to tokenizer model


    "models_root": "./available_models",
    "models": [
            "id": 100,
            "model": "",
            "timeout": 600,
            "on_timeout": "to_cpu",
            "load": true,
            "opt": {
                "gpu": 0,
                "beam_size": 5
            "tokenizer": {
                "type": "sentencepiece",
                "model": "wmtenfr.model"
            "model": "",
            "timeout": -1, 
            "on_timeout": "unload",
            "model_root": "../other_models",
            "opt": {
                "batch_size": 1,
                "beam_size": 10

II] Start the server

0) Get the code

The translation server has been merged into onmt-py master branch.
Keep in line with master for last fix / improvements.

1) Install flask

pip install flask

2) Put some models

mkdir available_models/
cp $path_to_my_model available_models

3) start the server

export IP=""
export PORT=5000
export URL_ROOT="/translator"
export CONFIG="./available_models/conf.json"

# NOTE that these parameters are optionnal
# here, we explicitely set to default values
python --ip $IP --port $PORT --url_root $URL_ROOT --config $CONFIG

III] API Usage

0) set the hostname

export HOST=""

1) list models

curl http://$HOST:$PORT$URL_ROOT/models

Result (example):

  "available": [
  "loaded": []

2) Translate

(this example involves subwords)

curl -i -X POST -H "Content-Type: application/json" \
    -d '[{"src": "this is a test for model 0", "id": 0}]' \


  "model_id": 0,
  "result": "\u2581die \u2581Formen kant en \u2581( K \u00f6r ner ) \u2581des \u2581Stahl g u\u00df form .\n",
  "status": "ok",
  "time": {
    "total": 8.510261535644531,
    "translation": 8.509992599487305,
    "writing_src": 0.0002689361572265625

Can I use trained modules to creat a translation server?
OpenNMT translate string as an input
(stano) #2

Am trying to run the server but I get an error message something like this

AttributeError: ‘dict’ object has no attribute ‘log_file’

Please help

(Pltrdy) #3

Hey @starno you should open an issue (tag me in) on Github with the full error trace along with your the commit hash you’re on.


(stano) #5

@pltrdy thanks i think i got what i wanted at that time.
currently i am facing a new challenge when it comes to rendering my translation to a browser… but thanks to @mzeid i got this link. Hope it workks out