Simple OpenNMT-py REST server

opennmt-py

(Pltrdy) #1

OpenNMT-py Translation Server – Update [2018-09-13]

I made a simple REST server to use OpenNMT-py models.

I] How it works?


The idea behind the translation server is to make a entry point for translation with multiple models.
A configuration file (default: ./available_models/conf.json) will describes such models, with the path of the checkpoint files along with other parameters.

Configuration:

  • models_root: (opt) folder containing model checkpoints, [default: ./available_models]
  • models: list of objects such as :
    • id: (opt) manually assign an id (int), [default: value from counter]
    • name: (opt) assing a name (str)
    • model: (required) path to checkpoint file i.e. *.pt
    • timeout: (opt) interval (seconds) before unloading, reset at each translation using the model
    • load: (opt) whether to load the model at start [default: False]
    • on_timeout: (opt) what to do on timeout: unload removes everything; to_cpu transfer the model to RAM (from GPU memory) this is faster to reload but takes RAM.
    • opt: (opt) dict of translation options (see ./translate.py)
    • tokenizer: (opt) set tokenizer options (if any), such as:
      • type: (str) value in {sentencepiece, pyonmttok}.
      • model: (str) path to tokenizer model

Example

{
    "models_root": "./available_models",
    "models": [
        {   
            "id": 100,
            "model": "model_0.pt",
            "timeout": 600,
            "on_timeout": "to_cpu",
            "load": true,
            "opt": {
                "gpu": 0,
                "beam_size": 5
            },  
            "tokenizer": {
                "type": "sentencepiece",
                "model": "wmtenfr.model"
            }   
        },{ 
            "model": "model_0.light.pt",
            "timeout": -1, 
            "on_timeout": "unload",
            "model_root": "../other_models",
            "opt": {
                "batch_size": 1,
                "beam_size": 10
            }   
        }   
    ]   
}

II] Start the server


0) Get the code

The translation server has been merged into onmt-py master branch.
Keep in line with master for last fix / improvements.

1) Install flask

pip install flask

2) Put some models

mkdir available_models/
cp $path_to_my_model available_models

3) start the server

export IP="0.0.0.0"
export PORT=5000
export URL_ROOT="/translator"
export CONFIG="./available_models/conf.json"

# NOTE that these parameters are optionnal
# here, we explicitely set to default values
python server.py --ip $IP --port $PORT --url_root $URL_ROOT --config $CONFIG

III] API Usage


0) set the hostname

export HOST="127.0.0.1"

1) list models

curl http://$HOST:$PORT$URL_ROOT/models

Result (example):

  "available": [
    "wmt14.en-de_acc_69.22_ppl_4.33_e9.pt",
    "wmt14.en-de_acc_69.22_ppl_4.33_e9.light.pt"
  ],
  "loaded": []
}

2) Translate

(this example involves subwords)

curl -i -X POST -H "Content-Type: application/json" \
    -d '[{"src": "this is a test for model 0", "id": 0}]' \
    http://$HOST:$PORT$URL_ROOT/translate

Result:

{
  "model_id": 0,
  "result": "\u2581die \u2581Formen kant en \u2581( K \u00f6r ner ) \u2581des \u2581Stahl g u\u00df form .\n",
  "status": "ok",
  "time": {
    "total": 8.510261535644531,
    "translation": 8.509992599487305,
    "writing_src": 0.0002689361572265625
  }
}

Can I use trained modules to creat a translation server?
OpenNMT translate string as an input
(stano) #2

Am trying to run the server but I get an error message something like this

AttributeError: ‘dict’ object has no attribute ‘log_file’

Please help


(Pltrdy) #3

Hey @starno you should open an issue (tag me in) on Github with the full error trace along with your the commit hash you’re on.

Paul


(stano) #5

@pltrdy thanks i think i got what i wanted at that time.
currently i am facing a new challenge when it comes to rendering my translation to a browser… but thanks to @mzeid i got this link. Hope it workks out