OpenNMT-py Translation Server – Update [2018-09-13]
I made a simple REST server to use OpenNMT-py models.
I] How it works?
The idea behind the translation server is to make a entry point for translation with multiple models.
A configuration file (default: ./available_models/conf.json) will describes such models, with the path of the checkpoint files along with other parameters.
Configuration:
-
models_root: (opt) folder containing model checkpoints, [default:./available_models] -
models: list of objects such as :-
id: (opt) manually assign an id (int), [default: value from counter] -
name: (opt) assing a name (str) -
model: (required) path to checkpoint file i.e.*.pt -
timeout: (opt) interval (seconds) before unloading, reset at each translation using the model -
load: (opt) whether to load the model at start [default: False] -
on_timeout: (opt) what to do on timeout:unloadremoves everything;to_cputransfer the model to RAM (from GPU memory) this is faster to reload but takes RAM. -
opt: (opt) dict of translation options (see./translate.py) -
tokenizer: (opt) set tokenizer options (if any), such as:-
type: (str) value in{sentencepiece, pyonmttok}. -
model: (str) path to tokenizer model
-
-
Example
{
"models_root": "./available_models",
"models": [
{
"id": 100,
"model": "model_0.pt",
"timeout": 600,
"on_timeout": "to_cpu",
"load": true,
"opt": {
"gpu": 0,
"beam_size": 5
},
"tokenizer": {
"type": "sentencepiece",
"model": "wmtenfr.model"
}
},{
"model": "model_0.light.pt",
"timeout": -1,
"on_timeout": "unload",
"model_root": "../other_models",
"opt": {
"batch_size": 1,
"beam_size": 10
}
}
]
}
II] Start the server
0) Get the code
The translation server has been merged into onmt-py master branch.
Keep in line with master for last fix / improvements.
1) Install flask
pip install flask
2) Put some models
mkdir available_models/
cp $path_to_my_model available_models
3) start the server
export IP="0.0.0.0"
export PORT=5000
export URL_ROOT="/translator"
export CONFIG="./available_models/conf.json"
# NOTE that these parameters are optionnal
# here, we explicitely set to default values
python server.py --ip $IP --port $PORT --url_root $URL_ROOT --config $CONFIG
III] API Usage
0) set the hostname
export HOST="127.0.0.1"
1) list models
curl http://$HOST:$PORT$URL_ROOT/models
Result (example):
"available": [
"wmt14.en-de_acc_69.22_ppl_4.33_e9.pt",
"wmt14.en-de_acc_69.22_ppl_4.33_e9.light.pt"
],
"loaded": []
}
2) Translate
(this example involves subwords)
curl -i -X POST -H "Content-Type: application/json" \
-d '[{"src": "this is a test for model 0", "id": 0}]' \
http://$HOST:$PORT$URL_ROOT/translate
Result:
{
"model_id": 0,
"result": "\u2581die \u2581Formen kant en \u2581( K \u00f6r ner ) \u2581des \u2581Stahl g u\u00df form .\n",
"status": "ok",
"time": {
"total": 8.510261535644531,
"translation": 8.509992599487305,
"writing_src": 0.0002689361572265625
}
}