OpenNMT Forum

Issues running the OpenNMT-py REST server

Hi ,

git pull and restarting the server.py did not help. I changed the parameters in conf.json as follows:
conf.json file:

{
“models_root”: “./available_models”,
“models”: [
{
“id”: 1,
“model”: “iwslt-brnn2.s131_acc_62.71_ppl_7.74_e20.pt”,
“timeout”: 600,
“on_timeout”: “to_cpu”,
“load”: true,
“opt”: {
“gpu”: -1,
“beam_size”: 5
},
“tokenizer”: {
“type”: “pyonmttok”,
“mode”: “str”,
“params”: {
“bpe_model_path”:"",
“vocabulary_path”:"",
“vocabulary_threshold”:0,
“sp_model_path”:"",
“sp_nbest_size”:0,
“sp_alpha”:0.1,
“joiner”:“■”,
“joiner_annotate”:false,
“joiner_new”:false,
“spacer_annotate”:false,
“spacer_new”:false,
“case_feature”:false,
“case_markup”:false,
“no_substitution”:false,
“preserve_placeholders”:false,
“preserve_segmented_tokens”:false,
“segment_case”:false,
“segment_numbers”:false,
“segment_alphabet_change”:false,
“segment_alphabet”:[]
}
}
},{
“model”: “model_0.light.pt”,
“timeout”: -1,
“on_timeout”: “unload”,
“model_root”: “…/other_models”,
“opt”: {
“batch_size”: 1,
“beam_size”: 10
}
}
]
}

the above conf parameters I got from the below link :

With the above changes I am getting the following error message:

administrator@:~/OpenNMT/OpenNMT-py$ python3 server.py --ip “0.0.0.0” --port “7785” --url_root “/translator” --config "./available_models/conf.json"
Pre-loading model 1
[2019-06-18 12:10:12,621 INFO] Loading model 1
[2019-06-18 12:10:19,179 INFO] Loading tokenizer
Traceback (most recent call last):
** File “server.py”, line 123, in **
** debug=args.debug)**
** File “server.py”, line 24, in start**
** translation_server.start(config_file)**
** File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 102, in start**
** self.preload_model(opt, model_id=model_id, kwargs)
** File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 140, in preload_model**
** model = ServerModel(opt, model_id, model_kwargs)
** File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 227, in init**
** self.load()**
** File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 319, in load**
** tokenizer = pyonmttok.Tokenizer(mode, tokenizer_params)
RuntimeError: basic_filebuf::underflow error reading the file: iostream error
administrator@:~/OpenNMT/OpenNMT-py$

Please assist me in resolving this issues as early as possible ?

Thank You,
Kishor.

Hi ,

git pull and restarting the server.py did not help. I changed the parameters in conf.json as follows:
conf.json file:

{
“models_root”: “./available_models”,
“models”: [
{
“id”: 1,
“model”: “iwslt-brnn2.s131_acc_62.71_ppl_7.74_e20.pt”,
“timeout”: 600,
“on_timeout”: “to_cpu”,
“load”: true,
“opt”: {
“gpu”: -1,
“beam_size”: 5
},
“tokenizer”: {
“type”: “pyonmttok”,
“mode”: “str”,
“params”: {
“bpe_model_path”:"",
“vocabulary_path”:"",
“vocabulary_threshold”:0,
“sp_model_path”:"",
“sp_nbest_size”:0,
“sp_alpha”:0.1,
“joiner”:“■”,
“joiner_annotate”:false,
“joiner_new”:false,
“spacer_annotate”:false,
“spacer_new”:false,
“case_feature”:false,
“case_markup”:false,
“no_substitution”:false,
“preserve_placeholders”:false,
“preserve_segmented_tokens”:false,
“segment_case”:false,
“segment_numbers”:false,
“segment_alphabet_change”:false,
“segment_alphabet”:[]
}
}
},{
“model”: “model_0.light.pt”,
“timeout”: -1,
“on_timeout”: “unload”,
“model_root”: “…/other_models”,
“opt”: {
“batch_size”: 1,
“beam_size”: 10
}
}
]
}

the above conf parameters I got from the below link :

With the above changes I am getting the following error message:

administrator@:~/OpenNMT/OpenNMT-py$ python3 server.py --ip “0.0.0.0” --port “7785” --url_root “/translator” --config "./available_models/conf.json"
Pre-loading model 1
[2019-06-18 12:10:12,621 INFO] Loading model 1
[2019-06-18 12:10:19,179 INFO] Loading tokenizer
Traceback (most recent call last):
** File “server.py”, line 123, in **
** debug=args.debug)**
** File “server.py”, line 24, in start**
** translation_server.start(config_file)**
** File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 102, in start**
** self.preload_model(opt, model_id=model_id, kwargs)
** File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 140, in preload_model**
** model = ServerModel(opt, model_id, model_kwargs)
** File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 227, in init**
** self.load()**
** File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 319, in load**
** tokenizer = pyonmttok.Tokenizer(mode, tokenizer_params)
RuntimeError: basic_filebuf::underflow error reading the file: iostream error
administrator@:~/OpenNMT/OpenNMT-py$

I recommend that you simply change conf.json to see if it works or not.

{
“models_root”: “./available_models”,
“models”: [
{
“id”: 100,
“model”: “model_step_70000.pt”,
“timeout”: 600,
“on_timeout”: “to_cpu”,
“load”: true,
“opt”: {
“gpu”: 2,
“beam_size”: 5,
“replace_unk”: true,
“verbose”: true
},
“tokenizer”: {
“type”: “sentencepiece”,
“model”: “src_model.model”
}
}
]
}

command

python3 server.py --ip 0.0.0.0 --port 5000 --url_root “/translator” --config “./available_models/conf.json”

Also check the firewall

Hi,

With the above change to the conf.json file, my conf.json file is as follows:

{
“models_root”: “./available_models”,
“models”: [
{
“id”: 1,
“model”: “iwslt-brnn2.s131_acc_62.71_ppl_7.74_e20.pt”,
“timeout”: 600,
“on_timeout”: “to_cpu”,
“load”: true,
“opt”: {
“gpu”: -1,
“beam_size”: 5,
“replace_unk”: true,
“verbose”: true
},
“tokenizer”: {
“type”: “sentencepiece”,
“model”: “src_model.model”
}
},{
“model”: “averaged-10-epoch.pt”,
“timeout”: -1,
“on_timeout”: “unload”,
“model_root”: “…/other_models”,
“opt”: {
“batch_size”: 1,
“beam_size”: 10
}
}
]
}

I am getting the following the error message :

**administrator@:~/OpenNMT/OpenNMT-py$ python3 server.py --ip “0.0.0.0” --port “7785” --url_root “/translator” --config “./available_models/conf.json”
Pre-loading model 1
[2019-06-18 13:13:35,235 INFO] Loading model 1
[2019-06-18 13:13:42,057 INFO] Loading tokenizer
Traceback (most recent call last):
File “server.py”, line 123, in
debug=args.debug)
File “server.py”, line 24, in start
translation_server.start(config_file)
File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 102, in start
self.preload_model(opt, model_id=model_id, **kwargs)
File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 140, in preload_model
model = ServerModel(opt, model_id, model_kwargs)
File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 227, in init
self.load()
File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 302, in load
sp.Load(model_path)
File “/home/administrator/Desktop/python/lib/python3.7/site-packages/sentencepiece.py”, line 118, in Load
return _sentencepiece.SentencePieceProcessor_Load(self, filename)
OSError: Not found: “./available_models/src_model.model”: No such file or directory Error #2
administrator@:~/OpenNMT/OpenNMT-py$

You need to change your conf.json with your own tokenizer.
If you use sentencepice

“type”: “sentencepiece”,
“model”: “YOURMODEL”

If you use Pyonmttok

“tokenizer”: {
“type”: “pyonmttok”,
“mode”: “aggressive”,
“params”: {
“no_substitution”: false,
“joiner_annotate”: true,
“joiner_new”: false,
“case_markup”: true,
“preserve_placeholders”: true,
“preserve_segmented_tokens”: true,
“segment_case”: true,
“segment_numbers”: true,
“segment_alphabet_change”: false

Hi,

  I had downloaded the GERMAN - ENGLISH model from the below link:

http://opennmt.net/Models-py/
The section model was downloaded was from Translation Section:

German-English - 2-layer BiLSTM (download)
Configuration 2-layer BiLSTM with hidden size 500 trained for 20 epochs
Data [IWSLT ‘14 DE-EN](https://github.com/pytorch/fairseq/blob/e734b0fa58fcf02ded15c236289b3bd61c4cffdf/data/prepare-iwslt14.sh)
BLEU 30.33

So where could I find the model associated with the Tokenizer?

Thank You ,
Kishor.

TOKENIZER=$SCRIPTS/tokenizer/tokenizer.perl

It is locate in /OpenNMT-py/tools/tokenizer.perl

Hi,

I could not find the tokenizer model associated in the file :
/OpenNMT-py/tools/tokenizer.perl

How do I find it there in the perl file? Please assist me here ///

Regards,
Kishor

Hi ,

I could find the tokenizer file mentioned in the earlier post.
i.e. OpenNMT-py/tools/tokenizer.perl

How do I use it in the conf.json file tokenizer section ?

Thank you,
Kishor.

{
“models_root”: “./available_models”,
“models”: [
{
“id”: 100,
“model”: “YOURMODEL”,
“timeout”: 600,
“on_timeout”: “to_cpu”,
“load”: true,
“opt”: {
“gpu”: -1,
“beam_size”: 5,
“replace_unk”: true,
“verbose”: true
},
}
]
}

change your conf.json
I know moses tokenizer is not supported separately.
Analyze server.py and apply the corresponding tokenizer to the input statement.

Hi,

 I checked translation_server.py, i could see that it uses "sentencepiece" and also "pyonmttok"

Let me try the pyonmttok as the option in tokenizer section in conf.json file. Am I right ?

Regards.
Kishor.

yes You can use pyonmttok or sentence piece.
In my case I use sentencepiece

Hi,

My Conf file:
{
“models_root”: “./available_models”,
“models”: [
{
“id”: 1,
“model”: “iwslt-brnn2.s131_acc_62.71_ppl_7.74_e20.pt”,
“timeout”: 600,
“on_timeout”: “to_cpu”,
“load”: true,
“opt”: {
“gpu”: -1,
“beam_size”: 5
},
“tokenizer”: {
“type”: “pyonmttok”,
“mode”: “str”,
“params”: {
“bpe_model_path”:"",
“vocabulary_path”:"",
“vocabulary_threshold”:0,
“sp_model_path”:"",
“sp_nbest_size”:0,
“sp_alpha”:0.1,
“joiner”:“■”,
“joiner_annotate”:false,
“joiner_new”:false,
“spacer_annotate”:false,
“spacer_new”:false,
“case_feature”:false,
“case_markup”:false,
“no_substitution”:false,
“preserve_placeholders”:false,
“preserve_segmented_tokens”:false,
“segment_case”:false,
“segment_numbers”:false,
“segment_alphabet_change”:false,
“segment_alphabet”:[]
}
}
},{
“model”: “model_0.light.pt”,
“timeout”: -1,
“on_timeout”: “unload”,
“model_root”: “…/other_models”,
“opt”: {
“batch_size”: 1,
“beam_size”: 10
}
}
]
}

the above conf parameters I got from the below link :

GitHub

OpenNMT/Tokenizer

OpenNMT tokenization library. Contribute to OpenNMT/Tokenizer development by creating an account on GitHub.

With the above changes I am getting the following error message:

administrator@:~/OpenNMT/OpenNMT-py$ python3 server.py --ip “0.0.0.0” --port “7785” --url_root “/translator” --config "./available_models/conf.json"
Pre-loading model 1
[2019-06-18 12:10:12,621 INFO] Loading model 1
[2019-06-18 12:10:19,179 INFO] Loading tokenizer
Traceback (most recent call last):
** File “server.py”, line 123, in **
** debug=args.debug)**
** File “server.py”, line 24, in start**
** translation_server.start(config_file)**
** File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 102, in start**
** self.preload_model(opt, model_id=model_id, kwargs)
** File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 140, in preload_model**
** model = ServerModel(opt, model_id, model_kwargs)
** File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 227, in init **
** self.load()**
** File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 319, in load**
** tokenizer = pyonmttok.Tokenizer(mode, tokenizer_params)
RuntimeError: basic_filebuf::underflow error reading the file: iostream error
administrator@:~/OpenNMT/OpenNMT-py$

Please assist me to resolve this issue?

Regards,
Kishor,

Hi,

Any update on the next post..I am getting the underflow file read error while using pyonmttok tokenizer.

Any assistance regarding to this …

Regards,
Kishor.

Please reference here.

Hi Park,

The  issue got resolved after help from Tokenizer team for "pyonmttok".
 By changing the config file the issue got resolved.

Regards,
Kishor.

@park
I get the error:

python3 server.py --ip 0.0.0.0 --port 5000 --url_root “/translator” --config “./available_models/conf.json”
Traceback (most recent call last):
File “server.py”, line 129, in
debug=args.debug)
File “server.py”, line 24, in start
translation_server.start(config_file)
File “/home/ng/OpenNMT-py-master/onmt/translate/translation_server.py”, line 80, in start
with open(self.config_file) as f:
FileNotFoundError: [Errno 2] No such file or directory: ‘“./available_models/conf.json”’

You need to define your configuration in the mentioned file.
See this post.

This is my conf.json file

{
“models_root”: “./available_models”,
“models”: [
{
“id”: 100,
“model”: “model_0.pt”,
“timeout”: 600,
“on_timeout”: “to_cpu”,
“load”: true,
“opt”: {
“gpu”: 0,
“beam_size”: 5
},
“tokenizer”: {
“type”: “sentencepiece”,
“model”: “wmtenfr.model”
}
},{
“model”: “model_0.light.pt”,
“timeout”: -1,
“on_timeout”: “unload”,
“model_root”: “…/other_models”,
“opt”: {
“batch_size”: 1,
“beam_size”: 10
}
}
]
}

The file is located in OpenNMT-py-master/available_models.