Issues running the OpenNMT-py REST server

KishorKP · June 18, 2019, 7:08am

Hi ,

git pull and restarting the server.py did not help. I changed the parameters in conf.json as follows:
conf.json file:

{
“models_root”: “./available_models”,
“models”: [
{
“id”: 1,
“model”: “iwslt-brnn2.s131_acc_62.71_ppl_7.74_e20.pt”,
“timeout”: 600,
“on_timeout”: “to_cpu”,
“load”: true,
“opt”: {
“gpu”: -1,
“beam_size”: 5
},
“tokenizer”: {
“type”: “pyonmttok”,
“mode”: “str”,
“params”: {
“bpe_model_path”:"",
“vocabulary_path”:"",
“vocabulary_threshold”:0,
“sp_model_path”:"",
“sp_nbest_size”:0,
“sp_alpha”:0.1,
“joiner”:“￭”,
“joiner_annotate”:false,
“joiner_new”:false,
“spacer_annotate”:false,
“spacer_new”:false,
“case_feature”:false,
“case_markup”:false,
“no_substitution”:false,
“preserve_placeholders”:false,
“preserve_segmented_tokens”:false,
“segment_case”:false,
“segment_numbers”:false,
“segment_alphabet_change”:false,
“segment_alphabet”:[]
}
}
},{
“model”: “model_0.light.pt”,
“timeout”: -1,
“on_timeout”: “unload”,
“model_root”: “…/other_models”,
“opt”: {
“batch_size”: 1,
“beam_size”: 10
}
}
]
}

the above conf parameters I got from the below link :

With the above changes I am getting the following error message:

administrator@:~/OpenNMT/OpenNMT-py$ python3 server.py --ip “0.0.0.0” --port “7785” --url_root “/translator” --config "./available_models/conf.json"
Pre-loading model 1
[2019-06-18 12:10:12,621 INFO] Loading model 1
[2019-06-18 12:10:19,179 INFO] Loading tokenizer
Traceback (most recent call last):
** File “server.py”, line 123, in **
** debug=args.debug)**
** File “server.py”, line 24, in start**
** translation_server.start(config_file)**
** File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 102, in start**
** self.preload_model(opt, model_id=model_id, kwargs)
** File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 140, in preload_model**
** model = ServerModel(opt, model_id, model_kwargs)
** File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 227, in init**
** self.load()**
** File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 319, in load**
** tokenizer = pyonmttok.Tokenizer(mode, tokenizer_params)
RuntimeError: basic_filebuf::underflow error reading the file: iostream error
administrator@:~/OpenNMT/OpenNMT-py$

Please assist me in resolving this issues as early as possible ?

Thank You,
Kishor.

KishorKP · June 18, 2019, 7:24am

Hi ,

git pull and restarting the server.py did not help. I changed the parameters in conf.json as follows:
conf.json file:

{
“models_root”: “./available_models”,
“models”: [
{
“id”: 1,
“model”: “iwslt-brnn2.s131_acc_62.71_ppl_7.74_e20.pt”,
“timeout”: 600,
“on_timeout”: “to_cpu”,
“load”: true,
“opt”: {
“gpu”: -1,
“beam_size”: 5
},
“tokenizer”: {
“type”: “pyonmttok”,
“mode”: “str”,
“params”: {
“bpe_model_path”:"",
“vocabulary_path”:"",
“vocabulary_threshold”:0,
“sp_model_path”:"",
“sp_nbest_size”:0,
“sp_alpha”:0.1,
“joiner”:“￭”,
“joiner_annotate”:false,
“joiner_new”:false,
“spacer_annotate”:false,
“spacer_new”:false,
“case_feature”:false,
“case_markup”:false,
“no_substitution”:false,
“preserve_placeholders”:false,
“preserve_segmented_tokens”:false,
“segment_case”:false,
“segment_numbers”:false,
“segment_alphabet_change”:false,
“segment_alphabet”:[]
}
}
},{
“model”: “model_0.light.pt”,
“timeout”: -1,
“on_timeout”: “unload”,
“model_root”: “…/other_models”,
“opt”: {
“batch_size”: 1,
“beam_size”: 10
}
}
]
}

the above conf parameters I got from the below link :

With the above changes I am getting the following error message:

administrator@:~/OpenNMT/OpenNMT-py$ python3 server.py --ip “0.0.0.0” --port “7785” --url_root “/translator” --config "./available_models/conf.json"
Pre-loading model 1
[2019-06-18 12:10:12,621 INFO] Loading model 1
[2019-06-18 12:10:19,179 INFO] Loading tokenizer
Traceback (most recent call last):
** File “server.py”, line 123, in **
** debug=args.debug)**
** File “server.py”, line 24, in start**
** translation_server.start(config_file)**
** File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 102, in start**
** self.preload_model(opt, model_id=model_id, kwargs)
** File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 140, in preload_model**
** model = ServerModel(opt, model_id, model_kwargs)
** File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 227, in init**
** self.load()**
** File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 319, in load**
** tokenizer = pyonmttok.Tokenizer(mode, tokenizer_params)
RuntimeError: basic_filebuf::underflow error reading the file: iostream error
administrator@:~/OpenNMT/OpenNMT-py$

park · June 18, 2019, 7:34am

I recommend that you simply change conf.json to see if it works or not.

{
“models_root”: “./available_models”,
“models”: [
{
“id”: 100,
“model”: “model_step_70000.pt”,
“timeout”: 600,
“on_timeout”: “to_cpu”,
“load”: true,
“opt”: {
“gpu”: 2,
“beam_size”: 5,
“replace_unk”: true,
“verbose”: true
},
“tokenizer”: {
“type”: “sentencepiece”,
“model”: “src_model.model”
}
}
]
}

command

python3 server.py --ip 0.0.0.0 --port 5000 --url_root “/translator” --config “./available_models/conf.json”

Also check the firewall

KishorKP · June 18, 2019, 7:48am

Hi,

With the above change to the conf.json file, my conf.json file is as follows:

{
“models_root”: “./available_models”,
“models”: [
{
“id”: 1,
“model”: “iwslt-brnn2.s131_acc_62.71_ppl_7.74_e20.pt”,
“timeout”: 600,
“on_timeout”: “to_cpu”,
“load”: true,
“opt”: {
“gpu”: -1,
“beam_size”: 5,
“replace_unk”: true,
“verbose”: true
},
“tokenizer”: {
“type”: “sentencepiece”,
“model”: “src_model.model”
}
},{
“model”: “averaged-10-epoch.pt”,
“timeout”: -1,
“on_timeout”: “unload”,
“model_root”: “…/other_models”,
“opt”: {
“batch_size”: 1,
“beam_size”: 10
}
}
]
}

I am getting the following the error message :

**administrator@:~/OpenNMT/OpenNMT-py$ python3 server.py --ip “0.0.0.0” --port “7785” --url_root “/translator” --config “./available_models/conf.json”
Pre-loading model 1
[2019-06-18 13:13:35,235 INFO] Loading model 1
[2019-06-18 13:13:42,057 INFO] Loading tokenizer
Traceback (most recent call last):
File “server.py”, line 123, in
debug=args.debug)
File “server.py”, line 24, in start
translation_server.start(config_file)
File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 102, in start
self.preload_model(opt, model_id=model_id, **kwargs)
File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 140, in preload_model
model = ServerModel(opt, model_id, model_kwargs)
File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 227, in init
self.load()
File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 302, in load
sp.Load(model_path)
File “/home/administrator/Desktop/python/lib/python3.7/site-packages/sentencepiece.py”, line 118, in Load
return _sentencepiece.SentencePieceProcessor_Load(self, filename)
OSError: Not found: “./available_models/src_model.model”: No such file or directory Error #2
administrator@:~/OpenNMT/OpenNMT-py$

park · June 18, 2019, 7:53am

You need to change your conf.json with your own tokenizer.
If you use sentencepice

“type”: “sentencepiece”,
“model”: “YOURMODEL”

If you use Pyonmttok

“tokenizer”: {
“type”: “pyonmttok”,
“mode”: “aggressive”,
“params”: {
“no_substitution”: false,
“joiner_annotate”: true,
“joiner_new”: false,
“case_markup”: true,
“preserve_placeholders”: true,
“preserve_segmented_tokens”: true,
“segment_case”: true,
“segment_numbers”: true,
“segment_alphabet_change”: false

KishorKP · June 18, 2019, 8:12am

Hi,

  I had downloaded the GERMAN - ENGLISH model from the below link:

http://opennmt.net/Models-py/
The section model was downloaded was from Translation Section:

German-English - 2-layer BiLSTM (download)
Configuration 2-layer BiLSTM with hidden size 500 trained for 20 epochs
Data [IWSLT ‘14 DE-EN](https://github.com/pytorch/fairseq/blob/e734b0fa58fcf02ded15c236289b3bd61c4cffdf/data/prepare-iwslt14.sh)
BLEU 30.33

So where could I find the model associated with the Tokenizer?

Thank You ,
Kishor.

park · June 18, 2019, 8:16am

TOKENIZER=$SCRIPTS/tokenizer/tokenizer.perl

It is locate in /OpenNMT-py/tools/tokenizer.perl

KishorKP · June 18, 2019, 8:32am

Hi,

I could not find the tokenizer model associated in the file :
/OpenNMT-py/tools/tokenizer.perl

How do I find it there in the perl file? Please assist me here ///

Regards,
Kishor

park · June 18, 2019, 8:35am

github.com

OpenNMT/OpenNMT-py/blob/master/tools/tokenizer.perl

#!/usr/bin/env perl
#
# This file is part of moses.  Its use is licensed under the GNU Lesser General
# Public License version 2.1 or, at your option, any later version.

use warnings;

# Sample Tokenizer
### Version 1.1
# written by Pidong Wang, based on the code written by Josh Schroeder and Philipp Koehn
# Version 1.1 updates:
#       (1) add multithreading option "-threads NUM_THREADS" (default is 1);
#       (2) add a timing option "-time" to calculate the average speed of this tokenizer;
#       (3) add an option "-lines NUM_SENTENCES_PER_THREAD" to set the number of lines for each thread (default is 2000), and this option controls the memory amount needed: the larger this number is, the larger memory is required (the higher tokenization speed);
### Version 1.0
# $Id: tokenizer.perl 915 2009-08-10 08:15:49Z philipp $
# written by Josh Schroeder, based on code by Philipp Koehn

binmode(STDIN, ":utf8");
binmode(STDOUT, ":utf8");

This file has been truncated. show original

KishorKP · June 18, 2019, 8:38am

Hi ,

I could find the tokenizer file mentioned in the earlier post.
i.e. OpenNMT-py/tools/tokenizer.perl

How do I use it in the conf.json file tokenizer section ?

Thank you,
Kishor.

park · June 18, 2019, 8:42am

{
“models_root”: “./available_models”,
“models”: [
{
“id”: 100,
“model”: “YOURMODEL”,
“timeout”: 600,
“on_timeout”: “to_cpu”,
“load”: true,
“opt”: {
“gpu”: -1,
“beam_size”: 5,
“replace_unk”: true,
“verbose”: true
},
}
]
}

change your conf.json
I know moses tokenizer is not supported separately.
Analyze server.py and apply the corresponding tokenizer to the input statement.

KishorKP · June 18, 2019, 8:55am

Hi,

 I checked translation_server.py, i could see that it uses "sentencepiece" and also "pyonmttok"

Let me try the pyonmttok as the option in tokenizer section in conf.json file. Am I right ?

Regards.
Kishor.

park · June 18, 2019, 9:03am

yes You can use pyonmttok or sentence piece.
In my case I use sentencepiece

KishorKP · June 18, 2019, 9:12am

Hi,

My Conf file:
{
“models_root”: “./available_models”,
“models”: [
{
“id”: 1,
“model”: “iwslt-brnn2.s131_acc_62.71_ppl_7.74_e20.pt”,
“timeout”: 600,
“on_timeout”: “to_cpu”,
“load”: true,
“opt”: {
“gpu”: -1,
“beam_size”: 5
},
“tokenizer”: {
“type”: “pyonmttok”,
“mode”: “str”,
“params”: {
“bpe_model_path”:"",
“vocabulary_path”:"",
“vocabulary_threshold”:0,
“sp_model_path”:"",
“sp_nbest_size”:0,
“sp_alpha”:0.1,
“joiner”:“￭”,
“joiner_annotate”:false,
“joiner_new”:false,
“spacer_annotate”:false,
“spacer_new”:false,
“case_feature”:false,
“case_markup”:false,
“no_substitution”:false,
“preserve_placeholders”:false,
“preserve_segmented_tokens”:false,
“segment_case”:false,
“segment_numbers”:false,
“segment_alphabet_change”:false,
“segment_alphabet”:[]
}
}
},{
“model”: “model_0.light.pt”,
“timeout”: -1,
“on_timeout”: “unload”,
“model_root”: “…/other_models”,
“opt”: {
“batch_size”: 1,
“beam_size”: 10
}
}
]
}

the above conf parameters I got from the below link :

GitHub

OpenNMT/Tokenizer

OpenNMT tokenization library. Contribute to OpenNMT/Tokenizer development by creating an account on GitHub.

With the above changes I am getting the following error message:

administrator@:~/OpenNMT/OpenNMT-py$ python3 server.py --ip “0.0.0.0” --port “7785” --url_root “/translator” --config "./available_models/conf.json"
Pre-loading model 1
[2019-06-18 12:10:12,621 INFO] Loading model 1
[2019-06-18 12:10:19,179 INFO] Loading tokenizer
Traceback (most recent call last):
** File “server.py”, line 123, in **
** debug=args.debug)**
** File “server.py”, line 24, in start**
** translation_server.start(config_file)**
** File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 102, in start**
** self.preload_model(opt, model_id=model_id, kwargs)
** File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 140, in preload_model**
** model = ServerModel(opt, model_id, model_kwargs)
** File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 227, in init **
** self.load()**
** File “/home/administrator/OpenNMT/OpenNMT-py/onmt/translate/translation_server.py”, line 319, in load**
** tokenizer = pyonmttok.Tokenizer(mode, tokenizer_params)
RuntimeError: basic_filebuf::underflow error reading the file: iostream error
administrator@:~/OpenNMT/OpenNMT-py$

Please assist me to resolve this issue?

Regards,
Kishor,

KishorKP · June 18, 2019, 12:35pm

Hi,

Any update on the next post..I am getting the underflow file read error while using pyonmttok tokenizer.

Any assistance regarding to this …

Regards,
Kishor.

park · June 18, 2019, 11:21pm

Please reference here.

KishorKP · July 30, 2019, 10:39am

Hi Park,

The  issue got resolved after help from Tokenizer team for "pyonmttok".
 By changing the config file the issue got resolved.

Regards,
Kishor.

ngdave · August 24, 2019, 2:44pm

@park
I get the error:

python3 server.py --ip 0.0.0.0 --port 5000 --url_root “/translator” --config “./available_models/conf.json”
Traceback (most recent call last):
File “server.py”, line 129, in
debug=args.debug)
File “server.py”, line 24, in start
translation_server.start(config_file)
File “/home/ng/OpenNMT-py-master/onmt/translate/translation_server.py”, line 80, in start
with open(self.config_file) as f:
FileNotFoundError: [Errno 2] No such file or directory: ‘“./available_models/conf.json”’

francoishernandez · August 24, 2019, 3:27pm

You need to define your configuration in the mentioned file.
See this post.

ngdave · August 26, 2019, 11:30am

This is my conf.json file

{
“models_root”: “./available_models”,
“models”: [
{
“id”: 100,
“model”: “model_0.pt”,
“timeout”: 600,
“on_timeout”: “to_cpu”,
“load”: true,
“opt”: {
“gpu”: 0,
“beam_size”: 5
},
“tokenizer”: {
“type”: “sentencepiece”,
“model”: “wmtenfr.model”
}
},{
“model”: “model_0.light.pt”,
“timeout”: -1,
“on_timeout”: “unload”,
“model_root”: “…/other_models”,
“opt”: {
“batch_size”: 1,
“beam_size”: 10
}
}
]
}

The file is located in OpenNMT-py-master/available_models.