Error in rest_translation_server.lua:105: attempt to index field 'preds' (a nil value) (500)


(Zuzanna Parcheta) #1

I am trying to run rest_translation_server.lua. I do it following the manual from the website: http://opennmt.net/OpenNMT/tools/servers/

I run the server and I receive the message:

th tools/rest_translation_server.lua -model ~/models/_epoch25_1.00.t7 -gpuid 1
[01/03/18 11:09:59 INFO] Using GPU(s): 1
[01/03/18 11:09:59 WARNING] The caching CUDA memory allocator is enabled. This allocator improves performance at the cost of a higher GPU memory usage. To optimize for memory, consider disabling it by setting the environment variable: THC_CACHING_ALLOCATOR=0
[01/03/18 11:09:59 INFO] Loading model
[01/03/18 11:09:59 INFO] Loading ‘/home/German/models/_epoch25_1.00.t7’…
[01/03/18 11:10:00 INFO] Model seq2seq trained on bitext
[01/03/18 11:10:00 INFO] Launch server
Xavante started on port(s) 7784

But when I make a curl I get the following error:

curl -v -H “Content-Type: application/json” -d ‘[{ “src” : “play zoo on spotify” }]’ http://127.0.0.1:7784/translator/translate

  • Trying 127.0.0.1…
  • Connected to 127.0.0.1 (127.0.0.1) port 7784 (#0)

POST /translator/translate HTTP/1.1
Host: 127.0.0.1:7784
User-Agent: curl/7.47.0
Accept: /
Content-Type: application/json
Content-Length: 35

  • upload completely sent off: 35 out of 35 bytes
    < HTTP/1.1 500 Internal Server Error
    < Content-Type: text/plain
    < Date: Wed, 03 Jan 2018 10:14:01 GMT
    < Server: RestServer
    < Content-Length: 193
    <
  • Connection #0 to host 127.0.0.1 left intact
    500 Internal Server Error - Error in application: tools/rest_translation_server.lua:105: unicode error in line tools/rest_translation_server.lua:98: attempt to index field ‘preds’ (a nil value)

Has anyone else get this error too? I instaled luarocks install restserver-xavante and luarocks install nn


(Vincent Nguyen) #2

are you on master ?


(Zuzanna Parcheta) #3

yes, I am on master.


(Vincent Nguyen) #4

try to check out on this hash and check if it works.


(Zuzanna Parcheta) #5

solved! It was solved in this commit https://github.com/OpenNMT/OpenNMT/commit/8e8970ee3a825f36c1ba627f1512f7bec0c535e7


(Wiktor Stribiżew) #6

Hi, this erorr persists at my end. I have defined a YML config at /Engines/server/models_cfg.yml with

-
  model: 'path-to-existing-GPU-nonreleased-model'
  replace_unk: true
  mode: aggressive
  joiner_annotate: true
  case_feature: true
  segment_case: true
  nparallel: 6
  segment_numbers: true
  segment_alphabet_change:true
  segment_alphabet: 'Han,Thai,Katakana,Hiragana'
  bpe_model: 'path-to-existing-relevant-BPE-model'
  beam size: 5

Ran luarocks install torch; luarocks install nn;, then cd /OpenNMT09/OpenNMT; th tools/rest_multi_models.lua -model_config /Engines/server/models_cfg.yml -gpuid 1 -log_level DEBUG.

I see Xavante started on port(s) 7784, but when sending the request like curl -v -H "Content-Type: application/json" -X POST -d '[{ "src" : "Hello World", "id" : 1 }]' http://127.0.0.1:7784/translator/translate (or even with "id": 0) I get

* upload completely sent off: 37 out of 37 bytes
< HTTP/1.1 500 Internal Server Error
< Content-Type: application/json
< Date: Thu, 29 Mar 2018 13:51:11 GMT
< Server: SGA
< Content-Length: 4

Any ideas what is going on? Note I tried to add gpuid: 1 to the model options in the config file, too, it does not change anything.


(Vincent Nguyen) #7
  1. give us your git log -1
  2. change your yml: beam size => beam_size

(Wiktor Stribiżew) #8

git log -1 returns

root@ip-172-30-0-140:/OpenNMT09/OpenNMT# git log  -1
commit 364000444b19814ebe30bfa06b6438e7f76c8efc
Author: Jean Senellart <jean@senellart.com>
Date:   Fri Feb 9 17:34:13 2018 +0100

    introduce log_tag option (#517)

Changing to beam_size did not help.

BTW, when I use

th tools/rest_translation_server.lua -model 'gpu_model_path' -gpuid 1 -port 7784 -mode aggressive -segment_numbers -segment_alphabet_change -segment_alphabet {Han,Thai,Katakana,Hiragana} -joiner_annotate -case_feature -bpe_model 'path_to_bpe_model' -log_level DEBUG

and then send a curl request like

curl -v -H "Content-Type: application/json" -X POST -d '{"src": "Hello world", "id" : 1}' http://127.0.0.1:7784/translator/translate

I get

Note: Unnecessary use of -X or --request, POST is already inferred.
*   Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 7784 (#0)
> POST /translator/translate HTTP/1.1
> Host: 127.0.0.1:7784
> User-Agent: curl/7.47.0
> Accept: */*
> Content-Type: application/json
> Content-Length: 44
>
* upload completely sent off: 44 out of 44 bytes
< HTTP/1.1 200 OK
< Content-Type: application/json
< Date: Fri, 30 Mar 2018 06:33:25 GMT
< Server: RestServer
< Content-Length: 2
<
* Connection #0 to host 127.0.0.1 left intact
[]

No translation is returned, but the response code is 200 OK. The debug shows

[03/30/18 06:33:25 DEBUG] receiving request
[03/30/18 06:33:25 DEBUG] Start Tokenization
[03/30/18 06:33:25 DEBUG] Start Translation
[03/30/18 06:33:25 DEBUG] End Translation
[03/30/18 06:33:25 DEBUG] sending response

Just the response has no translation in it :frowning:


(Vincent Nguyen) #9

did you luarocks install restserver-xavante ?


(Wiktor Stribiżew) #10

Yes, sure, I would not have started the server without it, would I? I followed the steps described at http://opennmt.net/OpenNMT/tools/servers. luarocks install restserver-xavante and luarocks install yaml are the first steps. I am launching the server and send requests with superuser permissions.


(Vincent Nguyen) #11

your git log should be fine
but just in case you modified things, can you please check this issue and see if it applies:


(Wiktor Stribiżew) #12

I ran

nano tools/rest_translation_server.lua

Change the local restserver = require("restserver") to local restserver = require("tools.restserver.restserver"), ran the REST server again with

th tools/rest_translation_server.lua -model 'path-to-model' -gpuid 1 -port 7784 -mode aggressive -segment_numbers -segment_alphabet_change -segment_alphabet {Han,Thai,Katakana,Hiragana} -joiner_annotate -case_feature -bpe_model 'path-to-bpe' -log_level DEBUG

and sent the translation request again curl -v -H "Content-Type: application/json" -X POST -d '[{ "src" : "What is going on?" }]' http://127.0.0.1:7784/translator/translate. Now, the server debug shows

[03/30/18 08:13:54 DEBUG] receiving request
[03/30/18 08:13:54 DEBUG] Start Tokenization
[03/30/18 08:18:47 DEBUG] receiving request
[03/30/18 08:18:47 DEBUG] Start Tokenization

and the client shows

* upload completely sent off: 33 out of 33 bytes
< HTTP/1.1 500 Internal Server Error
< Content-Type: text/plain
< Date: Fri, 30 Mar 2018 08:18:47 GMT
< Server: RestServer
< Content-Length: 193
<
* Connection #0 to host 127.0.0.1 left intact
500 Internal Server Error - Error in application: tools/rest_translation_server.lua:60: unicode error in line ./tools/utils/tokenizer.lua:340: attempt to index field 'hookManager' (a nil value)

``


(Wiktor Stribiżew) #13

Ok, single model REST server translation is solved by replacing my current rest_translation_server.lua with the latest one downloaded from Git from https://github.com/OpenNMT/OpenNMT/blob/master/tools/rest_translation_server.lua. I am checking the multi model now…


(Icho Katsarski) #14

Hi guys, I’m having the same issue with the rest_multi_models.lua . In my case I have nvidia-docker container, with mkl compiled lua. Translations on GPU and CPU work fine on rest_translation_server.lua .
CPU tranlations work on the rest_multi_models.lua as well, but I get 500 Internal Server Error, when translating on GPU.

Container started, things look good:

[06/18/18 08:36:58 INFO] Using GPU(s): 1
[06/18/18 08:36:58 INFO] Launch server
Xavante started on port(s) 7784

curl -v -H “Content-Type: application/json” -X POST -d ‘[{ “src” : “Hello World” , “id” : 1 }]’ 127.0.0.1:7784/translator/translate

Response:

  • Trying 127.0.0.1…
  • TCP_NODELAY set
  • Connected to 127.0.0.1 (127.0.0.1) port 7784 (#0)

POST /translator/translate HTTP/1.1
Host: 127.0.0.1:7784
User-Agent: curl/7.52.1
Accept: /
Content-Type: application/json
Content-Length: 38

  • upload completely sent off: 38 out of 38 bytes
    < HTTP/1.1 500 Internal Server Error
    < Content-Type: application/json
    < Date: Mon, 18 Jun 2018 08:52:02 GMT
    < Server: SGA
    < Content-Length: 4
    <
  • Curl_http_done: called premature == 0
  • Connection #0 to host 127.0.0.1 left intact
    null

Using latest version of OpenNMT:

~/opennmt# git log
commit e7aded4edf5c7acd50e672f8c12f2b8e0b684242
Author: Guillaume Klein guillaume.klein@systrangroup.com
Date: Thu Jun 14 14:19:04 2018 +0200

Fix learn_bpe.lua for Lua 5.2

commit 1db3dfcb66609f7ac1d655854c81ecc3e4f53659
Author: Guillaume Klein guillaumekln@users.noreply.github.com
Date: Fri May 25 10:38:06 2018 +0200

Fully ignore placeholders when learning BPE (#545)

Could not get much more information from the curl, about the error. Do you guys have any ideas?