Error in rest_translation_server.lua:105: attempt to index field 'preds' (a nil value) (500)

Support
1

I am trying to run rest_translation_server.lua. I do it following the manual from the website: http://opennmt.net/OpenNMT/tools/servers/

I run the server and I receive the message:

th tools/rest_translation_server.lua -model ~/models/_epoch25_1.00.t7 -gpuid 1
[01/03/18 11:09:59 INFO] Using GPU(s): 1
[01/03/18 11:09:59 WARNING] The caching CUDA memory allocator is enabled. This allocator improves performance at the cost of a higher GPU memory usage. To optimize for memory, consider disabling it by setting the environment variable: THC_CACHING_ALLOCATOR=0
[01/03/18 11:09:59 INFO] Loading model
[01/03/18 11:09:59 INFO] Loading ‘/home/German/models/_epoch25_1.00.t7’…
[01/03/18 11:10:00 INFO] Model seq2seq trained on bitext
[01/03/18 11:10:00 INFO] Launch server
Xavante started on port(s) 7784

But when I make a curl I get the following error:

curl -v -H “Content-Type: application/json” -d ‘[{ “src” : “play zoo on spotify” }]’ http://127.0.0.1:7784/translator/translate

  • Trying 127.0.0.1…
  • Connected to 127.0.0.1 (127.0.0.1) port 7784 (#0)

POST /translator/translate HTTP/1.1
Host: 127.0.0.1:7784
User-Agent: curl/7.47.0
Accept: /
Content-Type: application/json
Content-Length: 35

  • upload completely sent off: 35 out of 35 bytes
    < HTTP/1.1 500 Internal Server Error
    < Content-Type: text/plain
    < Date: Wed, 03 Jan 2018 10:14:01 GMT
    < Server: RestServer
    < Content-Length: 193
    <
  • Connection #0 to host 127.0.0.1 left intact
    500 Internal Server Error - Error in application: tools/rest_translation_server.lua:105: unicode error in line tools/rest_translation_server.lua:98: attempt to index field ‘preds’ (a nil value)

Has anyone else get this error too? I instaled luarocks install restserver-xavante and luarocks install nn

2

are you on master ?

3

yes, I am on master.

4

try to check out on this hash and check if it works.

5

solved! It was solved in this commit https://github.com/OpenNMT/OpenNMT/commit/8e8970ee3a825f36c1ba627f1512f7bec0c535e7

6

Hi, this erorr persists at my end. I have defined a YML config at /Engines/server/models_cfg.yml with

-
  model: 'path-to-existing-GPU-nonreleased-model'
  replace_unk: true
  mode: aggressive
  joiner_annotate: true
  case_feature: true
  segment_case: true
  nparallel: 6
  segment_numbers: true
  segment_alphabet_change:true
  segment_alphabet: 'Han,Thai,Katakana,Hiragana'
  bpe_model: 'path-to-existing-relevant-BPE-model'
  beam size: 5

Ran luarocks install torch; luarocks install nn;, then cd /OpenNMT09/OpenNMT; th tools/rest_multi_models.lua -model_config /Engines/server/models_cfg.yml -gpuid 1 -log_level DEBUG.

I see Xavante started on port(s) 7784, but when sending the request like curl -v -H "Content-Type: application/json" -X POST -d '[{ "src" : "Hello World", "id" : 1 }]' http://127.0.0.1:7784/translator/translate (or even with "id": 0) I get

* upload completely sent off: 37 out of 37 bytes
< HTTP/1.1 500 Internal Server Error
< Content-Type: application/json
< Date: Thu, 29 Mar 2018 13:51:11 GMT
< Server: SGA
< Content-Length: 4

Any ideas what is going on? Note I tried to add gpuid: 1 to the model options in the config file, too, it does not change anything.

7
  1. give us your git log -1
  2. change your yml: beam size => beam_size
8

git log -1 returns

root@ip-172-30-0-140:/OpenNMT09/OpenNMT# git log  -1
commit 364000444b19814ebe30bfa06b6438e7f76c8efc
Author: Jean Senellart <jean@senellart.com>
Date:   Fri Feb 9 17:34:13 2018 +0100

    introduce log_tag option (#517)

Changing to beam_size did not help.

BTW, when I use

th tools/rest_translation_server.lua -model 'gpu_model_path' -gpuid 1 -port 7784 -mode aggressive -segment_numbers -segment_alphabet_change -segment_alphabet {Han,Thai,Katakana,Hiragana} -joiner_annotate -case_feature -bpe_model 'path_to_bpe_model' -log_level DEBUG

and then send a curl request like

curl -v -H "Content-Type: application/json" -X POST -d '{"src": "Hello world", "id" : 1}' http://127.0.0.1:7784/translator/translate

I get

Note: Unnecessary use of -X or --request, POST is already inferred.
*   Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 7784 (#0)
> POST /translator/translate HTTP/1.1
> Host: 127.0.0.1:7784
> User-Agent: curl/7.47.0
> Accept: */*
> Content-Type: application/json
> Content-Length: 44
>
* upload completely sent off: 44 out of 44 bytes
< HTTP/1.1 200 OK
< Content-Type: application/json
< Date: Fri, 30 Mar 2018 06:33:25 GMT
< Server: RestServer
< Content-Length: 2
<
* Connection #0 to host 127.0.0.1 left intact
[]

No translation is returned, but the response code is 200 OK. The debug shows

[03/30/18 06:33:25 DEBUG] receiving request
[03/30/18 06:33:25 DEBUG] Start Tokenization
[03/30/18 06:33:25 DEBUG] Start Translation
[03/30/18 06:33:25 DEBUG] End Translation
[03/30/18 06:33:25 DEBUG] sending response

Just the response has no translation in it :frowning:

9

did you luarocks install restserver-xavante ?

10

Yes, sure, I would not have started the server without it, would I? I followed the steps described at http://opennmt.net/OpenNMT/tools/servers. luarocks install restserver-xavante and luarocks install yaml are the first steps. I am launching the server and send requests with superuser permissions.

11

your git log should be fine
but just in case you modified things, can you please check this issue and see if it applies:

12

I ran

nano tools/rest_translation_server.lua

Change the local restserver = require("restserver") to local restserver = require("tools.restserver.restserver"), ran the REST server again with

th tools/rest_translation_server.lua -model 'path-to-model' -gpuid 1 -port 7784 -mode aggressive -segment_numbers -segment_alphabet_change -segment_alphabet {Han,Thai,Katakana,Hiragana} -joiner_annotate -case_feature -bpe_model 'path-to-bpe' -log_level DEBUG

and sent the translation request again curl -v -H "Content-Type: application/json" -X POST -d '[{ "src" : "What is going on?" }]' http://127.0.0.1:7784/translator/translate. Now, the server debug shows

[03/30/18 08:13:54 DEBUG] receiving request
[03/30/18 08:13:54 DEBUG] Start Tokenization
[03/30/18 08:18:47 DEBUG] receiving request
[03/30/18 08:18:47 DEBUG] Start Tokenization

and the client shows

* upload completely sent off: 33 out of 33 bytes
< HTTP/1.1 500 Internal Server Error
< Content-Type: text/plain
< Date: Fri, 30 Mar 2018 08:18:47 GMT
< Server: RestServer
< Content-Length: 193
<
* Connection #0 to host 127.0.0.1 left intact
500 Internal Server Error - Error in application: tools/rest_translation_server.lua:60: unicode error in line ./tools/utils/tokenizer.lua:340: attempt to index field 'hookManager' (a nil value)

``

13

Ok, single model REST server translation is solved by replacing my current rest_translation_server.lua with the latest one downloaded from Git from https://github.com/OpenNMT/OpenNMT/blob/master/tools/rest_translation_server.lua. I am checking the multi model now…

14

Hi guys, I’m having the same issue with the rest_multi_models.lua . In my case I have nvidia-docker container, with mkl compiled lua. Translations on GPU and CPU work fine on rest_translation_server.lua .
CPU tranlations work on the rest_multi_models.lua as well, but I get 500 Internal Server Error, when translating on GPU.

Container started, things look good:

[06/18/18 08:36:58 INFO] Using GPU(s): 1
[06/18/18 08:36:58 INFO] Launch server
Xavante started on port(s) 7784

curl -v -H “Content-Type: application/json” -X POST -d ‘[{ “src” : “Hello World” , “id” : 1 }]’ 127.0.0.1:7784/translator/translate

Response:

  • Trying 127.0.0.1…
  • TCP_NODELAY set
  • Connected to 127.0.0.1 (127.0.0.1) port 7784 (#0)

POST /translator/translate HTTP/1.1
Host: 127.0.0.1:7784
User-Agent: curl/7.52.1
Accept: /
Content-Type: application/json
Content-Length: 38

  • upload completely sent off: 38 out of 38 bytes
    < HTTP/1.1 500 Internal Server Error
    < Content-Type: application/json
    < Date: Mon, 18 Jun 2018 08:52:02 GMT
    < Server: SGA
    < Content-Length: 4
    <
  • Curl_http_done: called premature == 0
  • Connection #0 to host 127.0.0.1 left intact
    null

Using latest version of OpenNMT:

~/opennmt# git log
commit e7aded4edf5c7acd50e672f8c12f2b8e0b684242
Author: Guillaume Klein guillaume.klein@systrangroup.com
Date: Thu Jun 14 14:19:04 2018 +0200

Fix learn_bpe.lua for Lua 5.2

commit 1db3dfcb66609f7ac1d655854c81ecc3e4f53659
Author: Guillaume Klein guillaumekln@users.noreply.github.com
Date: Fri May 25 10:38:06 2018 +0200

Fully ignore placeholders when learning BPE (#545)

Could not get much more information from the curl, about the error. Do you guys have any ideas?