Translation server crashing after initialization

Support
21

Ok - if this is the only way to get rest_translation_server I’ll try to work with the cloned repository. Could I please have the url and I’ll try it over the weekend.
Thanks.

22

Hello everyone,

@tek34 abd @dbl, the code is very similar to the initial one , just launching a waffle server in stead of zeromq.

You need to install the following dependencies if missing:

buffer: luarocks install buffer
async: luarocks install async
redis-async: luarocks install redis-async
htmlua: luarocks install https://raw.githubusercontent.com/benglard/htmlua/master/htmlua-scm-1.rockspec
waffle: luarocks install https://raw.githubusercontent.com/benglard/waffle/master/waffle-scm-1.rockspec

That’s it basically it, and then I have the following code:

local server = require("waffle") 

.......

local function main()
  local opt = cmd:parse(arg)
  local requiredOptions = {
    "model"
  }

  onmt.utils.Opt.init(opt, requiredOptions)

  _G.logger = onmt.utils.Logger.new(opt.log_file, opt.disable_logs, opt.log_level)

  _G.logger:info("Loading model")
  local translator = onmt.translate.Translator.new(opt)

  server.host = opt.host
  server.port = opt.port
  
  _G.logger:info("Server initialized at " .. server.host .. ":" .. server.port)
  
  
  server.post('/', function(req, res)
		-- Input format is a json batch of src strings.
		local jsonsource = req.body
		local source
		
		if ( opt.debugmode ) then _G.logger:info("Request body (JSON): " .. jsonsource .. "\n") end
		
		local source, pos, err_read_json = json.decode (jsonsource, 1, nil)
			-- Currently assuming that it is only one string. 
			-- The translation supports multiple at once; we can extend.
			
		print(source)
		
		local result 
		local flatten_result
		
		if err_read_json then
			_G.logger:info("Error: " ..  err_read_json  .. "\n")
			-- Hide paths included in the error message.
			err_read_json = err_read_json:gsub("/[^:]*/", "")
			result = err_read_json
		end
		
		if ( opt.debugmode ) then _G.logger:info("Translating: " .. #source .. "\n") end
		
		
		local _, err_translate = pcall(function ()
			if (isTable(source)) then
				result = translateMessage(translator, source)
			else
				result = translateMessage(translator, {{text=source}})
			end
		end)

		if err_translate then
			_G.logger:info("Error: " .. err_translate .. "\n")
			-- Hide paths included in the error message.
			err_translate = err_translate:gsub("/[^:]*/", "")
			result = err_translate
		end

		if ( opt.debugmode ) then _G.logger:info("Translation result: " .. #result .. "\n") end
		
		-- Now add back the translation and print it out
		local ret = json.encode(result)
		
		if ( opt.debugmode ) then _G.logger:info("Translation finished.") end
		res.send(ret)
		collectgarbage()
	end)  
	server.listen()
end

I have modified also the translateMessage/2 function to look for a ‘text’ field, rather than a ‘src’ field.

The code was originally made to work with XML but I have changed it to use json (dkjson). It is not fully tested for json, so apologies for that.

Dimitar.

2 Likes
23

Thanks, @dimitarsh1! I’ll see if I can give this a try on a docker container this weekend.

Unrelated question… Are you using CPU or GPU translation for your released engines on this server setup?

24

Thanks from me too @dimitarsh1. Looks like I’m going to have a busy Saturday morning - at least it’s raining here!
Terence

25

Hello David @dbl, we use both CPU and GPU machines with this server; also both models made for release and the intermediately trained ones from each epoch.

By the way, as the code became large to be placed in hte forum I haven’t included the rest of it, which is pretty much the same as in the translation_server.lua script. If you have any problems integrating it into your code, let me know.

Cheers,
Dimitar

26

Just head over to the GitHub page, you’ll find appropriate links

@dimitarsh1 Thanks for sharing. It may also be useful to share which version or revision of OpenNMT your script depends on. Things are still moving in the API side and your script may not be compatible with the latest version.

27

Hi Dimitar @dimitarsh1
Re:
buffer: luarocks install buffer
async: luarocks install async
redis-async: luarocks install redis-async

I’m getting “No results matching query were found” even trying with sudo.
Where are they located?
Thanks,
Terence

28

Dear Terrance,
here is the github repository for the redis-async.

You can get it from there, although I didn’t need to in any of the installations I tried. For me luarocks install redis-async works.

Hope this works.
Cheers,
Dimitar

29

The version I have is from the beginning of February. It is versino v 0.3.0.

Kind regards,
Dimitar

30

I have just installed release OpenNMT v05 and got the rest_translation_server up and running. I’d like to thank everyone who came up with ideas for a solution. In the end the prospect of built-in tokenization/detokenization won the day!
Terence

1 Like
31

Glad to hear that.

Just one thing:
If you intedn to use it in CPU mode, I highly recommend to use -beam_size 1 as a cmdline option.
Otherwise it will be slow. (default value is 5 and I tested it, it’s really slow even on fast procs)

1 Like
32

I’ve just been trying it in GPU mode and it seems blisteringly fast. I’ll bear your comment in mind if we put the set-up on a server with CPU only talking to the outside world.