Having done a fresh install on a new server, I had to look back to some comments I posted previously on this forum to succeed. Here is the exact procedure working for me, to make the server properly running:
sudo apt-get install libzmq1
sudo apt-get install libzmq-dev
luarocks install lua-zmq ZEROMQ_LIBDIR=/usr/lib/x86_64-linux-gnu/ ZEROMQ_INCDIR=/usr/include
luarocks install dkjson
Hope it will also works for you…
Thanks to @Etienne38 and @vince62s. I’ll get back on both of these. rest_translation_server with the built-in tokenization/detokenization is certainly appealing.
Terence
@vince62s rest_translation_server.lua is set up but when I run it with options following the order given at the top of your file it complains about -model being an invalid argument (although this is a functioning model under translate.lua) and also asks me to specify the mode for tokenize.lua. Is this also expected on the command line, if so in any particular order?
Thanks
@tel34 - what is the exact log you have? the fact that he asks for mode for tokenize.lua is just because when cmdline parse fail, it displays the help file - so you do not need to worry (except if you did not use default tokenization options).
There is no order in the commandline.
I suspect I am causing this problem myself.
rest_translation_server.lua requires ‘tools/utils/tokenizer’ which my system does not have. I decided to use ‘tools/tokenize.lua’ and this is clearly the root cause of my trouble. What rock do I need to install to get tokenizer.lua into ‘tools/utils’?
Terence
The rock is not shipped with tools as of now. Could you work with a cloned repository instead after removing the opennmt rock?
Would removing the onmt rock result in the removal of all the data directories I have created under the ~/OpenNMT directory? Sorry for the obvious question but most of my programming experience has been with Java and I am quite new to the Lua way. All I really want is to get a working server (be it translation_server.lua or rest_translation_server.lua). I have followed all the instructions given to me in this thread to the letter and I still have the problem I stated when I started.
Terence
No, removing the rock will only remove the OpenNMT library from the system.
Ok - if this is the only way to get rest_translation_server I’ll try to work with the cloned repository. Could I please have the url and I’ll try it over the weekend.
Thanks.
Hello everyone,
@tek34 abd @dbl, the code is very similar to the initial one , just launching a waffle server in stead of zeromq.
You need to install the following dependencies if missing:
buffer: luarocks install buffer
async: luarocks install async
redis-async: luarocks install redis-async
htmlua: luarocks install https://raw.githubusercontent.com/benglard/htmlua/master/htmlua-scm-1.rockspec
waffle: luarocks install https://raw.githubusercontent.com/benglard/waffle/master/waffle-scm-1.rockspec
That’s it basically it, and then I have the following code:
local server = require("waffle")
.......
local function main()
local opt = cmd:parse(arg)
local requiredOptions = {
"model"
}
onmt.utils.Opt.init(opt, requiredOptions)
_G.logger = onmt.utils.Logger.new(opt.log_file, opt.disable_logs, opt.log_level)
_G.logger:info("Loading model")
local translator = onmt.translate.Translator.new(opt)
server.host = opt.host
server.port = opt.port
_G.logger:info("Server initialized at " .. server.host .. ":" .. server.port)
server.post('/', function(req, res)
-- Input format is a json batch of src strings.
local jsonsource = req.body
local source
if ( opt.debugmode ) then _G.logger:info("Request body (JSON): " .. jsonsource .. "\n") end
local source, pos, err_read_json = json.decode (jsonsource, 1, nil)
-- Currently assuming that it is only one string.
-- The translation supports multiple at once; we can extend.
print(source)
local result
local flatten_result
if err_read_json then
_G.logger:info("Error: " .. err_read_json .. "\n")
-- Hide paths included in the error message.
err_read_json = err_read_json:gsub("/[^:]*/", "")
result = err_read_json
end
if ( opt.debugmode ) then _G.logger:info("Translating: " .. #source .. "\n") end
local _, err_translate = pcall(function ()
if (isTable(source)) then
result = translateMessage(translator, source)
else
result = translateMessage(translator, {{text=source}})
end
end)
if err_translate then
_G.logger:info("Error: " .. err_translate .. "\n")
-- Hide paths included in the error message.
err_translate = err_translate:gsub("/[^:]*/", "")
result = err_translate
end
if ( opt.debugmode ) then _G.logger:info("Translation result: " .. #result .. "\n") end
-- Now add back the translation and print it out
local ret = json.encode(result)
if ( opt.debugmode ) then _G.logger:info("Translation finished.") end
res.send(ret)
collectgarbage()
end)
server.listen()
end
I have modified also the translateMessage/2 function to look for a ‘text’ field, rather than a ‘src’ field.
The code was originally made to work with XML but I have changed it to use json (dkjson). It is not fully tested for json, so apologies for that.
Dimitar.
Thanks, @dimitarsh1! I’ll see if I can give this a try on a docker container this weekend.
Unrelated question… Are you using CPU or GPU translation for your released engines on this server setup?
Thanks from me too @dimitarsh1. Looks like I’m going to have a busy Saturday morning - at least it’s raining here!
Terence
Hello David @dbl, we use both CPU and GPU machines with this server; also both models made for release and the intermediately trained ones from each epoch.
By the way, as the code became large to be placed in hte forum I haven’t included the rest of it, which is pretty much the same as in the translation_server.lua script. If you have any problems integrating it into your code, let me know.
Cheers,
Dimitar
Just head over to the GitHub page, you’ll find appropriate links
@dimitarsh1 Thanks for sharing. It may also be useful to share which version or revision of OpenNMT your script depends on. Things are still moving in the API side and your script may not be compatible with the latest version.
Hi Dimitar @dimitarsh1
Re:
buffer: luarocks install buffer
async: luarocks install async
redis-async: luarocks install redis-async
I’m getting “No results matching query were found” even trying with sudo.
Where are they located?
Thanks,
Terence
Dear Terrance,
here is the github repository for the redis-async.
You can get it from there, although I didn’t need to in any of the installations I tried. For me luarocks install redis-async works.
Hope this works.
Cheers,
Dimitar
The version I have is from the beginning of February. It is versino v 0.3.0.
Kind regards,
Dimitar
I have just installed release OpenNMT v05 and got the rest_translation_server up and running. I’d like to thank everyone who came up with ideas for a solution. In the end the prospect of built-in tokenization/detokenization won the day!
Terence
Glad to hear that.
Just one thing:
If you intedn to use it in CPU mode, I highly recommend to use -beam_size 1 as a cmdline option.
Otherwise it will be slow. (default value is 5 and I tested it, it’s really slow even on fast procs)
I’ve just been trying it in GPU mode and it seems blisteringly fast. I’ll bear your comment in mind if we put the set-up on a server with CPU only talking to the outside world.