When I run
> cd ~/tds/OpenNMT/tools
> th tokenize.lua ende_srcs.txt ende_srcs.rawtok.txt
I get an error:
/home/wiktorstribizew/torch/install/bin/luajit: ...ktorstribizew/torch/install/share/lua/5.1/trepl/init.lua:389: module 'onmt.init' not found:No LuaRocks module found for onmt.init
no field package.preload['onmt.init']
no file '/home/wiktorstribizew/.luarocks/share/lua/5.1/onmt/init.lua'
no file '/home/wiktorstribizew/.luarocks/share/lua/5.1/onmt/init/init.lua'
no file '/home/wiktorstribizew/torch/install/share/lua/5.1/onmt/init.lua'
no file '/home/wiktorstribizew/torch/install/share/lua/5.1/onmt/init/init.lua'
no file './onmt/init.lua'
no file '/home/wiktorstribizew/torch/install/share/luajit-2.1.0-beta1/onmt/init.lua'
no file '/usr/local/share/lua/5.1/onmt/init.lua'
no file '/usr/local/share/lua/5.1/onmt/init/init.lua'
no file '/home/wiktorstribizew/torch/install/share/lua/5.2/onmt/init.lua'
no file '/home/wiktorstribizew/torch/install/share/lua/5.2/onmt/init/init.lua'
no file '/home/wiktorstribizew/.luarocks/lib/lua/5.1/onmt/init.so'
no file '/home/wiktorstribizew/torch/install/lib/lua/5.1/onmt/init.so'
no file '/home/wiktorstribizew/torch/install/lib/onmt/init.so'
no file './onmt/init.so'
no file '/usr/local/lib/lua/5.1/onmt/init.so'
no file '/usr/local/lib/lua/5.1/loadall.so'
no file '/home/wiktorstribizew/torch/install/lib/lua/5.2/onmt/init.so'
no file '/home/wiktorstribizew/.luarocks/lib/lua/5.1/onmt.so'
no file '/home/wiktorstribizew/torch/install/lib/lua/5.1/onmt.so'
no file '/home/wiktorstribizew/torch/install/lib/onmt.so'
no file './onmt.so'
no file '/usr/local/lib/lua/5.1/onmt.so'
no file '/usr/local/lib/lua/5.1/loadall.so'
no file '/home/wiktorstribizew/torch/install/lib/lua/5.2/onmt.so'
stack traceback:
[C]: in function 'error'
...ktorstribizew/torch/install/share/lua/5.1/trepl/init.lua:389: in function 'require'
tokenize.lua:2: in main chunk
[C]: in function 'dofile'
...izew/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x00406240
Note that I am using Lua 5.2:
lua -v
Lua 5.2.3 Copyright (C) 1994-2013 Lua.org, PUC-Rio
While following the Installation instructions, I could not make
luarocks install tds
work. It returned
Warning: Failed searching manifest: Failed fetching manifest for https://raw.githubusercontent.com/torch/rocks/master - Failed downloading https://raw.githubusercontent.com/torch/rocks/master/manifest - /home/wiktorstribizew/.cache/luarocks/https___raw.githubusercontent.com_torch_rocks_master/manifest
Warning: Failed searching manifest: Failed fetching manifest for https://raw.githubusercontent.com/rocks-moonscript-org/moonrocks-mirror/master - Failed downloading https://raw.githubusercontent.com/rocks-moonscript-org/moonrocks-mirror/master/manifest - /home/wiktorstribizew/.cache/luarocks/https___raw.githubusercontent.com_rocks-moonscript-org_moonrocks-mirror_master/manifest
Error: No results matching query were found.
I could not work around it - I tried all suggestions there are on this Web site, StackOverflow, and other sites (like in Tds not found when running preprocess.lua). I could not make it work the way described at Installing Torch7 and OpenNMT On Ubuntu 14.04. The solution here worked:
git clone https://github.com/torch/tds.git
cd tds
luarocks make rocks/tds-scm-1.rockspec
Now, running luarocks command, I get
CONFIGURATION
Lua version: 5.2
Configuration files:
System: /home/wiktorstribizew/torch/install/etc/luarocks/config.lua (ok)
User : /home/wiktorstribizew/.luarocks/config-5.2.lua (not found)
Rocks trees in use:
/home/wiktorstribizew/.luarocks
/home/wiktorstribizew/torch/install
No idea how to proceed. I can run
-
preprocess.luascript -
train.luascript
Basically, I tried the demo training for ENDE and was going to try in-domain training using the OpenNMT ENDE model and some in-domain corpus, and that is where I am stuck right now.
P.S. In addition to the answer marked as solved:
- Had to uninstall Torch (
cd /; sudo rm -rf ~/torch) and reinstall (cd ~/torch; git clone --recursive https://github.com/torch/distro.git ~/torch) - Since after I ran
./install.shI saw many errors, I had to install Moses manually:cd ~/torch; git clone git://github.com/Yonaba/Moses.git; cd ~/torch/Moses; luarocks make rockspec/moses-1.6.1-1.rockspec; cd ~/torch; ./install.shbut there were errors again:luarocks install bit32output errors - Then I had to install bits32 manually as explained in the accepted answer, but before, I ran
export {http,https,ftp}_proxy='http://LOGIN:PASSWORD@PROXY_IP:PORTand replaced;in my password with%3B - To run
tools/tokenize.lua, I needed to run it from theOpenNMTfolder usingth tools/tokenize.lua < input.txt > output.txt
