OpenNMT Forum

Core dump while loading the tokenizer

Hi,

I am getting the core dump error while loading the tokenizer ‘pyonmttok’

The config file I used is as follows:

The error I get is as follows:
(OfflineTrans) fujiadmin@offlinetranslation:~/pytorch/OpenNMT-py$ python server.py --ip “0.0.0.0” --port “7786” --url_root “/translator” --config “./available_models/conf_pyonmttok.json”
Pre-loading model 0
[2019-09-12 14:19:26,926 INFO] Loading model 0
[2019-09-12 14:19:28,101 INFO] Loading tokenizer
Segmentation fault (core dumped)
(OfflineTrans) fujiadmin@offlinetranslation:~/pytorch/OpenNMT-py$

Please assist me in resolving this issue.

Thank You,
Kishor.

Make sure to get the latest version:

pip install --upgrade pyonmttok

and then retry.

This was not reported before AFAIK.

Hi,

The core dump has repeated even after upgrading the Tokenizer. Please find the below
message after upgrading :

(OfflineTrans) fujiadmin@offlinetranslation:~$ pip install --upgrade pyonmttok
Collecting pyonmttok
** Downloading https://files.pythonhosted.org/packages/88/f3/889c2129317ae053ab6a9173351d3492222c9ae8eefffc71b95827403932/pyonmttok-1.15.4-cp37-cp37m-manylinux1_x86_64.whl (2.2MB)**
** |████████████████████████████████| 2.2MB 5.4MB/s**
Installing collected packages: pyonmttok
** Found existing installation: pyonmttok 1.15.2**
** Uninstalling pyonmttok-1.15.2:**
** Successfully uninstalled pyonmttok-1.15.2**
Successfully installed pyonmttok-1.15.4
(OfflineTrans) fujiadmin@offlinetranslation:~$ cd pytorch/OpenNMT-py/
(OfflineTrans) fujiadmin@offlinetranslation:~/pytorch/OpenNMT-py$ python server.py --ip “0.0.0.0” --port “7786” --url_root “/translator” --config "./available_models/conf_pyonmttok.json"
Pre-loading model 0
[2019-09-16 06:39:58,783 INFO] Loading model 0
[2019-09-16 06:39:59,941 INFO] Loading tokenizer
Segmentation fault (core dumped)
(OfflineTrans) fujiadmin@offlinetranslation:~/pytorch/OpenNMT-py$

Please give any suggestions to resolve this ?

Thank You,
Kishor.

Can you upgrade again (to version 1.15.5) and retry?