Ctranslate2 conversion error

I tried to convert the opennmt-py example provided here https://pypi.org/project/ctranslate2/#installation, then I got this error, any help?

Traceback (most recent call last):
File “/usr/local/bin/ct2-opennmt-py-converter”, line 8, in
File “/usr/local/lib/python3.6/dist-packages/ctranslate2/bin/opennmt_py_converter.py”, line 11, in main
File “/usr/local/lib/python3.6/dist-packages/ctranslate2/converters/converter.py”, line 40, in convert_from_args
File “/usr/local/lib/python3.6/dist-packages/ctranslate2/converters/converter.py”, line 52, in convert
src_vocab, tgt_vocab = self._load(model_spec)
File “/usr/local/lib/python3.6/dist-packages/ctranslate2/converters/opennmt_py.py”, line 22, in _load
checkpoint = torch.load(self._model_path, map_location=“cpu”)
File “/usr/local/lib/python3.6/dist-packages/torch/serialization.py”, line 593, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File “/usr/local/lib/python3.6/dist-packages/torch/serialization.py”, line 773, in _legacy_load
result = unpickler.load()
File “/usr/local/lib/python3.6/dist-packages/torchtext/vocab.py”, line 119, in setstate
if state[‘unk_index’] is None:
KeyError: ‘unk_index’



It’s a bug in torchtext 0.4.0. You need to update this package.

1 Like

Problem Solved!
Thank you.

I upgraded to torchtext-0.6 . But I get the following error while converting a transformer model:
File “/home/lto/.local/lib/python3.7/site-packages/ctranslate2/converters/converter.py”, line 52, in convert
src_vocab, tgt_vocab = self._load(model_spec)
File “/home/lto/.local/lib/python3.7/site-packages/ctranslate2/converters/opennmt_py.py”, line 24, in _load
variables[“generator.weight”] = checkpoint[“generator”][“0.weight”]
KeyError: ‘0.weight’
Any help is appreciated.

What type of model is it? It does not look like it is compatible with CTranslate2.

It is a transformer model from openNMT-py and normally works fine.

Not all models are compatible with CTranslate2. See https://github.com/OpenNMT/CTranslate2#converting-models

yes, I am following this page.

If you want more help, please post the full command line that you used for training.

Hi Guullaume,
I updated to openNMT-py 1.1.1 and Pytorch 1.4.
I used onmt_release_model as you suggested. It works now. Yes, I used int16 (float16 gives error).
Thanks a lot.