Question about ctranslate2

Support
,
1

Hi,

I have made some changes with OpenNMT-tf, such as removing some classes and methods, changing package names, and refining some methods.

In order to convert the model trained with new codes into ctranslate2 format, I modified class ctranslate2.converters.OpenNMTTFConverterV2 by changing openmt into my own package, and finally I converted my model into ctranslate2 format. In this process, no error occured, and model.bin together with two vocab files are produced.

But when I loaded ctranslate2 model with ctranslate2.Translator, the following error occured:

Blockquote
File “D:\kidden\mt\open\github\mt-core\yimt\api\translator.py”, line 278, in init
self.translator = ctranslate2.Translator(
RuntimeError: CUDA failed with error out of memory

Is there some information about package names in the converted ctranslate2 model?

Thanks!

2

Hello,

One of the reason could be that your trying to translate to many segments in one call.

You need to control how many segments you send at once (in batch mode) to ctranslate2 otherwise you will hit memory issue.

Best regards,
Samuel

1 Like
3

Thank you!

I just wanted to load ctranslate2 model using ctranslate2.Translator class, and during loading the error occurred.

I don’t read the cpp source codes of ctranslate2, and guess that there is some information about package names in the converted ctranslate2 model or model loading, because I changed some package names in OpenNMT-tf and ctranslate2.converters.OpenNMTTFConverterV2 class and the model was converted correctly.

I don’t know whether my guess is right or not.

Best regards,
Liu Xiaofeng

4

Have you tried loading the real ctranslate2? Just to see if you get the same error.

Best regards,
Samuel

5

Yes, I have tried it.

If I load the ctranslate2 model converted from checkpoints trained with OpenNMT-tf , there is no problem.

Best regards,
Liu Xiaofeng

6

Hi,

The converted model does not contain any information about Python modules.

Can you try loading the model on the CPU and check if the memory usage is normal?

1 Like
7

Thank you for your advice!

I will try it soon.

8

Hi,

After I wrote a seperate program to load ctranslate2 model, I found it can be loaded on CUDA correctly and worked well.

I checked my enviroment (I deployed several ctranslate2 models on a Flask Web server) and guess that the error may be due to my private envoriment.

Thank you for your patience and great work!

Best regards,
Liu Xiaofeng