Use gpu in c++ version

Support
1

I want to use the gpu in the c++ ctranslate2. I noticed that there is a variable which defaults to cpu when creating the model with ModelLoader. How can i set the device to cuda?

3

For C++ it should look like this:

ctranslate2::models::ModelLoader model_loader(model_path);
model_loader.device = ctranslate2::Device::CUDA;

ctranslate2::Translator translator(model_loader);
1 Like