OpenNMT Forum

How EXACTLY do I run TRANSLATE on a GPU?

I’m confused. @guillaumekln @francoishernandez

To TRAIN I can use CUDA and:

CUDA_VISIBLE_DEVICES=0,1,2,3 onmt_train -data . . . . -world_size 4 -gpu_ranks 0 1 2 3 &

but TRANSLATE doesn’t have world or gpu options. Does it? It doesn’t as far as I can tell.

Q1. So how EXACTLY do I run translate on a GPU (even a single GPU) if onmt_translate doesn’t have world/gpu options?
Q2. Is it true (according to this forum) that translate only works with one GPU at a time?
Q3. Is it true that translate on a CPU is really slow as I have discovered? eg about 600ms per sentence?

here

best is to set the cuda_visible_devices on which you want to translate and then -gpu 0

1 Like

So is the -gpu part of the CUDA or the onmt command?
How does CUDA know when the onmt command is finished?
How was I supposed to know to do this if -gpu is not listed in the omt_translate options?

You may want to overview the full documentation:
https://opennmt.net/OpenNMT-py/extended.html

1 Like

I was looking here which shows no -gpu option . .
https://opennmt.net/OpenNMT-py/options/translate.html

Thnx!

it is here:

image

1 Like