How EXACTLY do I run TRANSLATE on a GPU?

paulkp · April 24, 2020, 1:25am

I’m confused. @guillaumekln @francoishernandez

To TRAIN I can use CUDA and:

CUDA_VISIBLE_DEVICES=0,1,2,3 onmt_train -data . . . . -world_size 4 -gpu_ranks 0 1 2 3 &

but TRANSLATE doesn’t have world or gpu options. Does it? It doesn’t as far as I can tell.

Q1. So how EXACTLY do I run translate on a GPU (even a single GPU) if onmt_translate doesn’t have world/gpu options?
Q2. Is it true (according to this forum) that translate only works with one GPU at a time?
Q3. Is it true that translate on a CPU is really slow as I have discovered? eg about 600ms per sentence?

vince62s · April 24, 2020, 7:26am

here

github.com

OpenNMT/OpenNMT-py/blob/master/onmt/opts.py#L766


          help="If verbose is set, will output the n_best "
               "decoded sentences")


group = parser.add_argument_group('Efficiency')
group.add('--batch_size', '-batch_size', type=int, default=30,
          help='Batch size')
group.add('--batch_type', '-batch_type', default='sents',
          choices=["sents", "tokens"],
          help="Batch grouping for batch_size. Standard "
               "is sents. Tokens will do dynamic batching")
group.add('--gpu', '-gpu', type=int, default=-1,
          help="Device to run on")


# Options most relevant to speech.
group = parser.add_argument_group('Speech')
group.add('--sample_rate', '-sample_rate', type=int, default=16000,
          help="Sample rate.")
group.add('--window_size', '-window_size', type=float, default=.02,
          help='Window size for spectrogram in seconds')
group.add('--window_stride', '-window_stride', type=float, default=.01,
          help='Window stride for spectrogram in seconds')

best is to set the cuda_visible_devices on which you want to translate and then -gpu 0

paulkp · April 24, 2020, 11:33pm

So is the -gpu part of the CUDA or the onmt command?
How does CUDA know when the onmt command is finished?
How was I supposed to know to do this if -gpu is not listed in the omt_translate options?

vince62s · April 25, 2020, 6:57am

You may want to overview the full documentation:
https://opennmt.net/OpenNMT-py/extended.html

paulkp · April 25, 2020, 7:01am

I was looking here which shows no -gpu option . .
https://opennmt.net/OpenNMT-py/options/translate.html

Thnx!

vince62s · April 25, 2020, 7:06am

it is here: