Train process doesn't output any model

[cmatosv (Claudia)]

Hello!

I’m training models using opennmt-py but after the training process ends I don’t get any model. I’ve done the process before and I got model files, however since I changed to another server I don’t get the models.
I think the training process is working because in my log files I get all the outputs of the process, but not the models files.

Can you help me with that?
Kind regards!
Claudia

[guillaumekln (Guillaume Klein)]

Hi,

You should post the full command line you used for the training.

[cmatosv (Claudia)]

Hi! This is the command line I’m using

nohup python train.py -data sonar_data/sonar-prepro -save_model sonar-model -save_checkpoint_steps 500 -gpu_ranks 1 -log_file logfilemodel2 &> sonar_data/training2.txt &

Cheers
Claudia

[guillaumekln (Guillaume Klein)]

See for example:

Unable to use -save_checkpoint_steps with gpu enabled

Hi ! On Opennmt-py, the `-save_checkpoint_steps’ option does not work with the -gpu_ranks 1 option enabled. If I switch to plain cpu (by removing the gpu_ranks option) it does work. Can someone share inputs on how to make it work with the gpu, because on the cpu it takes a really long time. Thanks!

[cmatosv (Claudia)]

Hi Guillaume

I don’t get an error while running the command. My problem is that I don’t get the model files for using in translation. But the training process runs OK because in my log file I can see the models accuracy and ppl.

[guillaumekln (Guillaume Klein)]

Do you have a single GPU? Then use -gpu_ranks 0.
Do you have multiple GPUs? Then use CUDA_VISIBLE_DEVICES=1 python train.py -gpu_ranks 0

[cmatosv (Claudia)]

Oh now its working! Thanks