OpenNMT Forum

OOM error during validation in Transformer model


(Pranjali Basmatkar) #1

I am following the tutorial to train a transformer for NMT. I get the OOM error from the GPU at the time of validation i.e. after every 1000 steps. I am providing a large training dataset and pre-trained embeddings.
I have tried lowering the training batch_size to 512 and the validation batch_size to 32.
I am using a GTX 1080 8 GB GPU.
How do we resolve this?

Thanks in advance

(Guillaume Klein) #2


What is the size of your vocabulary?

(Pranjali Basmatkar) #3

I used the sentencepiece model to create the vocab file and my vocab size is 32000.