How to calculate the number of epochs?

I am new to ONMT, and am a bit confused about the number of epochs and training steps. I know the number of epochs is the number of times that we iterate through the training dataset, but how can I calculate the number of epochs that I trained my model with? e.g., let’s say I have 20000 training samples and my batch_size is 64 and had 3200 training steps. If this is the case, then what is the number of epochs?


Your calculation is relatively correct. You will have also to take into consideration “gradient accumulation”. In OpenNMT-py, this is determined by the accum_count option. In OpenNMT-tf, this is determined by the effective_batch_size option.

Note also that if your batch_type is examples, this is easier to determine than when it is tokens. In the case of tokens, it can only be a rough estimation.

There are a couple of good answers on the forum explaining the topic, like this one.

To force training on epochs rather than steps, you can use the option single_pass, supported by both OpenNMT-tf and OpenNMT-py.

Kind regards,

1 Like