I am new to ONMT, and am a bit confused about the number of epochs and training steps. I know the number of epochs is the number of times that we iterate through the training dataset, but how can I calculate the number of epochs that I trained my model with? e.g., let’s say I have 20000 training samples and my batch_size is 64 and had 3200 training steps. If this is the case, then what is the number of epochs?

Hello!

Your calculation is relatively correct. You will have also to take into consideration “gradient accumulation”. In OpenNMT-py, this is determined by the `accum_count`

option. In OpenNMT-tf, this is determined by the `effective_batch_size`

option.

Note also that if your `batch_type`

is `examples`

, this is easier to determine than when it is `tokens`

. In the case of tokens, it can only be a rough estimation.

There are a couple of good answers on the forum explaining the topic, like this one.

To force training on epochs rather than steps, you can use the option `single_pass`

, supported by both OpenNMT-tf and OpenNMT-py.

https://opennmt.net/OpenNMT-tf/faq.html#how-to-count-the-number-of-epochs

Kind regards,

Yasmin

1 Like