Two short questions on training

Support
1

Hi! (This is about opennmt-py)

  1. How could I calculate the number of training “steps” needed for 1 epoch? (An approximate answer is also OK)
  2. How come when I increase the batch size (even by 2x or 4x) the tok/s doesn’t really increase much?

Thank you so much.

1 Like
2

There are two modes in training: sentence batch and tokens batch.
If sentence batch, for instance 64, then it will require total_number_of_sentence / 64 steps to accomplish one epoch.
If token batch mode, you need an approximation, for instance 4096 tokens in one batch, you need an average number of token (words or subwords) per sentence to do the maths.
Just note that there is a verbose option thta will log a message when an epoch is completed.

For your second point, it will require the command lines and a bit more of context.

1 Like