The batch will be filled until we reach 1,its size may exceed 16 tokens

Support
1

Hello, congratulations for such great work.

Could you please help to understand this WARNING. Will it have any affect on the training? How can I avoid it?

I am running a transformer model with one GPU setting and a small batch size of 16.

[2020-11-21 20:29:00,185 INFO] Loading ParallelCorpus(phonemes2text_v2/src-val.txt, phonemes2text_v2/tgt-val.txt, align=None)…
[2020-11-21 20:50:33,074 INFO] Validation perplexity: 13.9302
[2020-11-21 20:50:33,074 INFO] Validation accuracy: 25.8812
[2020-11-21 20:50:44,975 WARNING] The batch will be filled until we reach 1,its size may exceed 16 tokens

2

This warning is triggered when using batch_type tokens, if one example has more tokens than batch_size. Do you really want to train on batches of 16 tokens? You probably meant sentences? (batch_type "sents")

1 Like
3

I also am having this issue. I have valid_batch_size: 8, batch_size: 4096, and batch_type: "tokens" which I copied from the transformer documentation. Should I switch to batch_type "sents"?

OpenNMT-py trained to 4000 steps then has this error a number of times:

[2021-02-22 22:01:33,091 WARNING] The batch will be filled until we reach 1,its size may exceed 8 tokens
[2021-02-22 22:01:33,100 WARNING] The batch will be filled until we reach 1,its size may exceed 8 tokens
[2021-02-22 22:01:33,130 WARNING] The batch will be filled until we reach 1,its size may exceed 8 tokens
[2021-02-22 22:14:37,309 WARNING] The batch will be filled until we reach 1,its size may exceed 8 tokens
[2021-02-22 22:14:37,366 WARNING] The batch will be filled until we reach 1,its size may exceed 8 tokens
[2021-02-22 22:27:16,582 WARNING] The batch will be filled until we reach 1,its size may exceed 8 tokens
[2021-02-22 22:27:16,648 WARNING] The batch will be filled until we reach 1,its size may exceed 8 tokens
[2021-02-22 22:27:16,650 WARNING] The batch will be filled until we reach 1,its size may exceed 8 tokens
[2021-02-22 22:27:16,650 WARNING] The batch will be filled until we reach 1,its size may exceed 8 tokens
[2021-02-22 22:27:16,654 WARNING] The batch will be filled until we reach 1,its size may exceed 8 tokens
[2021-02-22 22:27:16,660 WARNING] The batch will be filled until we reach 1,its size may exceed 8 tokens
[2021-02-22 22:27:16,664 WARNING] The batch will be filled until we reach 1,its size may exceed 8 tokens
[2021-02-22 22:27:16,695 WARNING] The batch will be filled until we reach 1,its size may exceed 8 tokens
[2021-02-22 22:40:09,601 WARNING] The batch will be filled until we reach 1,its size may exceed 8 tokens
[2021-02-22 22:40:09,612 WARNING] The batch will be filled until we reach 1,its size may exceed 8 tokens
[2021-02-22 22:40:10,204 WARNING] The batch will be filled until we reach 1,its size may exceed 8 tokens
[2021-02-22 22:40:10,216 WARNING] The batch will be filled until we reach 1,its size may exceed 8 tokens
[2021-02-22 22:40:10,216 WARNING] The batch will be filled until we reach 1,its size may exceed 8 tokens
[2021-02-22 22:40:10,256 WARNING] The batch will be filled until we reach 1,its size may exceed 8 tokens
[2021-02-22 22:40:10,294 WARNING] The batch will be filled until we reach 1,its size may exceed 8 tokens
[2021-02-22 22:53:09,395 WARNING] The batch will be filled until we reach 1,its size may exceed 8 tokens
[2021-02-22 22:53:10,063 WARNING] The batch will be filled until we reach 1,its size may exceed 8 tokens
[2021-02-22 22:53:10,065 WARNING] The batch will be filled until we reach 1,its size may exceed 8 tokens
[2021-02-22 22:53:10,080 WARNING] The batch will be filled until we reach 1,its size may exceed 8 tokens

The GPU usage shown by nvidia-smi dropped to ~50% from ~95% and didn’t make progress in 8 hours of training. onmt_train also shows as using 100% of one CPU core.

# Batching
queue_size: 10000
bucket_size: 32768
world_size: 1
gpu_ranks: [0]
batch_type: "tokens"
batch_size: 4096
valid_batch_size: 8
max_generator_batches: 2
accum_count: [4]
accum_steps: [0]

Full code

4

Your valid_batch_size should also be expressed in tokens.
The valid_batch_size in “tokens” mode was introduced in release 2.0.0, here:

5

Thanks, I switched to valid_batch_size: 4096