Config default values

Site Feedback
1

Hello!

In the documentation under “train”, it says:

  # (optional) Batch size is the number of "examples" or "tokens" (default: "examples").
  batch_type: examples
  # (optional) Tune gradient accumulation to train with at least this effective batch size
  # (default: null).
  effective_batch_size: 25000

I think the default batch_type for training is tokens (not examples), and the default effective_batch_size is 25000 (not null).

Many thanks,
Yasmin

2

Hi,

These default values are correct when not using --auto_config. When --auto_config is used, each model may come with different default values so they are not included in the documentation.

1 Like