Preprocess with -dynamic_dict does not create a valid data file?

Support
1

I am trying to train a model with the following flags:

-copy_attn -copy_attn_force -copy_loss_by_seqlength -coverage_attn

My understanding is that I need to preprocess the data files with the flag

-dynamic_dict

However, if I got a tiny preprocessed file of only 137 bits, ending with .train.1.pt. There are no .valid.1.pt or .vocab.pt files. I wonder why this happens.

If I do not provide the flag of > -dynamic_dict, everything works perfectly but I cannot train with the coverage loss and copy mechanism.

2

I found it’s because I ran out of memory. Adding “max_shard_size” solved my problem.