Bad alignement line exists. Please check file!

Jie · August 20, 2020, 8:07am

hello

I use fast_ Align generates alignment files.

Pretreatment increase --train_align --valid_align

Complete command:
onmt_preprocess -src_seq_length 260 -tgt_seq_length 260 -filter_valid -train_align train_align.txt -valid_align valid_align.txt -train_src tgt-train.txt -train_tgt src-train.txt -tgt-val.txt -valid_tgt src-val.txt -save_data pretreatment/demo -overwrite

But get wrong:

I don’t know where the problem is. Check that there are no blank lines in the file.
Asking for help.

francoishernandez · August 20, 2020, 12:33pm

You probably have a blank alignment line.

github.com

OpenNMT/OpenNMT-py/blob/eaaae6a7a40fd276a6ab6814cd4afe128cc63496/onmt/inputters/inputter.py#L92-L97


try:
    src_idx, tgt_idx = align.split('-')
except ValueError:
    logger.warning("{} in `{}`".format(align, align_pharaoh))
    logger.warning("Bad alignement line exists. Please check file!")
    raise

Jie · August 24, 2020, 10:01am

Thank you for your help. My problem has been solved.