Bleu Script Throws OSError: [Errno 7] Argument list too long: '/bin/sh'

Hi everyone,
While doing inference and trying to compute the bleu score for a dataset that I’m using, I get the following error:

Traceback (most recent call last):
File “”, line 63, in
File “”, line 32, in main
File “/workspace/model_lib/OpenNMTpy/onmt/translate/”, line 391, in translate
msg = self._report_bleu(tgt)
File “/workspace/model_lib/OpenNMTpy/onmt/translate/”, line 833, in _report_bleu
stdin=self.out_file, shell=True
File “/opt/conda/lib/python3.6/”, line 356, in check_output
File “/opt/conda/lib/python3.6/”, line 423, in run
with Popen(*popenargs, **kwargs) as process:
File “/opt/conda/lib/python3.6/”, line 729, in init
restore_signals, start_new_session)
File “/opt/conda/lib/python3.6/”, line 1364, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 7] Argument list too long: ‘/bin/sh’

The command that I’m using to run inference is:
python -model models/zh-en/ -src data/src-test.txt -tgt data/tgt-test.txt -output pred.txt --report_bleu -replace_unk -verbose

Any ideas on what is causing this? That location is at the place where it calls the script and it works fine when I run without the bleu argument.

Not sure what the issue was with, but I found that you can run the script manually with perl tools/multi-bleu.perl data/tgt-test.txt < predPost.txt which works fine.

Yes this is a current issue (#1429 & #1530). There is no consensus yet on the best way to handle this.
You can also use sacrebleu instead of the perl script.