System and reference streams have different lengths


I am using the Open NMT translating the test set with the following command and my checking point. It generated system steams as length 2134. However, the length of refs is 3004. Thus, it will produce the error when using the sacrebleu. Is there any reasons for this? However if I change my checkpoint to other model I trained, the error dispears. If possible, is there any reasons for this? Thanks very much!

for checkpoint in data/wmt/run/model_step*.pt; do
echo “# Translating with checkpoint $checkpoint”
base=$(basename $checkpoint)
-gpu 0
-batch_size 16384 -batch_type tokens
-beam_size 5
-model $checkpoint
-src data/wmt/test.en.sp
-tgt data/wmt/
-output data/wmt/${base%.*}.sp


Can you check if you have empty predictions in your output file?