Test Target length Issue (Always 200)

AASHISHAG · November 22, 2020, 9:59pm

Hello,

Thank you for the wonderful framework.

I have trained a character based translation model using Opennmt-py.

When I use the trained model to get the target sequences for the test set, unfortunately my target length is getting always restricted to 200. Could you please help me on it, so that I can output targets with length > 200?

In onmt/*/misc.py I have already set the max src and target length to 2000 and in YAML i have also added the flag transforms: [‘filtertoolong’].

Since I have character inputs (not words) I have to have longer sequences.

Kindly guide to avoid max length of 200.

francoishernandez · November 23, 2020, 8:13am

Strange that it’s 200. The default max_length is 100:

github.com

OpenNMT/OpenNMT-py/blob/bc95e03875aabf4363161aac70830ecbdb762d91/onmt/opts.py#L660


          default=1., type=float,
          help="If doing random sampling, divide the logits by "
               "this before computing softmax during decoding.")
_add_reproducibility_opts(parser)

group = parser.add_argument_group('Beam Search')
group.add('--beam_size', '-beam_size', type=int, default=5,
          help='Beam size')
group.add('--min_length', '-min_length', type=int, default=0,
          help='Minimum prediction length')
group.add('--max_length', '-max_length', type=int, default=100,
          help='Maximum prediction length.')
group.add('--max_sent_length', '-max_sent_length', action=DeprecateAction,
          help="Deprecated, use `-max_length` instead")

# Alpha and Beta values for Google Length + Coverage penalty
# Described here: https://arxiv.org/pdf/1609.08144.pdf, Section 7
group.add('--stepwise_penalty', '-stepwise_penalty', action='store_true',
          help="Apply penalty at every decoding step. "
               "Helpful for summary penalty.")
group.add('--length_penalty', '-length_penalty', default='none',

AASHISHAG · November 24, 2020, 4:56pm

I mistakenly also counted spaces.

Thanks a lot.