OpenNMT-py- NMT class description

nehasoni3 · January 20, 2021, 12:12pm

Hi,
I have found that NMT class of OpenNMT-py has (src,tgt,length) in its forward method.

OpenNMT/OpenNMT-py/blob/7e1556fcaeab154defa14b52f02b3a8c539ca0bd/onmt/models/model.py#L14


class BaseModel(nn.Module):
    """
    Core trainable object in OpenNMT. Implements a trainable interface
    for a simple, generic encoder / decoder or decoder only model.
    """
    def __init__(self, encoder, decoder):
        super(BaseModel, self).__init__()
    def forward(self, src, tgt, lengths, bptt=False, with_align=False):
        """Forward propagate a `src` and `tgt` pair for training.
        Possible initialized with a beginning decoder state.
        Args:
            src (Tensor): A source sequence passed to encoder.
                typically for inputs this will be a padded `LongTensor`
                of size ``(len, batch, features)``. However, may be an
                image or other generic input depending on encoder.
            tgt (LongTensor): A target sequence passed to decoder.
                Size ``(tgt_len, batch, features)``.

When I do inferencing using Library — OpenNMT-py documentation. I have found that src have input text but tgt is NONE. How do I evaluate/pass tgt to forward().
Please help me to figure these src,tgt and length?

francoishernandez · January 21, 2021, 6:00pm

By default, TGT is not needed to perform inference. Indeed, the goal of inference is to produce a TGT sequence…
This arg is there to allow gold score computation. See:

nehasoni3 · January 22, 2021, 10:33am

OK. So decoder uses this TGT for scoring.
I am trying to convert this OpenNMT-py model into ONNX. When I try to convert, it asked me about(src,tgt,length) as dummy input.

src and length can be obtained, but How does this TGT.
@francoishernandez @guillaumekln Can you guide me in this regard.

francoishernandez · January 22, 2021, 10:59am

It’s defined in the code you cited:

            tgt (LongTensor): A target sequence passed to decoder.
                Size ``(tgt_len, batch, features)``.

As for ONNX export, I’m not sure how realistic this would be, as you would need to redefine quite a bit of logic to use the model.
Have you tried CTranslate2?

nehasoni3 · January 22, 2021, 12:11pm

Yes, I have used the Ctranslate2. But issue is, it is not worked on arm(Pi).

francoishernandez · January 22, 2021, 1:33pm

Have you checked this?

nehasoni3 · February 9, 2021, 7:35am

Yes, It not works.