I have found that NMT class of OpenNMT-py has (src,tgt,length) in its forward method.
class BaseModel(nn.Module): """ Core trainable object in OpenNMT. Implements a trainable interface for a simple, generic encoder / decoder or decoder only model. """ def __init__(self, encoder, decoder): super(BaseModel, self).__init__() def forward(self, src, tgt, lengths, bptt=False, with_align=False): """Forward propagate a `src` and `tgt` pair for training. Possible initialized with a beginning decoder state. Args: src (Tensor): A source sequence passed to encoder. typically for inputs this will be a padded `LongTensor` of size ``(len, batch, features)``. However, may be an image or other generic input depending on encoder. tgt (LongTensor): A target sequence passed to decoder. Size ``(tgt_len, batch, features)``.
When I do inferencing using
Library — OpenNMT-py documentation. I have found that src have input text but tgt is NONE. How do I evaluate/pass tgt to forward().
Please help me to figure these src,tgt and length?
By default, TGT is not needed to perform inference. Indeed, the goal of inference is to produce a TGT sequence…
This arg is there to allow gold score computation. See:
I understand that the target sentence is passed through the decoder part of the model and then the output from that step is pass through the generator (liner + softmax function) to obtain log probability- and the gold score is related to this probability. But what exactly does it signify? Should the gold score be better (lower) than the pred score?
OK. So decoder uses this TGT for scoring.
I am trying to convert this OpenNMT-py model into ONNX. When I try to convert, it asked me about(src,tgt,length) as dummy input.
src and length can be obtained, but How does this TGT.
@francoishernandez @guillaumekln Can you guide me in this regard.
It’s defined in the code you cited:
tgt (LongTensor): A target sequence passed to decoder.
Size ``(tgt_len, batch, features)``.
As for ONNX export, I’m not sure how realistic this would be, as you would need to redefine quite a bit of logic to use the model.
Have you tried CTranslate2?
Yes, I have used the Ctranslate2. But issue is, it is not worked on arm(Pi).