Is it possible to convert OpenNMT model to TensorRT?

Support
1

I can see examples where people are using tensorflow model and convert it to tensorRT to speedup the inference process. Is it possible to do that with OpenNMT model.

Thanks

2

NVIDIA did it with a OpenNMT-lua model and used it in TensorRT benchmarks. Unfortunately it required custom CUDA code for decoding that is not available in the public TensorRT release.

1 Like
3

Yes I can see that in their comparison table. Thanks so much