Is it possible to convert OpenNMT model to TensorRT?


(Md Asadul Islam) #1

I can see examples where people are using tensorflow model and convert it to tensorRT to speedup the inference process. Is it possible to do that with OpenNMT model.

Thanks


(Guillaume Klein) #2

NVIDIA did it with a OpenNMT-lua model and used it in TensorRT benchmarks. Unfortunately it required custom CUDA code for decoding that is not available in the public TensorRT release.


(Md Asadul Islam) #3

Yes I can see that in their comparison table. Thanks so much