I have created a ctranslate2 optimized model and tried to get inferences from the converted model. I have used sentence piece model to encode the segments before passing it to model for inference.
I am getting “unk” token at the end of every segment in the output of ctranslate2.
Example: (I am showing mock data here)
Some_ text_ here_ < ukn >
Nex_t_ line_ < ukn >
Note: I am using default values for all parameters in model inference API. Using a docker image to generate inferences from windows machine.