I deployed a Translation Server and made some codes to enable this server can be accessed remotely as following (C++ translator section):
As I know the input source string should be tokenized first and translation should be detokenized,
My question is: If the tokenization for Source and detokenization for translation can be executed automatically by the Translation Server? Or we need extra codes to tokenize the source before sending to Translation server and detokize the returned translation?
It seems not refer to my testing:
I tokenize the source string manually - “This ’ is a testing only .”, the translation can be returned without unk, but the translation seems NOT detokenized.
I did NOT tokenize the source string - “This’s a testing only.”, the translation returned with several unk, and translation seems NOT detokenized also.
Please correct me if wrong.