I have been using the
target_prefix (that’s different per line) in CTranslate2. I’ve been able to do it using the Python API as such:
translations_tok = translator.translate_batch(source=source_tokenized, target_prefix=prefix_phrases_tok, prefix_bias_beta=0.9999999, min_decoding_length = 1, max_decoding_length = 700, beam_size=10, num_hypotheses=1)
I’ve also been using the OpenNMT-py server to serve about 100 models on demand without support for
target_prefix. However, I’m not sure how I can pass the
target_prefix to the server or if it’s even supported by the OpenNMT-py codebase. Looking at the code, it seems there’s some support for
ct2_translate_batch_args but it seems like it can’t have different target_prefixes on the fly (it depends on the source sentence being translated, not the model being served).
Would appreciate it if someone can help me