OpenNMT

OpenNMT-py Server + CTranslate2 target_prefix support?

I have been using the target_prefix (that’s different per line) in CTranslate2. I’ve been able to do it using the Python API as such:

translations_tok = translator.translate_batch(source=source_tokenized,
                                                  target_prefix=prefix_phrases_tok,
                                                  prefix_bias_beta=0.9999999,
                                                  min_decoding_length = 1,
                                                  max_decoding_length = 700,
                                                  beam_size=10,
                                                  num_hypotheses=1)  

I’ve also been using the OpenNMT-py server to serve about 100 models on demand without support for target_prefix. However, I’m not sure how I can pass the target_prefix to the server or if it’s even supported by the OpenNMT-py codebase. Looking at the code, it seems there’s some support for ct2_translate_batch_args but it seems like it can’t have different target_prefixes on the fly (it depends on the source sentence being translated, not the model being served).

Would appreciate it if someone can help me :slight_smile:

Dear James,

It seems the target_prefix option was supported by this PR.

Could you please try it and then report the exact issue you have, if any.

All the best,
Yasmin

Thanks, got it to work!
Generally, this code snippet will work:
The prefix token needs to be stored in “ref”

sourcetokenList.append({"src": tokenizedsrc, "ref": prefix_phrases_tok, "id":100})
url = "http://127.0.0.1:3000/translator/translate"
headers = {"Content-Type": "application/json"}
response = requests.post(url, json=sourcetokenList, headers=headers)

The OpenNMT-py conf.json needs to have "tgt_prefix": true in the “opt” section to turn on this functionality.

1 Like