OpenNMT-py Server + CTranslate2 target_prefix support?

Support
1

I have been using the target_prefix (that’s different per line) in CTranslate2. I’ve been able to do it using the Python API as such:

translations_tok = translator.translate_batch(source=source_tokenized,
                                                  target_prefix=prefix_phrases_tok,
                                                  prefix_bias_beta=0.9999999,
                                                  min_decoding_length = 1,
                                                  max_decoding_length = 700,
                                                  beam_size=10,
                                                  num_hypotheses=1)

I’ve also been using the OpenNMT-py server to serve about 100 models on demand without support for target_prefix. However, I’m not sure how I can pass the target_prefix to the server or if it’s even supported by the OpenNMT-py codebase. Looking at the code, it seems there’s some support for ct2_translate_batch_args but it seems like it can’t have different target_prefixes on the fly (it depends on the source sentence being translated, not the model being served).

Would appreciate it if someone can help me :slight_smile:

2

Dear James,

It seems the target_prefix option was supported by this PR.

Could you please try it and then report the exact issue you have, if any.

All the best,
Yasmin

3

Thanks, got it to work!
Generally, this code snippet will work:
The prefix token needs to be stored in “ref”

sourcetokenList.append({"src": tokenizedsrc, "ref": prefix_phrases_tok, "id":100})
url = "http://127.0.0.1:3000/translator/translate"
headers = {"Content-Type": "application/json"}
response = requests.post(url, json=sourcetokenList, headers=headers)

The OpenNMT-py conf.json needs to have "tgt_prefix": true in the “opt” section to turn on this functionality.

1 Like