Ctranslate2 gives KeyError: 'vocab' when translating HF Llama2 model

Support
1

Hi,

I have fine-tuned a Llama2 model from Meta with QLORA. After merging and saving the weights (model.merge_and_unload()), I am trying to convert it with ct2-transformers-converter --model merged/pytorch_model.bin --output_dir ct2_model. This gives me KeyError: 'vocab'. Why is Ctranslate2 expecting the model binary to include a vocab?

2

See the description of the option --model:

Name of the pretrained model to download, or path to a directory containing the pretrained model.

So you should try passing merged/ to this option, not the checkpoint file.

3

Tried that. Gives an error something along the lines merged/ is a directory