Issues when exporting to CTranslate2 from a saved model

Support
1

When I export from a saved model with the converter script, it seems the resulting CTranslate2 model is unusable. Instead, when I export from a checkpoint, the model works fine. Also, the size of the model differs. With int8, the saved model is ~240 MB and the checkpoint model is ~220 MB.
Any hints?

2

I think there is an issue when converting SavedModels with shared embeddings (more specifically when the target input embeddings and output softmax weights are shared).

I will look for a fix. However, I would advise against converting SavedModel. I plan to remove this feature in the next major version of CTranslate2.

3

That makes sense, indeed the model has shared embeddings. No big deal, converting from checkpoints works fine. Thanks!

1 Like
4

For reference this was fixed in:

1 Like