OpenNMT Forum

Combining char embeddings with contextualized/static pretrained word embeddings in opennmt-py

Hi everyone!

I want to combine char embeddings with word embeddings (preferably BERT) in opennmt-py.

It is my understanding that there is a way to combine different types of embeddings in opennmt-ft, e.g. as discussed in these threads:


Is there something similar in opennmt-py, preferably with support of BERT embeddings?

Hi there,

There are a few bricks for using pre-trained embeddings, but you may need to adapt a few things (conversion scripts, etc.)
It would not be combing several types of embeddings though.