I’m new in tensorflow and I have little problem during in-domain adaptation.
I trained my general model with
source_embedding:de.wiki.bpe.d300, so my general model has 300 dimension.
Now when I want to retrain my model with new vocab, I don’t set embedding anymore (I supposed it’s not necessary anymore) I get error which there is dimension diferences between these two model. So if I set previouse embedding, everything works correctly.
How can I set the dimension of my new model to 300? or is it necessary to use previus embedding while i want to adapt my indomain model?
When an embedding file is configured, the embedding size is inferred from this file, otherwise the model defines a default size. For example, the default embedding size for a
TransformerBase model is 512.
If you don’t change the vocabulary size, you can just keep the same embedding configuration for a retraining. The pretrained embeddings are only used for the weight initialization and will be overriden when loading a checkpoint.
However, if the vocabulary size is different, you may need to define a custom model with the correct embedding size. For example, you can create a file
my_transformer.py with this content:
model = MyTransformer
and then use
onmt-main --model my_transformer.py [...]