Model's retraining

(Sergey Zhitansky) #1


For example, I’ve created model based on some sources. After that I got additional materials and I know that I can run training started from latest Epoch. How I can extend model by additional vocabulary.

In other words - can model be retrained by additional materials ?

Many thanks !

(Guillaume Klein) #2


You can’t change the vocabulary of a trained model. A common approach is to use sub-tokenization like BPE to have an “open” vocabulary.

See for example: