Retraining model with new datasets

[relyer]

We have a trained model on basic dateset, languages dnt matter.
When we try to retrain model with new dataset (for example with medical pairs) blue decreased dramatically for basic dataset validation after firts iteration. Where i can found docs, wp or something else with a right path of retrain and Idealistically without basic datasets presence.

[guillaumekln (Guillaume Klein)]

You are looking for domain adaptation techniques.

See this tutorial by @ymoslem:

Domain Adaptation Techniques

Domain Adaptation can be useful when you want to train a Machine Translation model, but you have only too limited data. There are several approaches of Domain Adaptation including: Incremental Training / Re-training: So you have a big pre-trained model trained on a big corpus, and you continue training it with the new data from the small corpus. Ensemble Decoding (of two models): You have two models and you use both models during translation. Combining Training Data: You merge the two c…

[relyer]

@guillaumekln thx for answer

Incremental Training encounter a case of “catastrophic forgetting”, after that case this topic was created
Ensemble Decoding are not implemented yet.
Combining Training more useful but not for us case (we a’nt basic datasets).

What another type of Domain Adaptation can you advise in case of use ct2 without basic datasets. Maybe someone came across this.