Why my BLEU Score falls ,lets say if I continue retraining over 200k epochs
It can depend on many things. You should be more specific if you want help or explanations.
@guillaumekln
Basically My corpus lets say of size 2m is composed of two domain. 1M from general day to day sentences and 1M from Judicial Domain.
I also have two categories of test sentences first is from general sentences 2.5k sentences and the other from judiciary domain 5k sentences.
Lets say I train my model upto 200k steps and I get a BLEU score of 0.19 on general sentences and 0.24 on Judiciary. If I continue training upto 250k steps my BLEU for general falls to 0.17 and it increases for judiciary to 0.25. What is the reason for this behaviour? PS: I have my data shuffled before training.
Secondly, with training accuracy, shouldn’t it be like it should increase and then stagnate. What happens in my case is it increases lets say upto 70% accuracy when I train upto 200k then at 210k it falls to 68% and then again at 220k it increases to 71%. What could be the reason for this?