Sorry, my English is not good.
I want to translate English to Japanese.
I create “Linear model” with a linear layer added from the default state.
I extended Models.py.
class RNNEncoder
def init():
self.linear = nn.Linear(500, 500)
def foward():
emb = self.embeddings(src)
emb = self.linear(emb)
I wanted to ascertain the effect of the linear layer.
But, the following happened during learning.
Epoch 7, 7100/14218; acc: 54.66; ppl: 8.99; xent: 2.20; 9200 src tok/s; 11449 tgt tok/s; 1102 s elapsed
Epoch 7, 7150/14218; acc: 54.77; ppl: 8.93; xent: 2.19; 9361 src tok/s; 11479 tgt tok/s; 1110 s elapsed
Epoch 7, 7200/14218; acc: 48.95; ppl: 22.90; xent: 3.13; 9710 src tok/s; 11969 tgt tok/s; 1118 s elapsed
Epoch 7, 7250/14218; acc: 15.71; ppl: 38654.97; xent: 10.56; 10128 src tok/s; 12591 tgt tok/s; 1126 s elapsed
Epoch 7, 7300/14218; acc: 7.25; ppl: 486307.25; xent: 13.09; 10236 src tok/s; 12560 tgt tok/s; 1133 s elapsed
Epoch 7, 7350/14218; acc: 4.37; ppl: 1705051.62; xent: 14.35; 9832 src tok/s; 12094 tgt tok/s; 1141 s elapsed
Epoch 7, 7400/14218; acc: 3.43; ppl: 3982725.34; xent: 15.20; 9037 src tok/s; 11253 tgt tok/s; 1149 s elapsed
Epoch 7, 7450/14218; acc: 3.40; ppl: 7776059.38; xent: 15.87; 9868 src tok/s; 11925 tgt tok/s; 1157 s elapsed
Performance sharply declined.
I want to know why this happened.