Bad score when Linear Model

Support
1

Sorry, my English is not good.
I want to translate English to Japanese.
I create “Linear model” with a linear layer added from the default state.

I extended Models.py.

class RNNEncoder
def init():
self.linear = nn.Linear(500, 500)
def foward():
emb = self.embeddings(src)
emb = self.linear(emb)

I wanted to ascertain the effect of the linear layer.
But, the following happened during learning.

Epoch 7, 7100/14218; acc: 54.66; ppl: 8.99; xent: 2.20; 9200 src tok/s; 11449 tgt tok/s; 1102 s elapsed
Epoch 7, 7150/14218; acc: 54.77; ppl: 8.93; xent: 2.19; 9361 src tok/s; 11479 tgt tok/s; 1110 s elapsed
Epoch 7, 7200/14218; acc: 48.95; ppl: 22.90; xent: 3.13; 9710 src tok/s; 11969 tgt tok/s; 1118 s elapsed
Epoch 7, 7250/14218; acc: 15.71; ppl: 38654.97; xent: 10.56; 10128 src tok/s; 12591 tgt tok/s; 1126 s elapsed
Epoch 7, 7300/14218; acc: 7.25; ppl: 486307.25; xent: 13.09; 10236 src tok/s; 12560 tgt tok/s; 1133 s elapsed
Epoch 7, 7350/14218; acc: 4.37; ppl: 1705051.62; xent: 14.35; 9832 src tok/s; 12094 tgt tok/s; 1141 s elapsed
Epoch 7, 7400/14218; acc: 3.43; ppl: 3982725.34; xent: 15.20; 9037 src tok/s; 11253 tgt tok/s; 1149 s elapsed
Epoch 7, 7450/14218; acc: 3.40; ppl: 7776059.38; xent: 15.87; 9868 src tok/s; 11925 tgt tok/s; 1157 s elapsed

Performance sharply declined.
I want to know why this happened.

2

Possibly a gradient explosion. Are you clipping the gradients?

3

Yes.

My model has “-max_grad_norm” option.
Opts.py

‘-max_grad_norm’, type=float, default=5, help=""“If the norm of the gradient vector exceeds this, renormalize it to have the norm equal to max_grad_norm”""

Optim.py:

if self.max_grad_norm:
clip_grad_norm(self.params, self.max_grad_norm)

I am using the default settings.