Cnn encoder + rnn decoder, got failed

pytorch

(Arvid) #1

I use OpenNMT-py to build a convolutional-Decoder and rnn decoder model, but got wrong as show below, can anyone tell me why?
I already set bridge to be True(an additional layer between the last encoder state and the first decoder state)
NMTModel(

(encoder): CNNEncoder(
   (embeddings): Embeddings(
     (make_embedding): Sequential(
       (emb_luts): Elementwise(
         (0): Embedding(41809, 500, padding_idx=1)
       )
     )
   )
   (linear): Linear(in_features=500, out_features=1000, bias=True)
   (cnn): StackedCNN(
     (layers): ModuleList(
       (0): GatedConv(
         (conv): WeightNormConv2d(1000, 2000, kernel_size=(3, 1), stride=(1, 1), padding=(1, 0))
         (dropout): Dropout(p=0.3)
       )
       (1): GatedConv(
         (conv): WeightNormConv2d(1000, 2000, kernel_size=(3, 1), stride=(1, 1), padding=(1, 0))
         (dropout): Dropout(p=0.3)
       )
       (2): GatedConv(
         (conv): WeightNormConv2d(1000, 2000, kernel_size=(3, 1), stride=(1, 1), padding=(1, 0))
         (dropout): Dropout(p=0.3)
       )
       (3): GatedConv(
         (conv): WeightNormConv2d(1000, 2000, kernel_size=(3, 1), stride=(1, 1), padding=(1, 0))
         (dropout): Dropout(p=0.3)
       )
     )
   )
 )
 (decoder): InputFeedRNNDecoder(
   (embeddings): Embeddings(
     (make_embedding): Sequential(
       (emb_luts): Elementwise(
         (0): Embedding(49653, 500, padding_idx=1)
       )
     )
   )
   (dropout): Dropout(p=0.3)
   (rnn): StackedLSTM(
     (dropout): Dropout(p=0.3)
     (layers): ModuleList(
       (0): LSTMCell(1500, 1000)
       (1): LSTMCell(1000, 1000)
       (2): LSTMCell(1000, 1000)
       (3): LSTMCell(1000, 1000)
     )
   )
   (attn): GlobalAttention(
     (linear_in): Linear(in_features=1000, out_features=1000, bias=False)
     (linear_out): Linear(in_features=2000, out_features=1000, bias=False)
     (sm): Softmax()
     (tanh): Tanh()
     (linear_cover): Linear(in_features=1, out_features=1000, bias=False)
   )
 )
 (generator): Sequential(
   (0): Linear(in_features=1000, out_features=49653, bias=True)
   (1): LogSoftmax()
 )
)
* number of parameters: 156983653
('encoder: ', 45421500)
('decoder: ', 111562153)
Making optimizer for training.

Traceback (most recent call last):
 File "train.py", line 443, in <module>
   main()
 File "train.py", line 435, in main
   train_model(model, fields, optim, data_type, model_opt)
 File "train.py", line 252, in train_model
   train_stats = trainer.train(train_iter, epoch, report_func)
 File "PyTorchWorker@b05e01674.nt12#0/onmt/Trainer.py", line 178, in train
   report_stats, normalization)
 File "PyTorchWorker@b05e01674.nt12#0/onmt/Trainer.py", line 307, in _gradient_accumulation
   self.model(src, tgt, src_lengths, dec_state)
 File "/opt/conda/envs/py2.7/lib/python2.7/site-packages/torch/nn/modules/module.py", line 357, in __call__
   result = self.forward(*input, **kwargs)
 File "/PyTorchWorker@b05e01674.nt12#0/onmt/Models.py", line 592, in forward
   memory_lengths=lengths)
 File "/opt/conda/envs/py2.7/lib/python2.7/site-packages/torch/nn/modules/module.py", line 357, in __call__
   result = self.forward(*input, **kwargs)
 File "/PyTorchWorker@b05e01674.nt12#0/onmt/Models.py", line 313, in forward
   tgt, memory_bank, state, memory_lengths=memory_lengths)
 File "/PyTorchWorker@b05e01674.nt12#0/onmt/Models.py", line 494, in _run_forward_pass
   rnn_output, hidden = self.rnn(decoder_input, hidden)
 File "/opt/conda/envs/py2.7/lib/python2.7/site-packages/torch/nn/modules/module.py", line 357, in __call__
   result = self.forward(*input, **kwargs)
 File "/PyTorchWorker@b05e01674.nt12#0/onmt/modules/StackedRNN.py", line 21, in forward
   h_0, c_0 = hidden
ValueError: need more than 1 value to unpack

(Guillaume Klein) #2

You got an answer on the issue you opened on GitHub. I’m just linking to it below for reference: