Embeddings training process

Support
1

Hello

I’m a little confused with the use of embeddings on the training process for the Encoder-decoder model. Can you please explain the main steps in the training and how the embeddings in this process are trained?

Thanks
Claudia

2

Hi Claudia,
there is related information in the following discussion:

3

Hi! I’m trying to feed my pre-trained embeddings (word2vec) in the training process but I get an error.

nohup: ignoring input
[2019-04-09 10:15:58,734 INFO]  * src vocab size = 9907
[2019-04-09 10:15:58,735 INFO]  * tgt vocab size = 7948
[2019-04-09 10:15:58,735 INFO] Building model...
Traceback (most recent call last):
  File "train.py", line 109, in <module>
    main(opt)
  File "train.py", line 39, in main
    single_main(opt, 0)
  File "/home/speech/cmatosve/OpenNMT-py/onmt/train_single.py", line 83$
    model = build_model(model_opt, opt, fields, checkpoint)
  File "/home/speech/cmatosve/OpenNMT-py/onmt/model_builder.py", line 2$
    model = build_base_model(model_opt, fields, use_gpu(opt), checkpoin$
  File "/home/speech/cmatosve/OpenNMT-py/onmt/model_builder.py", line 2$
    model_opt.pre_word_vecs_enc)
  File "/home/speech/cmatosve/OpenNMT-py/onmt/modules/embeddings.py", l$
    pretrained = torch.load(emb_file)
  File "/home/speech/cmatosve/OpenNMT-py/env/local/lib/python2.7/site-p$
    return _load(f, map_location, pickle_module)
  File "/home/speech/cmatosve/OpenNMT-py/env/local/lib/python2.7/site-p$
    magic_number = pickle_module.load(f)
cPickle.UnpicklingError: invalid load key, '9'.

Can you help me with that?
Regards
Claudia