Error in Chinese Abstractive summarization preprocessing

tuzcsap · May 22, 2021, 5:58am

Hello! I was trying to use Chinese Abstractive summarization model from Report for Chinese Abstractive summarization performance · Issue #547 · OpenNMT/OpenNMT-py · GitHub on colab with the next steps:

created config.yaml
ran onmt_build_vocab on my dataset
executed onmt_translate with pretrained chinese model on my dataset
but encountered this error:

Traceback (most recent call last):
  File "/usr/local/bin/onmt_translate", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/dist-packages/onmt/bin/translate.py", line 44, in main
    translate(opt)
  File "/usr/local/lib/python3.7/dist-packages/onmt/bin/translate.py", line 15, in translate
    translator = build_translator(opt, logger=logger, report_score=True)
  File "/usr/local/lib/python3.7/dist-packages/onmt/translate/translator.py", line 32, in build_translator
    fields, model, model_opt = load_test_model(opt)
  File "/usr/local/lib/python3.7/dist-packages/onmt/model_builder.py", line 96, in load_test_model
    opt.gpu)
  File "/usr/local/lib/python3.7/dist-packages/onmt/model_builder.py", line 238, in build_base_model
    model = build_task_specific_model(model_opt, fields)
  File "/usr/local/lib/python3.7/dist-packages/onmt/model_builder.py", line 150, in build_task_specific_model
    encoder, src_emb = build_encoder_with_embeddings(model_opt, fields)
  File "/usr/local/lib/python3.7/dist-packages/onmt/model_builder.py", line 121, in build_encoder_with_embeddings
    src_emb = build_src_emb(model_opt, fields)
  File "/usr/local/lib/python3.7/dist-packages/onmt/model_builder.py", line 112, in build_src_emb
    src_field = fields["src"]
TypeError: list indices must be integers or slices, not str

Could you, please, tell me, which steps did I miss?

francoishernandez · May 25, 2021, 12:20pm

The checkpoint’s vocab is not compatible with the latest releases.
You can fix it with something like this:

github.com

OpenNMT/OpenNMT-py/blob/585499a4509b1c2dac7683b7588f3f544f89119e/onmt/inputters/inputter.py#L234-L283


def load_old_vocab(vocab, data_type="text", dynamic_dict=False):
    """Update a legacy vocab/field format.


    Args:
        vocab: a list of (field name, torchtext.vocab.Vocab) pairs. This is the
            format formerly saved in *.vocab.pt files. Or, text data
            not using a :class:`TextMultiField`.
        data_type (str): text, img, or audio
        dynamic_dict (bool): Used for copy attention.


    Returns:
        a dictionary whose keys are the field names and whose values Fields.
    """


    if _old_style_vocab(vocab):
        # List[Tuple[str, Vocab]] -> List[Tuple[str, Field]]
        # -> dict[str, Field]
        vocab = dict(vocab)
        n_src_features = sum('src_feat_' in k for k in vocab)
        n_tgt_features = sum('tgt_feat_' in k for k in vocab)

This file has been truncated. show original

tuzcsap · May 28, 2021, 10:31am

Thank you for your reply! Checking out revision 585499a450 from provided link helped with loading model’s vocab.