In preprocess.py
issue about "torch.save(fields, vocab_path)
Please let me know the command
python preprocess.py -train_src data/src-train.txt -train_tgt data/tgt-train.txt -valid_src data/src-val.txt -valid_tgt data/tgt-val.txt -save_data data/data -src_vocab_size 1000 -tgt_vocab_size 1000
please
pip3 install torch
pip3 install -r requirements.txt
Also check the python version
Try
python3 preprocess.py -train_src data/src-train.txt -train_tgt data/tgt-train.txt -valid_src data/src-val.txt -valid_tgt data/tgt-val.txt -save_data data/data -src_vocab_size 1000 -tgt_vocab_size 1000
my python -V is Python 3.7.0
I have installed requirements.txt and torch, but the problem does still exists.
Traceback (most recent call last):
File “preprocess.py”, line 215, in
main(opt)
File “preprocess.py”, line 196, in main
‘train’, fields, src_reader, tgt_reader, opt)
File “preprocess.py”, line 139, in build_save_dataset
torch.save(fields, vocab_path)
File “C:\Users\Administrator\Anaconda3\lib\site-packages\torch\serialization.py”, line 209, in save
return _with_file_like(f, “wb”, lambda f: _save(obj, f, pickle_module, pickle_protocol))
File “C:\Users\Administrator\Anaconda3\lib\site-packages\torch\serialization.py”, line 134, in _with_file_like
return body(f)
File “C:\Users\Administrator\Anaconda3\lib\site-packages\torch\serialization.py”, line 209, in
return _with_file_like(f, “wb”, lambda f: _save(obj, f, pickle_module, pickle_protocol))
File “C:\Users\Administrator\Anaconda3\lib\site-packages\torch\serialization.py”, line 282, in _save
pickler.dump(obj)
_pickle.PicklingError: Can’t pickle <function Field. at 0x0000013D8D5D6510>: attribute lookup Field. on torchtext.data.field failed
please command like below
python3 setup.py build
python3 setup.py install
it doesn’t work, but thank you for your help
I see you are using Anaconda on Windows. What is the Torch version installed for Anaconda?
You need to update to the latest version of PyTorch. To do so, open the PyTorch website, scroll down, and click on the relevant options: Stable, Windows, Conda, Python 3.7, and CUDA version if you are on a GPU. This will change the field “Run this Command” that you have to use.
Please note also that PyTorch doesn’t work on 32-bit system. Please use Windows and Python 64-bit version.
Kind regards,
Yasmin