Carmela
November 26, 2022, 7:40pm
1
Hi,
line 406, in multiprocessing_context
Can someone help me?
vince62s
November 28, 2022, 3:59pm
2
this is because you are on Windows or MacOS.
opened 08:37PM - 23 Nov 22 UTC
type:docs
type:enhancement
Attempting to begin training Transformer Encoder-Decoder NMT model, but issue at… startup when running `onmt_train -config config.yml`
Source and target use same tokenizer, and `onmt_build_vocab` was ran with -share_vocab flag, hence only `source.vocab`
config.yml
<img width="198" alt="image" src="https://user-images.githubusercontent.com/52015929/203641054-15032568-5011-4607-9d71-f996f456d90e.png">
```python
[2022-11-23 15:27:37,761 INFO] encoder: 54743040
[2022-11-23 15:27:37,761 INFO] decoder: 96950640
[2022-11-23 15:27:37,761 INFO] * number of parameters: 151693680
[2022-11-23 15:27:37,761 INFO] * src vocab size = 70000
[2022-11-23 15:27:37,761 INFO] * tgt vocab size = 70000
Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\ProgramData\Anaconda3\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\ProgramData\Anaconda3\Scripts\onmt_train.exe\__main__.py", line 7, in <module>
File "C:\ProgramData\Anaconda3\lib\site-packages\onmt\bin\train.py", line 65, in main
train(opt)
File "C:\ProgramData\Anaconda3\lib\site-packages\onmt\bin\train.py", line 50, in train
train_process(opt, device_id=0)
File "C:\ProgramData\Anaconda3\lib\site-packages\onmt\train_single.py", line 149, in main
_train_iter = _build_train_iter(opt, transforms_cls, vocabs,
File "C:\ProgramData\Anaconda3\lib\site-packages\onmt\train_single.py", line 117, in _build_train_iter
train_iter = build_dynamic_dataset_iter(
File "C:\ProgramData\Anaconda3\lib\site-packages\onmt\inputters\dynamic_iterator.py", line 338, in build_dynamic_dataset_iter
data_loader = DataLoader(data_iter, batch_size=None,
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 255, in __init__
self.multiprocessing_context = multiprocessing_context
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 418, in __setattr__
super(DataLoader, self).__setattr__(attr, val)
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 394, in multiprocessing_context
raise ValueError(
ValueError: multiprocessing_context option should specify a valid start method in ['spawn'], but got multiprocessing_context='fork'
```
I will attempt downgrading OpenNMT-py version to v2.3.0 until I can find a fix
OpenNMT-py v3.0.1
Pytorch v1.13.0 w/CUDA
CUDA v11.7
Python v3.9.13
OS Windows 11 x64
use num_workers=0 in your config to avoid multi_processing.
1 Like