Hi,
I observed that training gets paused without continuing and no error information comes.
I am using Transformer model and early stopping was in patience 7/10. I started with train_from a previous model.
Hi,
This is presumably fixed with the latest PyTorch version. Try OpenNMT-py 0.9.2 with PyTorch 1.2.
I using pytorch 1.6.0+cu101 and OpenNMT-py 1.2.0(latest version)
I found that i have problem like @merc85garcia’s issue last week
For small data size;(train.pt size is 34M), I made models successfully but for bigger train data size, i didn’t that
Would I select method that @guillaumekln recommended to merc85gracia ?
If you know my train configure and others, i’ll tell you
+) When I change configure 4gpu to only one gpu, it’s worked without pausing
probably I think that this issue is involved with gpu memory or gpu exclusive mode?