Speech to text - Memory error

Support
1

Hi, has anyone used OpenNMT-py to train a speech to text model on a relatively big data set (one that is bigger than AN4, used in the tutorial: e.g. librispeech)? I am trying to do so and I keep having memory errors.

I am using shard size 60, batch size 2, max audio length = 15s.

*** Error in `python’: double free or corruption (fasttop): 0xaf9031c0
...
...
...
(core dumped)

I am also printing nvidia-smi output each 3s and at some point the GPU memory jumped from 2G to 10G; which I found unreasonable.

Thanks.

2

Hi,

Does the stack trace contain more information? If so, please post the full output.