OpenNMT Forum

Resource exhausted error

Hi, I wonder why I always get ResourceExhaustedError even though I had set my batch_size to 512.

So after my model running local_init_op (like the picture below), it always shows the error as the last picture. Last time, I tried with 3017 as batch_size and it works, but now I don’t know why it’s not working. Thank you

Hi,

Does the error happen during evaluation? If yes, you probably have a very long sentence in your evaluation set. You should clean it up.

Thank you for your reply,

I don’t think it happen during evaluation, because I think this picture says that the evaluation is working fine. But i think the problem is when tensorflow Done running local_init_op.

And how long is a very long sentence that you mean? Because i tried to run the same command and data on different device (but same specification) and it works. Thank you

According to the logs, it does happen during evaluation.

A shape of [32, 8, 3711, 3711] is reported which corresponds to a sequence of length 3711. It probably should not be more than 100-200 tokens.