Graph compilation time

tensorflow

#1

Hi, could you tell me how long does it take to create computation graph in Tensorflow for a model from nmt_small.py and standard configuration? It’s been almost 2 hours and the training haven’t started yet. Not sure if I need to wait or if something is wrong.
My training data is quite small, only 200k sentences. Thank you for your help!


#2

It turns out the problem was with --num-gpus? Because I used 4 instead of 1 and it got stuck.


(Guillaume Klein) #3

Hello,

Could you describe your hardware?

I did not record a specific duration but it should not take more than 1 or 2 minutes for big graphs.


#4

Yeah, I noticed that it’s almost instant, but I got confused when it got stuck. Tensorflow supports multiGPU thankfully by default.
I’m using 4 GeForce GTX 1080 Titans if that may help.