The progress of multi-GPU feature


(Rylanchiu) #1

I wonder what is the progress of multi-GPU, and (1) Even though the current version can not gain speed improvements, I still want to use it for enabling large batch size. Can I run the in bug-free mode. I tried once and it returned Error: AttributeError: 'DataParallel' object has no attribute 'generator'. (2) Can I train the model via multi-GPU with Lua version and load and fine-tune it with Python version? Thanks!

(Vincent Nguyen) #2

there is a pending PR

if you’re on a hurry to test, you can give a try.

(Rylanchiu) #3

Hi Vincent,

Thanks for your work. I have taken a try but seems that it has the same bug of this

(Vincent Nguyen) #4

Yes coverage_attn does not work in multi gpu mode right now.
Hope someone will look into it.

(Rylanchiu) #5

Not only coverge_attn, copy_attn also raised the same error.