What would be the best recommendation for GPUs?
RTX 3080, V100 … etc
Taking into consideration cost and performance.
Something with as much VRAM as possible for your budget. 8GB is just nice to train a base-transformer NMT model. Big-transformer will need about 16GB.
1 Like
That is the minimal.
What if I want to increase performance x10s, is that possible, and how? What GPUs should I use to achieve that with the least cost?
Any advice is welcomed.
1 Like
Right now the GPU market is a scam and prices are outrageous, so there’s no good price/performance choice for any middle to high end GPU. Once the market stabilizes again, I would suggest the RTX 3090.
2 Likes
Without a budget, it can go as high as six digits in USD (DGX A100 for example).
Would recommend 3090s or RTX A6000s.
2 Likes
It also depends on how many cards you would like to use. Performance gains are more significant when parallelizing on several GPUs than when increasing the vRAM. In my opinion, having two more modest cards may be more convenient than having one with a lot of vRAM. RTX 3090 is probably the best in terms of cost/performance, but these cards are also more expensive and difficult to mount on parallel due to their power and space requirements. I haven’t tried the RTX 3080, but 2/4 of those in parallel could be a good tradeoff in terms of cost/performance. Also keep an eye on Nvidia’s site, as it is rumoured that in May a new 3080 Ti with 12GB (not confirmed) may be presented. But as @panosk said, it is very complicated to get any card now… (I think it was Nvidia who said that this shortage would last until 2023…).
2 Likes
Guys, can somebody answer me, what GPU is the best in 2023 by cost/perfomance for training model and maybe somebody made some test for new nv GPU (4070/ti, 4080 etc with compare to 30 series)?