Hi ,
I have a machine on which I am training the model. The machine has GPU on it. But while I use the following command :
(OpenNMT) admin123@admin123-MS-7A15:~/OpenNMT-Trans/OpenNMT-py$ python train.py -data data/demo -save_model demo-model
[2019-08-21 13:33:20,148 INFO] * src vocab size = 50002
[2019-08-21 13:33:20,148 INFO] * tgt vocab size = 50004
[2019-08-21 13:33:20,148 INFO] Building model…
[2019-08-21 13:33:21,079 INFO] NMTModel(
(encoder): RNNEncoder(
(embeddings): Embeddings(
(make_embedding): Sequential(
(emb_luts): Elementwise(
(0): Embedding(50002, 500, padding_idx=1)
)
)
)
(rnn): LSTM(500, 500, num_layers=2, dropout=0.3)
)
(decoder): InputFeedRNNDecoder(
(embeddings): Embeddings(
(make_embedding): Sequential(
(emb_luts): Elementwise(
(0): Embedding(50004, 500, padding_idx=1)
)
)
)
(dropout): Dropout(p=0.3)
(rnn): StackedLSTM(
(dropout): Dropout(p=0.3)
(layers): ModuleList(
(0): LSTMCell(1000, 500)
(1): LSTMCell(500, 500)
)
)
(attn): GlobalAttention(
(linear_in): Linear(in_features=500, out_features=500, bias=False)
(linear_out): Linear(in_features=1000, out_features=500, bias=False)
)
)
(generator): Sequential(
(0): Linear(in_features=500, out_features=50004, bias=True)
(1): Cast()
(2): LogSoftmax()
)
)
[2019-08-21 13:33:21,079 INFO] encoder: 29009000
[2019-08-21 13:33:21,079 INFO] decoder: 55812004
[2019-08-21 13:33:21,079 INFO] * number of parameters: 84821004
[2019-08-21 13:33:21,080 INFO] Starting training on CPU, could be very slow
[2019-08-21 13:33:21,080 INFO] Start training loop and validate every 10000 steps…
[2019-08-21 13:33:21,080 INFO] Loading dataset from data/demo.train.0.pt
[2019-08-21 13:33:32,000 INFO] number of examples: 924406
[2019-08-21 13:35:27,137 INFO] Step 50/100000; acc: 4.05; ppl: 399127.76; xent: 12.90; lr: 1.00000; 510/568 tok/s; 126 sec
[2019-08-21 13:37:23,537 INFO] Step 100/100000; acc: 4.04; ppl: 14495.96; xent: 9.58; lr: 1.00000; 569/656 tok/s; 242 sec
[2019-08-21 13:39:23,609 INFO] Step 150/100000; acc: 4.57; ppl: 4633.62; xent: 8.44; lr: 1.00000; 572/649 tok/s; 363 sec
[2019-08-21 13:41:09,781 INFO] Step 200/100000; acc: 5.63; ppl: 3155.02; xent: 8.06; lr: 1.00000; 564/643 tok/s; 469 sec
But I have GPU on my machine. Please find the below info for that :
(base) admin123@admin123-MS-7A15:~$ nvidia-smi
Wed Aug 21 13:43:52 2019
±----------------------------------------------------------------------------+
| NVIDIA-SMI 430.40 Driver Version: 430.40 CUDA Version: 10.1 |
|-------------------------------±---------------------±---------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce GTX 106… Off | 00000000:01:00.0 On | N/A |
| 0% 45C P8 12W / 120W | 334MiB / 6075MiB | 0% Default |
±------------------------------±---------------------±---------------------+
±----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 1210 G /usr/lib/xorg/Xorg 18MiB |
| 0 1256 G /usr/bin/gnome-shell 48MiB |
| 0 1585 G /usr/lib/xorg/Xorg 135MiB |
| 0 1737 G /usr/bin/gnome-shell 119MiB |
±----------------------------------------------------------------------------+
(base) admin123@admin123-MS-7A15:~$
It shows that it is using CPU for training, How do i enable GPU to be used for training.
Please assist me in resolving this issue as early as possible ?
Thank You,
Kishor.