my configuration file
{
"models_root": "./available_models",
"models": [
{ "id":100,
"model": "model_0.pt",
"opt": {
"gpu": 3,
"batch_size":1,
"beam_size": 1,
"n_best":1
},
"timeout": 3600,
"on_timeout": "to_cpu"
}
]
}
[2019-11-11 14:05:42,809 INFO] xxx - - [11/Nov/2019 14:05:42] “POST /translator/translate HTTP/1.1” 200 -
[2019-11-11 14:05:45,945 INFO] Running translation using 100
[2019-11-11 14:05:46,299 INFO] Using model #100 1 inputs
translation time: 0.351007
[2019-11-11 14:05:46,300 INFO] Translation Results: 1
[2019-11-11 14:05:46,300 INFO] - - [11/Nov/2019 14:05:46] “POST /translator/translate HTTP/1.1” 200 -
[2019-11-11 14:05:49,877 INFO] Running translation using 100
[2019-11-11 14:05:50,247 INFO] Using model #100 1 inputs
translation time: 0.367194
[2019-11-11 14:05:50,247 INFO] Translation Results: 1
[2019-11-11 14:05:50,248 INFO] - - [11/Nov/2019 14:05:50] “POST /translator/translate HTTP/1.1” 200 -
[2019-11-11 14:05:51,825 INFO] Running translation using 100
[2019-11-11 14:05:52,147 INFO] Using model #100 1 inputs
translation time: 0.319511
the speed is 0.3+ seconds, how to improve the speed of the prediction ?