Rest server Performance throughput

Hi,the REST server is quite slow for me(GTX 1080Ti) 。 What can I do to improve throughput?
See the picture:
I hope someone can help me .thanks.


You should provide more information: how you start the server, how you send requests, are you batching multiple sentences, etc.

start command:
gunicorn -w 4 -b -t 100 --worker-class=egg:meinheld#gunicorn_worker server_ar_en:app

Use http one-sentence request, no batch。But I have 100 users concurrent。
I don’t know where it is blocked.

Are you using the original What’s your baseline version?

I have not modified。OpenNMT-py:v0.8.2 PyTorch:v1.0
use flask gunicorn start server。
How much is your throughput?
my server code: