OpenNMT Forum

Rest server Performance throughput

Hi,the REST server is quite slow for me(GTX 1080Ti) 。 What can I do to improve throughput?
See the picture:
QQ%E6%88%AA%E5%9B%BE20190611102210
I hope someone can help me .thanks.

Hi,

You should provide more information: how you start the server, how you send requests, are you batching multiple sentences, etc.

@guillaumekln
start command:
gunicorn -w 4 -b 0.0.0.0:8081 -t 100 --worker-class=egg:meinheld#gunicorn_worker server_ar_en:app

Use http one-sentence request, no batch。But I have 100 users concurrent。
I don’t know where it is blocked.
Thanks。

Are you using the original translation_server.py? What’s your baseline version?

@eduamf
I have not modified。OpenNMT-py:v0.8.2 PyTorch:v1.0
use flask gunicorn start server。
How much is your throughput?
my server code:


config: