Problem regarding serving

Support
1

I want to set up a translation server, I’m trying to make the github example work first

I download the example model and uncompress it in examples/serving folder

Then I launch the server using Docker via

nvidia-docker run -d --rm -p 9000:9000 -v $PWD:/models \
  --name tensorflow_serving --entrypoint tensorflow_model_server \
  tensorflow/serving:1.13.0-gpu \
  --enable_batching=true --batching_parameters_file=/models/batching_parameters.txt \
  --port=9000 --model_base_path=/models/averaged-ende-export500k --model_name=ende

In a second docker I then run:
pip install -r requirements.txt

and from this second docker:
python ende_client.py --port 9000 --model_name ende \
--sentencepiece_model averaged-ende-export500k/1554540232/assets.extra/wmtende.model

The client asks me for a source sentence, but when I enter it I get

Error

Traceback (most recent call last):
File “ende_client.py”, line 108, in
main()
File “ende_client.py”, line 102, in main
output = translate(stub, args.model_name, [text], tokenizer, timeout=args.timeout)
File “ende_client.py”, line 78, in translate
result = future.result()
File “/usr/local/lib/python3.5/dist-packages/grpc/_channel.py”, line 294, in result
raise self
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = “Connect Failed”
debug_error_string = “{“created”:”@1558715430.344807244",“description”:“Failed to create subchannel”,“file”:“src/core/ext/filters/client_channel/client_channel.cc”,“file_line”:2636,“referenced_errors”:[{“created”:"@1558715430.344772741",“description”:“Pick Cancelled”,“file”:“src/core/ext/filters/client_channel/lb_policy/pick_first/pick_first.cc”,“file_line”:241,“referenced_errors”:[{“created”:"@1558715430.344747178",“description”:“Connect Failed”,“file”:“src/core/ext/filters/client_channel/subchannel.cc”,“file_line”:663,“grpc_status”:14,“referenced_errors”:[{“created”:"@1558715430.344704984",“description”:“HTTP proxy returned response code 403”,“file”:“src/core/ext/filters/client_channel/http_connect_handshaker.cc”,“file_line”:211}]}]}]}"

I am a little bit lost, some help would be greatly appreciated

2

Running the client within Docker is not in the scope of this example. Can you try --net=host on the client Docker? If it does not work, you should probably search on Google on how to access a host port from a Docker container.

3

--net=host isn’t an available option in my ende_client.py

I’ve tried to launch my client docker with

docker run <image> --network="host"

but I still have the exact same error, I also tried to add --host=127.0.0.1 and --port=9000 to my client

My docker ps after lauching server looks like:

docker
docker.png1580×41 8.23 KB

4

You should search on Google or StackOverflow. This is a Docker issue.

5

I’ll come back to it later and post if I have a solution