Need help with proper format of REST request to TF serving

Support
1

Hello!

I created a Turkish-English translation model with onmt train_and_eval and then strarted Tensorflow Serving with tensorflow_model_server and set the rest_api_port argument. So far so good, its running.
Then I would like to call a request:

curl -v -H "Content-Type: application/json" -X POST -d '{"signature_name": "serving_default", "instances": [{ "tokens" : [["bir", "zaman", "misin"]] , "length" : [3] }]}' http://localhost:8501/v1/models/saved_model:predict

I get the following output:

*   Trying 192.168.0.134...
* Connected to 192.168.0.134 (192.168.0.134) port 8501 (#0)
> POST /v1/models/saved_model:predict HTTP/1.1
> Host: 192.168.0.134:8501
> User-Agent: curl/7.47.0
> Accept: */*
> Content-Type: application/json
> Content-Length: 113
>
* upload completely sent off: 113 out of 113 bytes
< HTTP/1.1 400 Bad Request
< Content-Type: application/json
< Date: Tue, 30 Oct 2018 15:30:16 GMT
< Content-Length: 332
<
* Connection #0 to host 192.168.0.134 left intact
{ "error": "Expected multiples argument to be a vector of length 3 but got length 2\n\t [[{{node seq2seq/decoder/tile_batch_2/Tile}} = Tile[T=DT_INT32, Tmultiples=DT_INT32, _output_shapes=[[?,5]], _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"](seq2seq/decoder/tile_batch_2/ExpandDims, seq2seq/decoder/Tile/multiples)]]" }

On server side the following message:

2018-10-30 16:30:16.255672: W external/org_tensorflow/tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at transpose_op.cc:157 : Invalid argument: transpose expects a vector of size 4. But input(1) is a vector of size 3

Please help with the format of the request.
Thanks in advance!

2

Hi,

Try the column format instead (see the TensorFlow Serving documentation):

{
  "inputs": {
    "tokens" : [["Hello", "world", "!", ""], ["How", "are", "you", "?"]],
    "length": [3, 4]
  }
}

This should work. Note that elements in the tokens array should be padded to the same size (here 4).

3

Wow
Thanks for the quick response, it Works!
Thank you very much