Nmt-wizard-docker for OpenNMT-tf V2

Support
1

Hi, Are the developers able to give an indication when the nmt-wizard-docker will be released for OpenNMT-tf V2? Although I have started building models with V2, for production I am serving V1 models in a virtual environment with the wizard which is a very valuable tool for me.

2

It will hopefully happen next month.

Note that because TensorFlow removed tf.contrib in 2.0, V1 SavedModel will not be serveable with this new image. Here are some possible options:

  • Export a V1 checkpoint with OpenNMT-tf 2.x
  • Use a previous version of the image for V1 models
3

Thanks for that info @guillaumekln

4

For reference, the changes are on master:

However, we won’t build new images until next year. You can try it out by building the image manually:

docker build -t nmtwizard/opennmt-tf -f frameworks/opennmt_tf/Dockerfile .
5

Thank you @guillaumekln. Will come in handy in the New Year. I’m currently working with TensorFlow 2.0. Happy Christmas :slight_smile:

6

The new image is up:

$ docker run -it --rm --entrypoint onmt-main nmtwizard/opennmt-tf:latest -v
OpenNMT-tf 2.5.1
7

Great, thanks.

8

I have two questions, Guillaume. Are all the Cuda 10.1 libraries and settings contained within this docker image. or is there still reliance on some host settings?
Is the config.json format now mandatory, or can the *.yml config format also be parsed?
Thanks,
Terence

9

All CUDA libraries are in the Docker image. Only the GPU driver is loaded from the host system.

The nmtwizard images only work with the config.json format.

10

Maybe this is not clear enough.

The JSON format is required to use the framework wrapper that is implemented in https://github.com/OpenNMT/nmt-wizard-docker/. But you can also choose to ignore this wrapper and only use this image for OpenNMT-tf, in which case the YML config is the required format.

11

I apologize for troubling you again. You write “only use this image for OpenNMT-tf,”. Which image do you mean? There doesn’t appear to be a docker image to pull for OpenNMT-tf.

12

I meant the image mentioned in this thread: nmtwizard/opennmt-tf. It contains a complete installation of OpenNMT-tf and some additional scripts from https://github.com/OpenNMT/nmt-wizard-docker/. You can use this image if you are looking for a Docker image including OpenNMT-tf.

13

Hi @guillaumekln,

Are the steps here still valid for serving a model trained with vanilla OpenNMT-tf? (https://github.com/OpenNMT/nmt-wizard-docker/issues/46)

I can’t get it to work. Here are the steps:
I export an averaged model like this:

CUDA_VISIBLE_DEVICES=0 onmt-main --config $WORK_DIR/config/data.yml --auto_config export --export_dir $WORK_DIR/model/export/170000/1

Then I run this command:

nvidia-docker run -e CUDA_VISIBLE_DEVICES=0 -p 7784:7784 -i --rm -v $WORK_DIR/model/export:/root/models nmtwizard/opennmt-tf --model 170000 --model_storage /root/models --gpuid 1 serve --host 0.0.0.0 --port 7784

And I get this error:

ValueError: SavedModel exported with OpenNMT-tf 1.x are no longer supported. They include ops from tf.contrib which is not included in TensorFlow 2.x binaries. To upgrade automatically, you can release or serve from a OpenNMT-tf 1.x training checkpoint.

which confuses me as the model is trained with OpenNMT-tf v2.7 and I use the latest nmtwizard/opennmt-tf image.

14

Hi,

We also changed the directory name of the released model directory. You should replace “1” by “saved_model” in the export path (you can mv it to the new name).

I will update the linked issue to use a V2 model.

15

Thank you, that let me move further, but now I get this error:

KeyError: 'Addons>GatherTree'
16

Mmh, right. This used to work.

This issue is that the library defining the operation “Addons>GatherTree” is now using a lazy mechanism to register it to the TensorFlow runtime:

Can you check if the model can be loaded and executed with the image nmtwizard/opennmt-tf:2.0.0?

17

It is great to see this image up. In the meantime I had been working on tensorflow model serving without the wizard and got things functioning nicely for individual models via an adapted grpc client. I am about to start work on using tensorflow model server to serve multiple models via ModelServer Configuration. Is this facility available within the nmtwizard?

18

Yes, with this image I can serve the model just fine, thanks!

19

No, it is not. It currently requires to start one instance per model.

20

Just to let other people know, the serving of multiple TensorFlow models works fine with your docker image “opennmt/tensorflow-serving:2.0.0-gpu” using the option “–model_config_file=/models/models.config”. I’ve only tried with a couple of models until now but I guess the only limit is that imposed by the hardware. In case anyone’s interested the model_config_file looks like this:
model_config_list {
config {
name: ‘sp2en_model’
base_path: ‘/models/sp2en_model’
model_platform:“tensorflow”
}
config {
name: ‘eng2turk_model’
base_path: ‘/models/eng2turk_model’
model_platform:“tensorflow”
}
}