OpenNMT Forum

Running TensorBoard

Tutorials
,
#1

TensorBoard is a tool that provides useful visualization of how the training is going on. Here is how to activate it with OpenNMT.

1- Activating TensorBoard

  • For OpenNMT-tf, TensorBoard is enabled by default. For OpenNMT-py, you need to enable TensorBoard, and optionally customize the log directory. Add these lines to the training configuration YAML file.
tensorboard: true
tensorboard_log_dir: logs

  • Start your OpenNMT training as usual.

  • Create a screen for TensorBoard: screen -S tensorboard

  • Open the directory of the log files. In OpenNMT-tf, by default the log files are in the same folder as the model. In OpenNMT-py, the logs are in a directory with today’s date inside “runs/onmt” or the path you specified for tensorboard_log_dir

  • Start TensorBoard and specify the log directory: tensorboard --logdir="."

  • At this point, you should see a message that TensorBoard is running on localhost http://localhost:6006/ and that’s how to access it from a local browser if you are working on the same machine.

  • Get out of this screen by pressing: Ctrl+A+D.

2- Accessing TensorBoard from the Internet

Disclaimer: This method should be only used for research or demonstration purposes. For corporate and security-sensitive purposes, consult with your team first. Depending on the infrastructure you are using, there might be better methods.

  • Sign up to ngrok and download the suitable version; for example the one for Linux.

  • Unzip the downloaded ngrok archive.

  • Find your authentication key here and run the command: ./ngrok authtoken <your_authentication_key>

  • Start a new screen: screen -S ngrok

  • Start ngrok on TensorBoard’s default port 6006: ./ngrok https 6006

  • If everything works well, you should see a black screen with “Session Status Online” and other details, including “Forwarding”.

  • Copy the “Forwarding” HTTP or HTTPs and run it in your browser. You should be able to see something like this:

tensorboard
tensorboard3104Ă—1876 442 KB

Kind regards,
Yasmin

2 Likes
#2

Great tutorial, Yasmin. I have tensorboard working in my colab environment which uses OpenNMT-py. I can get access to the accuracy, loss and learning rate metrics. However, it would be great to graph the models similar to what can be seen in the following link: Examining the TensorFlow Graph  |  TensorBoard

Any idea if this can be done with OpenNMT-py ?

I see that @guillaumekln has some work done on this but I’m not sure if it’s implemented. If it is, I don’t how to replicate it. Thanks.

SĂ©amus.

#3

Dear SĂ©amus,

If you mean something out-of-the-box, my understanding is that these are the supported statistics. @guillaumekln or @francoishernandez can confirm otherwise, if needed.

Kind regards,
Yasmin

#4

Hi Yasmin, That’s correct. I’ve seen the statistics.py and these are the readily available stats that are being logged. Ideally, it would be great to also see a graph of the model. Once the model has been built and saved, it might be easier to simply load the model and display a graph of its structure. Regards, Séamus.

#5

I’m a little confused on how this is done with OpenNMT-py still. How do we set up the “screen” command? I’m running my code in Google Colab. We start the training and then start the screen command, which then allows us to connect to tensorboard? Or how else does that work? Thanks

#6

Dear Jose,

The approach above is for a regular server machine, not Google Colab. You do not need screen or even ngrok to use TensorBorad in Google Clab.

  • Check this notebook for an example using %tensorboard magic.
  • Check also answers here to see other ways.

Basically, currently using %tensorboard magic is the standard way if you are working in Google Clab.

Kind regards,
Yasmin

1 Like