I noticed that OpenNMT appears to have support for Crayon (as per what I read in the list of parameters for running train.lua). From what I gathered from this post:
OpenNMT forum - Support Crayon Logging
it appears that it’s supposed to be similar to TensorBoard?
I was able to successfully install the crayon luarock, but how do I use it?
Can I use it to display information about a model that I already trained? Or does it only collect data while training is running?
It’s pretty neat, it uses tensorboard directly.
Basically you set up a tb/docker server following the instructions here https://github.com/torrvision/crayon (for instance my sample server is here http://crayon.eastus.cloudapp.azure.com:8888/). And then give that path to opennmt. It will then log its data directly there so you can see graphs of progress.
It only does it while running. Not after the fact.
I will add this to the opennmt-py version as well.
Hello,
I would like to better understand how to apply the Crayon-Tensorboard function, but can’t seem to incorporate it successfully.
I have managed to install Crayon successfully following the instruction provided in their GitHub page (https://github.com/torrvision/crayon). By inputting the following command in my terminal, Tensorboard is receiving information which it can visualize from my training session:
$ th train.lua -save_config training_configs -data wmt15-all-de-en-train.t7 -report_every 50 -save_model wmt15-de-en -exp wmt15_de_en -gpuid 1 -log_file training_log
However, I can only see the learning rate, perplexity, and tokens per sec. How can see other metrics such as accuracy, blue, loss, etc? From reading the Crayon GitHub page, I get the impression that I have to create a .lua file specifying what I want to extract for visualization. If so, where should that file be saved so that when executing the training command it gets read?
As a follow up, what if, in the one hand, I want to run other experiments with the same options as a previous run and on the other hand experiments with less options? Am I then creating multiple .lua files? How would I indicate that I want configurations from experiment A vs those from B when executing train.lua?
I thank you in advance for any help to get me out of the woods. 