Automatic Backup

ymoslem · January 12, 2022, 3:11pm

Hello!

I am going to add a couple of lines to this file to upload each new model checkpoint to a Google storage bucket. However, this means the lines will be removed with each OpenNMT-py update. Any better or more elegant way to achieve this? Thanks!

github.com

OpenNMT/OpenNMT-py/blob/7c314f41dc1b017ac105144beeb53cb072960a54/onmt/models/model_saver.py#L145

    
      
              checkpoint = {
                  'model': model_state_dict,
                  'generator': generator_state_dict,
                  'vocab': vocab,
                  'opt': self.model_opt,
                  'optim': self.optim.state_dict(),
              }
          
          
    logger.info("Saving checkpoint %s_step_%d.pt" % (self.base_path, step))
              checkpoint_path = '%s_step_%d.pt' % (self.base_path, step)
              torch.save(checkpoint, checkpoint_path)
              return checkpoint, checkpoint_path
          
          
def _rm_checkpoint(self, name):
              if os.path.exists(name):
                  os.remove(name)