Tutorial for OpenNMT-py using Colab

Tutorials
23

!sed "s/@@ //g" works fine, so no fish hook brackets like < >, but an exclamation mark ! to run a linux command

1 Like
24

Hi
I’ve trained and hosted my model but it’s produces very poor predictions and no sentence translates right, I used the default tokenizer that is learn bpe and apply bpe on the sentences but in the configuration of the json file, I used a sentencepiece pretrained model downloaded. I used sentencepiece because I’m using windows and pyonmttok does not work on windows.

25

Traceback (most recent call last):
File “/usr/lib/python3.6/multiprocessing/pool.py”, line 119, in worker
result = (True, func(*args, **kwds))
File “/content/OpenNMT-py/onmt/bin/preprocess.py”, line 54, in process_one_shard
assert len(src_shard) == len(tgt_shard)
AssertionError

…
I am getting the above error while running preprocess.py on my dataset.
I am applying bpe before preprocessing.
please, help me to solve the problem.

26

Good day! Do we have an updated github repo for this tutorial? One where the previously mentioned errors no longer exist (preprocess.py not found, functions not found, etc.)

27

I also found out that the preprocess.py file is actually empty; there is no code inside.

28

Here is a new google colab notebook for OpenNMT.py 2.x

1 Like
29

This notebook helps with Cuda 10.1 incompatibility on colab. Thanks!

1 Like