Using a learned BPE Model for Transformer

Hi,

I learned BPE for my dataset using other tool. Is it correct to do this?

From this config

data:
train_features_file: ./data/filipino.data
train_labels_file: ./data/cebuano.data
eval_features_file: ./data/filipino.val
eval_labels_file: ./data/cebuano.val
source_vocabulary: ./data/fil.vocab
target_vocabulary: ./data/ceb.vocab

to this
image (5)

1 Like