|
Low bleu score with Sentencepiece comparing to othoner tokenizers
|
|
2
|
95
|
August 12, 2022
|
|
Getting no output when usig SentencePiece
|
|
1
|
146
|
April 23, 2022
|
|
Using Sentencepiece/Byte Pair Encoding on Model
|
|
42
|
7148
|
March 16, 2022
|
|
Overfitting Model
|
|
2
|
242
|
February 28, 2022
|
|
Question about English to Chinese
|
|
7
|
368
|
February 17, 2022
|
|
Translation Example in OpenNMT 2.0 Docs
|
|
12
|
750
|
October 2, 2021
|
|
Single character tokenization?
|
|
10
|
3278
|
August 26, 2021
|
|
Hard spaces lost when tokenizing
|
|
5
|
410
|
July 19, 2021
|
|
How Much Does Tokenization Affect Neural Machine Translation?
|
|
1
|
440
|
June 21, 2021
|
|
How to use pre-trained BPEmb subword embeddings with latest versions of OpenNMT and OpenNMT-py?
|
|
4
|
701
|
March 31, 2021
|
|
Different subword tokenization in same word pattern
|
|
3
|
319
|
November 20, 2020
|
|
Tokenizer v1.20.0 with SentencePiece v0.1.92 potentially problematic?
|
|
5
|
571
|
October 3, 2020
|
|
Tokenizer (sp_model, vocabulary_threshold) with unexpected results
|
|
6
|
406
|
September 29, 2020
|
|
How to define the tokenization technique in opennmt-tf
|
|
1
|
431
|
July 8, 2020
|
|
Korean - English Model
|
|
14
|
2344
|
May 25, 2020
|
|
Tensor conversion/ValueError when training with online tokenizer
|
|
7
|
532
|
May 11, 2020
|
|
Data not being Tokenized properly
|
|
4
|
431
|
May 10, 2020
|
|
Character tokenizer with TF2 version
|
|
2
|
480
|
March 17, 2020
|
|
Problems with pyonmttok
|
|
4
|
1084
|
October 2, 2019
|
|
Core dump while loading the tokenizer
|
|
3
|
753
|
September 16, 2019
|
|
Issue with special character U+FF5F
|
|
18
|
1249
|
August 14, 2019
|