Hello,
I am getting Missing case feature error while training at the evaluation step
This is the data.yml file, I am using.
model_dir: run/ data: train_features_file: src-train.txt train_labels_file: tgt-train.txt eval_features_file: src-val.txt eval_labels_file: tgt-val.txt source_vocabulary: src-vocab.txt target_vocabulary: tgt-vocab.txt source_tokenization: type: OpenNMTTokenizer params: mode: aggressive case_feature: true sp_model_path: src.model # sp_nbest_size: 64 # sp_alpha: 0.1 joiner_annotate: true segment_numbers: true target_tokenization: type: OpenNMTTokenizer params: mode: aggressive case_feature: true sp_model_path: tgt.model # sp_nbest_size: 64 # sp_alpha: 0.1 joiner_annotate: true segment_numbers: true train: save_checkpoints_steps: 2000 eval: external_evaluators: bleu
Error Message:
INFO:tensorflow:Step = 10000 ; steps/s = 0.41, source words/s = 9277, target words/s = 10388 ; Learning rate = 0.000884 ; Loss = 1.437881 INFO:tensorflow:Saved checkpoint run/ckpt-10000 INFO:tensorflow:Running evaluation for step 10000 Traceback (most recent call last): File "/usr/local/bin/onmt-main", line 8, in <module> sys.exit(main()) File "/usr/local/lib/python3.6/dist-packages/opennmt/bin/main.py", line 224, in main hvd=hvd) File "/usr/local/lib/python3.6/dist-packages/opennmt/runner.py", line 236, in train moving_average_decay=train_config.get("moving_average_decay")) File "/usr/local/lib/python3.6/dist-packages/opennmt/training.py", line 122, in __call__ early_stop = self._evaluate(evaluator, step, moving_average=moving_average) File "/usr/local/lib/python3.6/dist-packages/opennmt/training.py", line 149, in _evaluate evaluator(step) File "/usr/local/lib/python3.6/dist-packages/opennmt/evaluation.py", line 304, in __call__ ordered_writer.push(prediction) File "/usr/local/lib/python3.6/dist-packages/opennmt/utils/misc.py", line 382, in push return self._try_notify() File "/usr/local/lib/python3.6/dist-packages/opennmt/utils/misc.py", line 368, in _try_notify self._callback_fn(value) File "/usr/local/lib/python3.6/dist-packages/opennmt/evaluation.py", line 284, in <lambda> self._model.print_prediction(prediction, stream=output_file)) File "/usr/local/lib/python3.6/dist-packages/opennmt/models/sequence_to_sequence.py", line 358, in print_prediction sentence = self.labels_inputter.tokenizer.detokenize(tokens) File "/usr/local/lib/python3.6/dist-packages/opennmt/tokenizers/tokenizer.py", line 119, in detokenize return self._detokenize(tokens, sequence_length) File "/usr/local/lib/python3.6/dist-packages/opennmt/tokenizers/tokenizer.py", line 144, in _detokenize return self._detokenize_string(tokens) File "/usr/local/lib/python3.6/dist-packages/opennmt/tokenizers/opennmt_tokenizer.py", line 47, in _detokenize_string return self._tokenizer.detokenize(tokens) RuntimeError: Missing case feature