[2025-02-25 12:12:19,690 INFO] valid stats calculation
took: 10.005232334136963 s.
[2025-02-25 12:12:19,691 INFO] Train perplexity: 14.9597
[2025-02-25 12:12:19,692 INFO] Train accuracy: 95.2032
[2025-02-25 12:12:19,692 INFO] Sentences processed: 60283
[2025-02-25 12:12:19,692 INFO] Average bsz: 438/ 663/30
[2025-02-25 12:12:19,692 INFO] Validation perplexity: 1.18745
[2025-02-25 12:12:19,692 INFO] Validation accuracy: 95.4726
[2025-02-25 12:12:19,693 INFO] Weighted corpora loaded so far:
* corpus_1: 36382
[2025-02-25 12:12:19,694 INFO] Saving checkpoint data/run/kisii_en_model_step_1000.pt
[2025-02-25 12:12:19,710 INFO] Weighted corpora loaded so far:
* corpus_1: 36383
[2025-02-25 12:12:19,720 INFO] Weighted corpora loaded so far:
* corpus_1: 36384
[2025-02-25 12:12:19,733 INFO] Weighted corpora loaded so far:
* corpus_1: 36385
[2025-02-25 12:12:19,750 INFO] Weighted corpora loaded so far:
* corpus_1: 36386
[2025-02-25 12:12:19,760 INFO] Weighted corpora loaded so far:
* corpus_1: 36387
[2025-02-25 12:12:19,773 INFO] Weighted corpora loaded so far:
* corpus_1: 36388
[2025-02-25 12:12:19,790 INFO] Weighted corpora loaded so far:
* corpus_1: 36389
[2025-02-25 12:12:19,804 INFO] Weighted corpora loaded so far:
* corpus_1: 36390
[2025-02-25 12:12:19,815 INFO] Weighted corpora loaded so far:
* corpus_1: 36391
[2025-02-25 12:12:19,831 INFO] Weighted corpora loaded so far:
* corpus_1: 36392
[2025-02-25 12:12:19,840 INFO] Weighted corpora loaded so far:
* corpus_1: 36393
[2025-02-25 12:12:19,853 INFO] Weighted corpora loaded so far:
* corpus_1: 36394
[2025-02-25 12:12:19,869 INFO] Weighted corpora loaded so far:
* corpus_1: 36395
[2025-02-25 12:12:19,880 INFO] Weighted corpora loaded so far:
* corpus_1: 36396
[2025-02-25 12:12:19,892 INFO] Weighted corpora loaded so far:
* corpus_1: 36397
[2025-02-25 12:12:19,905 INFO] Weighted corpora loaded so far:
* corpus_1: 36398
[2025-02-25 12:12:19,915 INFO] Weighted corpora loaded so far:
* corpus_1: 36399
[2025-02-25 12:12:19,925 INFO] Weighted corpora loaded so far:
* corpus_1: 36400
[2025-02-25 12:12:19,941 INFO] Weighted corpora loaded so far:
* corpus_1: 36401
[2025-02-25 12:12:19,951 INFO] Weighted corpora loaded so far:
* corpus_1: 36402
[2025-02-25 12:12:19,963 INFO] Weighted corpora loaded so far:
* corpus_1: 36403
[2025-02-25 12:12:19,976 INFO] Weighted corpora loaded so far:
* corpus_1: 36404
[2025-02-25 12:12:19,985 INFO] Weighted corpora loaded so far:
* corpus_1: 36405
[2025-02-25 12:12:19,995 INFO] Weighted corpora loaded so far:
* corpus_1: 36406
[2025-02-25 12:12:20,007 INFO] Weighted corpora loaded so far:
* corpus_1: 36407
[2025-02-25 12:12:20,016 INFO] Weighted corpora loaded so far:
* corpus_1: 36408
[2025-02-25 12:12:20,025 INFO] Weighted corpora loaded so far:
* corpus_1: 36409
[2025-02-25 12:12:20,038 INFO] Weighted corpora loaded so far:
* corpus_1: 36410
[2025-02-25 12:12:20,048 INFO] Weighted corpora loaded so far:
* corpus_1: 36411
[2025-02-25 12:12:20,059 INFO] Weighted corpora loaded so far:
* corpus_1: 36412
[2025-02-25 12:12:20,072 INFO] Weighted corpora loaded so far:
* corpus_1: 36413
[2025-02-25 12:12:20,082 INFO] Weighted corpora loaded so far:
* corpus_1: 36414
[2025-02-25 12:12:20,091 INFO] Weighted corpora loaded so far:
* corpus_1: 36415
[2025-02-25 12:12:20,104 INFO] Weighted corpora loaded so far:
* corpus_1: 36416
[2025-02-25 12:12:20,114 INFO] Weighted corpora loaded so far:
* corpus_1: 36417
[2025-02-25 12:12:20,123 INFO] Weighted corpora loaded so far:
* corpus_1: 36418
[2025-02-25 12:12:20,134 INFO] Weighted corpora loaded so far:
* corpus_1: 36419
[2025-02-25 12:12:20,142 INFO] Weighted corpora loaded so far:
* corpus_1: 36420
[2025-02-25 12:12:20,151 INFO] Weighted corpora loaded so far:
* corpus_1: 36421
[2025-02-25 12:12:20,165 INFO] Weighted corpora loaded so far:
* corpus_1: 36422
[2025-02-25 12:12:20,179 INFO] Weighted corpora loaded so far:
* corpus_1: 36423
[2025-02-25 12:12:20,190 INFO] Weighted corpora loaded so far:
* corpus_1: 36424
[2025-02-25 12:12:20,202 INFO] Weighted corpora loaded so far:
* corpus_1: 36425
[2025-02-25 12:12:20,211 INFO] Weighted corpora loaded so far:
* corpus_1: 36426
[2025-02-25 12:12:20,220 INFO] Weighted corpora loaded so far:
* corpus_1: 36427
[2025-02-25 12:12:20,233 INFO] Weighted corpora loaded so far:
* corpus_1: 36428
[2025-02-25 12:12:20,242 INFO] Weighted corpora loaded so far:
* corpus_1: 36429
[2025-02-25 12:12:20,251 INFO] Weighted corpora loaded so far:
* corpus_1: 36430
[2025-02-25 12:12:20,265 INFO] Weighted corpora loaded so far:
* corpus_1: 36431
[2025-02-25 12:12:20,273 INFO] Weighted corpora loaded so far:
* corpus_1: 36432
[2025-02-25 12:12:20,283 INFO] Weighted corpora loaded so far:
* corpus_1: 36433
[2025-02-25 12:12:20,297 INFO] Weighted corpora loaded so far:
* corpus_1: 36434
[2025-02-25 12:12:20,306 INFO] Weighted corpora loaded so far:
* corpus_1: 36435
[2025-02-25 12:12:20,317 INFO] Weighted corpora loaded so far:
* corpus_1: 36436
[2025-02-25 12:12:20,330 INFO] Weighted corpora loaded so far:
* corpus_1: 36437
[2025-02-25 12:12:20,338 INFO] Weighted corpora loaded so far:
* corpus_1: 36438
[2025-02-25 12:12:20,347 INFO] Weighted corpora loaded so far:
* corpus_1: 36439
[2025-02-25 12:12:20,356 INFO] Weighted corpora loaded so far:
* corpus_1: 36440
[2025-02-25 12:12:20,369 INFO] Weighted corpora loaded so far:
* corpus_1: 36441
[2025-02-25 12:12:20,378 INFO] Weighted corpora loaded so far:
* corpus_1: 36442
[2025-02-25 12:12:20,390 INFO] Weighted corpora loaded so far:
* corpus_1: 36443
[2025-02-25 12:12:20,407 INFO] Weighted corpora loaded so far:
* corpus_1: 36444
[2025-02-25 12:12:20,418 INFO] Weighted corpora loaded so far:
* corpus_1: 36445
[2025-02-25 12:12:20,428 INFO] Weighted corpora loaded so far:
* corpus_1: 36446
[2025-02-25 12:12:20,443 INFO] Weighted corpora loaded so far:
* corpus_1: 36447
[2025-02-25 12:12:20,453 INFO] Weighted corpora loaded so far:
* corpus_1: 36448
[2025-02-25 12:12:20,463 INFO] Weighted corpora loaded so far:
* corpus_1: 36449
[2025-02-25 12:12:20,478 INFO] Weighted corpora loaded so far:
* corpus_1: 36450
[2025-02-25 12:12:20,487 INFO] Weighted corpora loaded so far:
* corpus_1: 36451
[2025-02-25 12:12:20,496 INFO] Weighted corpora loaded so far:
* corpus_1: 36452
[2025-02-25 12:12:20,508 INFO] Weighted corpora loaded so far:
* corpus_1: 36453
[2025-02-25 12:12:20,517 INFO] Weighted corpora loaded so far:
* corpus_1: 36454
[2025-02-25 12:12:20,526 INFO] Weighted corpora loaded so far:
* corpus_1: 36455
[2025-02-25 12:12:20,538 INFO] Weighted corpora loaded so far:
* corpus_1: 36456
[2025-02-25 12:12:20,548 INFO] Weighted corpora loaded so far:
* corpus_1: 36457
[2025-02-25 12:12:20,560 INFO] Weighted corpora loaded so far:
* corpus_1: 36458
[2025-02-25 12:12:20,572 INFO] Weighted corpora loaded so far:
* corpus_1: 36459
[2025-02-25 12:12:20,581 INFO] Weighted corpora loaded so far:
* corpus_1: 36460
[2025-02-25 12:12:20,590 INFO] Weighted corpora loaded so far:
* corpus_1: 36461
[2025-02-25 12:12:20,602 INFO] Weighted corpora loaded so far:
* corpus_1: 36462
[2025-02-25 12:12:20,612 INFO] Weighted corpora loaded so far:
* corpus_1: 36463
[2025-02-25 12:12:20,620 INFO] Weighted corpora loaded so far:
* corpus_1: 36464
[2025-02-25 12:12:20,635 INFO] Weighted corpora loaded so far:
* corpus_1: 36465
[2025-02-25 12:12:20,644 INFO] Weighted corpora loaded so far:
* corpus_1: 36466
[2025-02-25 12:12:20,653 INFO] Weighted corpora loaded so far:
* corpus_1: 36467
[2025-02-25 12:12:20,666 INFO] Weighted corpora loaded so far:
* corpus_1: 36468
[2025-02-25 12:12:20,675 INFO] Weighted corpora loaded so far:
* corpus_1: 36469
[2025-02-25 12:12:20,686 INFO] Weighted corpora loaded so far:
* corpus_1: 36470
[2025-02-25 12:12:20,701 INFO] Weighted corpora loaded so far:
* corpus_1: 36471
[2025-02-25 12:12:20,709 INFO] Weighted corpora loaded so far:
* corpus_1: 36472
[2025-02-25 12:12:20,719 INFO] Weighted corpora loaded so far:
* corpus_1: 36473
[2025-02-25 12:12:20,734 INFO] Weighted corpora loaded so far:
* corpus_1: 36474
[2025-02-25 12:12:20,743 INFO] Weighted corpora loaded so far:
* corpus_1: 36475
[2025-02-25 12:12:20,753 INFO] Weighted corpora loaded so far:
* corpus_1: 36476
[2025-02-25 12:12:20,768 INFO] Weighted corpora loaded so far:
* corpus_1: 36477
[2025-02-25 12:12:20,777 INFO] Weighted corpora loaded so far:
* corpus_1: 36478
[2025-02-25 12:12:20,787 INFO] Weighted corpora loaded so far:
* corpus_1: 36479
[2025-02-25 12:12:20,801 INFO] Weighted corpora loaded so far:
* corpus_1: 36480
[2025-02-25 12:12:20,813 INFO] Weighted corpora loaded so far:
* corpus_1: 36481
[2025-02-25 12:12:20,822 INFO] Weighted corpora loaded so far:
* corpus_1: 36482
[2025-02-25 12:12:20,835 INFO] Weighted corpora loaded so far:
* corpus_1: 36483
[2025-02-25 12:12:20,845 INFO] Weighted corpora loaded so far:
* corpus_1: 36484
[2025-02-25 12:12:20,855 INFO] Weighted corpora loaded so far:
* corpus_1: 36485
[2025-02-25 12:12:20,869 INFO] Weighted corpora loaded so far:
* corpus_1: 36486
[2025-02-25 12:12:20,879 INFO] Weighted corpora loaded so far:
* corpus_1: 36487
[2025-02-25 12:12:20,888 INFO] Weighted corpora loaded so far:
* corpus_1: 36488
[2025-02-25 12:12:20,902 INFO] Weighted corpora loaded so far:
* corpus_1: 36489
[2025-02-25 12:12:20,911 INFO] Weighted corpora loaded so far:
* corpus_1: 36490
[2025-02-25 12:12:20,921 INFO] Weighted corpora loaded so far:
* corpus_1: 36491
[2025-02-25 12:12:20,937 INFO] Weighted corpora loaded so far:
* corpus_1: 36492
[2025-02-25 12:12:20,947 INFO] Weighted corpora loaded so far:
* corpus_1: 36493
[2025-02-25 12:12:20,956 INFO] Weighted corpora loaded so far:
* corpus_1: 36494
[2025-02-25 12:12:20,970 INFO] Weighted corpora loaded so far:
* corpus_1: 36495
[2025-02-25 12:12:20,979 INFO] Weighted corpora loaded so far:
* corpus_1: 36496
[2025-02-25 12:12:20,988 INFO] Weighted corpora loaded so far:
* corpus_1: 36497
[2025-02-25 12:12:21,001 INFO] Weighted corpora loaded so far:
* corpus_1: 36498
[2025-02-25 12:12:21,009 INFO] Weighted corpora loaded so far:
* corpus_1: 36499
[2025-02-25 12:12:21,019 INFO] Weighted corpora loaded so far:
* corpus_1: 36500
[2025-02-25 12:12:21,028 INFO] Weighted corpora loaded so far:
* corpus_1: 36501
[2025-02-25 12:12:21,040 INFO] Weighted corpora loaded so far:
* corpus_1: 36502
[2025-02-25 12:12:21,057 INFO] Weighted corpora loaded so far:
* corpus_1: 36503
[2025-02-25 12:12:21,072 INFO] Weighted corpora loaded so far:
* corpus_1: 36504
[2025-02-25 12:12:21,092 INFO] Weighted corpora loaded so far:
* corpus_1: 36505
[2025-02-25 12:12:21,109 INFO] Weighted corpora loaded so far:
* corpus_1: 36506
[2025-02-25 12:12:21,119 INFO] Weighted corpora loaded so far:
* corpus_1: 36507
[2025-02-25 12:12:21,134 INFO] Weighted corpora loaded so far:
* corpus_1: 36508
[2025-02-25 12:12:21,147 INFO] Weighted corpora loaded so far:
* corpus_1: 36509
[2025-02-25 12:12:21,159 INFO] Weighted corpora loaded so far:
* corpus_1: 36510
[2025-02-25 12:12:21,182 INFO] Weighted corpora loaded so far:
* corpus_1: 36511
[2025-02-25 12:12:21,202 INFO] Weighted corpora loaded so far:
* corpus_1: 36512
[2025-02-25 12:12:21,230 INFO] Weighted corpora loaded so far:
* corpus_1: 36513
[2025-02-25 12:12:21,258 INFO] Weighted corpora loaded so far:
* corpus_1: 36514
[2025-02-25 12:12:21,271 INFO] Weighted corpora loaded so far:
* corpus_1: 36515
[2025-02-25 12:12:21,281 INFO] Weighted corpora loaded so far:
* corpus_1: 36516
[2025-02-25 12:12:21,298 INFO] Weighted corpora loaded so far:
* corpus_1: 36517
[2025-02-25 12:12:21,308 INFO] Weighted corpora loaded so far:
* corpus_1: 36518
[2025-02-25 12:12:21,318 INFO] Weighted corpora loaded so far:
* corpus_1: 36519
[2025-02-25 12:12:21,334 INFO] Weighted corpora loaded so far:
* corpus_1: 36520
[2025-02-25 12:12:21,344 INFO] Weighted corpora loaded so far:
* corpus_1: 36521
[2025-02-25 12:12:21,355 INFO] Weighted corpora loaded so far:
* corpus_1: 36522
[2025-02-25 12:12:21,372 INFO] Weighted corpora loaded so far:
* corpus_1: 36523
[2025-02-25 12:12:21,387 INFO] Weighted corpora loaded so far:
* corpus_1: 36524
[2025-02-25 12:12:21,400 INFO] Weighted corpora loaded so far:
* corpus_1: 36525
[2025-02-25 12:12:21,423 INFO] Weighted corpora loaded so far:
* corpus_1: 36526
[2025-02-25 12:12:21,438 INFO] Weighted corpora loaded so far:
* corpus_1: 36527
[2025-02-25 12:12:21,450 INFO] Weighted corpora loaded so far:
* corpus_1: 36528
[2025-02-25 12:12:21,468 INFO] Weighted corpora loaded so far:
* corpus_1: 36529
[2025-02-25 12:12:21,482 INFO] Weighted corpora loaded so far:
* corpus_1: 36530
[2025-02-25 12:12:21,491 INFO] Weighted corpora loaded so far:
* corpus_1: 36531
[2025-02-25 12:12:21,506 INFO] Weighted corpora loaded so far:
* corpus_1: 36532
[2025-02-25 12:12:21,516 INFO] Weighted corpora loaded so far:
* corpus_1: 36533
[2025-02-25 12:12:21,526 INFO] Weighted corpora loaded so far:
* corpus_1: 36534
[2025-02-25 12:12:21,539 INFO] Weighted corpora loaded so far:
* corpus_1: 36535
[2025-02-25 12:12:21,548 INFO] Weighted corpora loaded so far:
* corpus_1: 36536
[2025-02-25 12:12:21,559 INFO] Weighted corpora loaded so far:
* corpus_1: 36537
[2025-02-25 12:12:21,572 INFO] Weighted corpora loaded so far:
* corpus_1: 36538
[2025-02-25 12:12:21,582 INFO] Weighted corpora loaded so far:
* corpus_1: 36539
[2025-02-25 12:12:21,594 INFO] Weighted corpora loaded so far:
* corpus_1: 36540
[2025-02-25 12:12:21,609 INFO] Weighted corpora loaded so far:
* corpus_1: 36541
[2025-02-25 12:12:21,621 INFO] Weighted corpora loaded so far:
* corpus_1: 36542
[2025-02-25 12:12:21,634 INFO] Weighted corpora loaded so far:
* corpus_1: 36543
[2025-02-25 12:12:21,650 INFO] Weighted corpora loaded so far:
* corpus_1: 36544
[2025-02-25 12:12:21,663 INFO] Weighted corpora loaded so far:
* corpus_1: 36545
[2025-02-25 12:12:21,674 INFO] Weighted corpora loaded so far:
* corpus_1: 36546
[2025-02-25 12:12:21,689 INFO] Weighted corpora loaded so far:
* corpus_1: 36547
[2025-02-25 12:12:21,702 INFO] Weighted corpora loaded so far:
* corpus_1: 36548
[2025-02-25 12:12:21,712 INFO] Weighted corpora loaded so far:
* corpus_1: 36549
[2025-02-25 12:12:21,727 INFO] Weighted corpora loaded so far:
* corpus_1: 36550
[2025-02-25 12:12:21,741 INFO] Weighted corpora loaded so far:
* corpus_1: 36551
[2025-02-25 12:12:21,755 INFO] Weighted corpora loaded so far:
* corpus_1: 36552
I managed to do training but my Train Perplexity: 14.96: Measures prediction quality on training data (919 verses). Higher than validation (1.19), suggesting overfitting, but reasonable for small data.
If it’s overfitting is it ok?