---
language:
- en
license: apache-2.0
tags:
- sentence-transformers
- sparse-encoder
- sparse
- splade
- generated_from_trainer
- dataset_size:390000
- loss:SpladeLoss
- loss:SparseMultipleNegativesRankingLoss
- loss:FlopsLoss
base_model: drexalt/NeoBERT-RetroMAE-pretrain
widget:
- text: placeholder
datasets:
- lightonai/ms-marco-en-bge-gemma
pipeline_tag: feature-extraction
library_name: sentence-transformers
metrics:
- dot_accuracy@1
- dot_accuracy@3
- dot_accuracy@5
- dot_accuracy@10
- dot_precision@1
- dot_precision@3
- dot_precision@5
- dot_precision@10
- dot_recall@1
- dot_recall@3
- dot_recall@5
- dot_recall@10
- dot_ndcg@10
- dot_mrr@10
- dot_map@100
- query_active_dims
- query_sparsity_ratio
- corpus_active_dims
- corpus_sparsity_ratio
model-index:
- name: splade-NeoBERT-RetroMAE-pretrain trained on LightOn MS MARCO (triplets)
results:
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoSciFact
type: NanoSciFact
metrics:
- type: dot_accuracy@1
value: 0.54
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.66
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.74
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.8
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.54
name: Dot Precision@1
- type: dot_precision@3
value: 0.24
name: Dot Precision@3
- type: dot_precision@5
value: 0.168
name: Dot Precision@5
- type: dot_precision@10
value: 0.09
name: Dot Precision@10
- type: dot_recall@1
value: 0.515
name: Dot Recall@1
- type: dot_recall@3
value: 0.635
name: Dot Recall@3
- type: dot_recall@5
value: 0.73
name: Dot Recall@5
- type: dot_recall@10
value: 0.79
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.655473775938405
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.6167142857142857
name: Dot Mrr@10
- type: dot_map@100
value: 0.6155237632737632
name: Dot Map@100
- type: query_active_dims
value: 214.10000610351562
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9929853873893089
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 527.3963623046875
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9827207796899061
name: Corpus Sparsity Ratio
- type: dot_accuracy@1
value: 0.54
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.68
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.74
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.8
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.54
name: Dot Precision@1
- type: dot_precision@3
value: 0.24666666666666665
name: Dot Precision@3
- type: dot_precision@5
value: 0.168
name: Dot Precision@5
- type: dot_precision@10
value: 0.09
name: Dot Precision@10
- type: dot_recall@1
value: 0.515
name: Dot Recall@1
- type: dot_recall@3
value: 0.655
name: Dot Recall@3
- type: dot_recall@5
value: 0.73
name: Dot Recall@5
- type: dot_recall@10
value: 0.79
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.65598376976016
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.6173809523809524
name: Dot Mrr@10
- type: dot_map@100
value: 0.6161873435206768
name: Dot Map@100
- type: query_active_dims
value: 215.0
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9929559006618177
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 529.1915283203125
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9826619642120336
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoMSMARCO
type: NanoMSMARCO
metrics:
- type: dot_accuracy@1
value: 0.46
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.64
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.7
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.76
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.46
name: Dot Precision@1
- type: dot_precision@3
value: 0.21333333333333332
name: Dot Precision@3
- type: dot_precision@5
value: 0.14
name: Dot Precision@5
- type: dot_precision@10
value: 0.07600000000000001
name: Dot Precision@10
- type: dot_recall@1
value: 0.46
name: Dot Recall@1
- type: dot_recall@3
value: 0.64
name: Dot Recall@3
- type: dot_recall@5
value: 0.7
name: Dot Recall@5
- type: dot_recall@10
value: 0.76
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.6033192844653051
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.5536904761904761
name: Dot Mrr@10
- type: dot_map@100
value: 0.5669218226959646
name: Dot Map@100
- type: query_active_dims
value: 99.13999938964844
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9967518511437766
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 295.5086364746094
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.990318175857591
name: Corpus Sparsity Ratio
- type: dot_accuracy@1
value: 0.46
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.64
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.7
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.76
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.46
name: Dot Precision@1
- type: dot_precision@3
value: 0.21333333333333332
name: Dot Precision@3
- type: dot_precision@5
value: 0.14
name: Dot Precision@5
- type: dot_precision@10
value: 0.07600000000000001
name: Dot Precision@10
- type: dot_recall@1
value: 0.46
name: Dot Recall@1
- type: dot_recall@3
value: 0.64
name: Dot Recall@3
- type: dot_recall@5
value: 0.7
name: Dot Recall@5
- type: dot_recall@10
value: 0.76
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.6030305868428701
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.5534126984126982
name: Dot Mrr@10
- type: dot_map@100
value: 0.5665743193221454
name: Dot Map@100
- type: query_active_dims
value: 99.80000305175781
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9967302272769885
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 296.68511962890625
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9902796304426675
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoTouche2020
type: NanoTouche2020
metrics:
- type: dot_accuracy@1
value: 0.7142857142857143
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.9183673469387755
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.9591836734693877
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.9591836734693877
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.7142857142857143
name: Dot Precision@1
- type: dot_precision@3
value: 0.6190476190476191
name: Dot Precision@3
- type: dot_precision@5
value: 0.6163265306122448
name: Dot Precision@5
- type: dot_precision@10
value: 0.4857142857142858
name: Dot Precision@10
- type: dot_recall@1
value: 0.04901801483377477
name: Dot Recall@1
- type: dot_recall@3
value: 0.12839324801527874
name: Dot Recall@3
- type: dot_recall@5
value: 0.20197488305019035
name: Dot Recall@5
- type: dot_recall@10
value: 0.31116392892910627
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.5505974886287978
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.8153061224489796
name: Dot Mrr@10
- type: dot_map@100
value: 0.40077665094673615
name: Dot Map@100
- type: query_active_dims
value: 93.40816497802734
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.996939644683244
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 369.0992126464844
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9879071092115036
name: Corpus Sparsity Ratio
- type: dot_accuracy@1
value: 0.7142857142857143
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.9183673469387755
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.9591836734693877
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.9795918367346939
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.7142857142857143
name: Dot Precision@1
- type: dot_precision@3
value: 0.6190476190476191
name: Dot Precision@3
- type: dot_precision@5
value: 0.6163265306122448
name: Dot Precision@5
- type: dot_precision@10
value: 0.4897959183673469
name: Dot Precision@10
- type: dot_recall@1
value: 0.04901801483377477
name: Dot Recall@1
- type: dot_recall@3
value: 0.12839324801527874
name: Dot Recall@3
- type: dot_recall@5
value: 0.20197488305019035
name: Dot Recall@5
- type: dot_recall@10
value: 0.3156954697119897
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.5537250207448905
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.816326530612245
name: Dot Mrr@10
- type: dot_map@100
value: 0.4010315264845556
name: Dot Map@100
- type: query_active_dims
value: 93.69387817382812
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9969302837896
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 370.4649353027344
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9878623636949501
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoSCIDOCS
type: NanoSCIDOCS
metrics:
- type: dot_accuracy@1
value: 0.36
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.56
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.66
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.8
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.36
name: Dot Precision@1
- type: dot_precision@3
value: 0.26
name: Dot Precision@3
- type: dot_precision@5
value: 0.22
name: Dot Precision@5
- type: dot_precision@10
value: 0.146
name: Dot Precision@10
- type: dot_recall@1
value: 0.07566666666666667
name: Dot Recall@1
- type: dot_recall@3
value: 0.16166666666666668
name: Dot Recall@3
- type: dot_recall@5
value: 0.22566666666666663
name: Dot Recall@5
- type: dot_recall@10
value: 0.29966666666666664
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.29573664808275163
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.49310317460317465
name: Dot Mrr@10
- type: dot_map@100
value: 0.2176904693649033
name: Dot Map@100
- type: query_active_dims
value: 151.5399932861328
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9950350569003954
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 496.980712890625
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9837172952987803
name: Corpus Sparsity Ratio
- type: dot_accuracy@1
value: 0.36
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.56
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.66
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.8
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.36
name: Dot Precision@1
- type: dot_precision@3
value: 0.26
name: Dot Precision@3
- type: dot_precision@5
value: 0.21599999999999997
name: Dot Precision@5
- type: dot_precision@10
value: 0.146
name: Dot Precision@10
- type: dot_recall@1
value: 0.07566666666666667
name: Dot Recall@1
- type: dot_recall@3
value: 0.16166666666666668
name: Dot Recall@3
- type: dot_recall@5
value: 0.22166666666666665
name: Dot Recall@5
- type: dot_recall@10
value: 0.29966666666666664
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.29553880630562923
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.49405555555555564
name: Dot Mrr@10
- type: dot_map@100
value: 0.2171795470333841
name: Dot Map@100
- type: query_active_dims
value: 152.10000610351562
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9950167090589241
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 498.65167236328125
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9836625492312667
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoNFCorpus
type: NanoNFCorpus
metrics:
- type: dot_accuracy@1
value: 0.44
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.6
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.64
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.64
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.44
name: Dot Precision@1
- type: dot_precision@3
value: 0.4
name: Dot Precision@3
- type: dot_precision@5
value: 0.344
name: Dot Precision@5
- type: dot_precision@10
value: 0.27399999999999997
name: Dot Precision@10
- type: dot_recall@1
value: 0.062138556107558876
name: Dot Recall@1
- type: dot_recall@3
value: 0.09968051520546126
name: Dot Recall@3
- type: dot_recall@5
value: 0.11598389889948522
name: Dot Recall@5
- type: dot_recall@10
value: 0.14031421345268424
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.3556239717444623
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.52
name: Dot Mrr@10
- type: dot_map@100
value: 0.1637676115687091
name: Dot Map@100
- type: query_active_dims
value: 90.4800033569336
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9970355807824869
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 496.4185485839844
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9837357136300379
name: Corpus Sparsity Ratio
- type: dot_accuracy@1
value: 0.44
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.6
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.64
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.64
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.44
name: Dot Precision@1
- type: dot_precision@3
value: 0.4
name: Dot Precision@3
- type: dot_precision@5
value: 0.34
name: Dot Precision@5
- type: dot_precision@10
value: 0.272
name: Dot Precision@10
- type: dot_recall@1
value: 0.062138556107558876
name: Dot Recall@1
- type: dot_recall@3
value: 0.09968051520546126
name: Dot Recall@3
- type: dot_recall@5
value: 0.11594070235520877
name: Dot Recall@5
- type: dot_recall@10
value: 0.13985966799813876
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.35414887626851177
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.52
name: Dot Mrr@10
- type: dot_map@100
value: 0.16371114767459527
name: Dot Map@100
- type: query_active_dims
value: 91.0
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9970185440010484
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 497.9546203613281
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9836853869221766
name: Corpus Sparsity Ratio
- task:
type: sparse-nano-beir
name: Sparse Nano BEIR
dataset:
name: NanoBEIR mean
type: NanoBEIR_mean
metrics:
- type: dot_accuracy@1
value: 0.5028571428571429
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.6756734693877552
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.7398367346938776
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.7918367346938775
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.5028571428571429
name: Dot Precision@1
- type: dot_precision@3
value: 0.3464761904761905
name: Dot Precision@3
- type: dot_precision@5
value: 0.297665306122449
name: Dot Precision@5
- type: dot_precision@10
value: 0.21434285714285717
name: Dot Precision@10
- type: dot_recall@1
value: 0.23236464752160013
name: Dot Recall@1
- type: dot_recall@3
value: 0.33294808597748127
name: Dot Recall@3
- type: dot_recall@5
value: 0.3947250897232685
name: Dot Recall@5
- type: dot_recall@10
value: 0.4602289618096915
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.4921502337719444
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.5997628117913832
name: Dot Mrr@10
- type: dot_map@100
value: 0.39293606357001526
name: Dot Map@100
- type: query_active_dims
value: 129.87951883829263
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9957447244991059
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 407.1836875842223
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.986659337933811
name: Corpus Sparsity Ratio
- type: dot_accuracy@1
value: 0.5457142857142858
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.7106436420722138
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.7599372056514914
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.8153532182103612
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.5457142857142858
name: Dot Precision@1
- type: dot_precision@3
value: 0.32967032967032966
name: Dot Precision@3
- type: dot_precision@5
value: 0.2587943485086342
name: Dot Precision@5
- type: dot_precision@10
value: 0.17690737833594974
name: Dot Precision@10
- type: dot_recall@1
value: 0.3189603120082316
name: Dot Recall@1
- type: dot_recall@3
value: 0.45789247549185824
name: Dot Recall@3
- type: dot_recall@5
value: 0.5139907551178431
name: Dot Recall@5
- type: dot_recall@10
value: 0.5825516902460118
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.5623474779935465
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.6345031397174254
name: Dot Mrr@10
- type: dot_map@100
value: 0.4832343408137437
name: Dot Map@100
- type: query_active_dims
value: 156.4915262466219
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9948728285745816
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 359.4252604414698
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.98822405935255
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoClimateFEVER
type: NanoClimateFEVER
metrics:
- type: dot_accuracy@1
value: 0.26
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.46
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.52
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.62
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.26
name: Dot Precision@1
- type: dot_precision@3
value: 0.15999999999999998
name: Dot Precision@3
- type: dot_precision@5
value: 0.11599999999999999
name: Dot Precision@5
- type: dot_precision@10
value: 0.07400000000000001
name: Dot Precision@10
- type: dot_recall@1
value: 0.1433333333333333
name: Dot Recall@1
- type: dot_recall@3
value: 0.22833333333333333
name: Dot Recall@3
- type: dot_recall@5
value: 0.26
name: Dot Recall@5
- type: dot_recall@10
value: 0.30066666666666664
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.2726746068133177
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.3772460317460318
name: Dot Mrr@10
- type: dot_map@100
value: 0.22213421141987527
name: Dot Map@100
- type: query_active_dims
value: 233.4199981689453
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9923524016064168
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 401.8573913574219
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.986833844723235
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoDBPedia
type: NanoDBPedia
metrics:
- type: dot_accuracy@1
value: 0.78
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.92
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.94
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.94
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.78
name: Dot Precision@1
- type: dot_precision@3
value: 0.6133333333333333
name: Dot Precision@3
- type: dot_precision@5
value: 0.588
name: Dot Precision@5
- type: dot_precision@10
value: 0.502
name: Dot Precision@10
- type: dot_recall@1
value: 0.10391478675297891
name: Dot Recall@1
- type: dot_recall@3
value: 0.1658458784908766
name: Dot Recall@3
- type: dot_recall@5
value: 0.2364483581106884
name: Dot Recall@5
- type: dot_recall@10
value: 0.33804540691659585
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.6253305386409256
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.8406666666666668
name: Dot Mrr@10
- type: dot_map@100
value: 0.46218362306786676
name: Dot Map@100
- type: query_active_dims
value: 101.41999816894531
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9966771509675334
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 330.4876708984375
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9891721489123113
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoFEVER
type: NanoFEVER
metrics:
- type: dot_accuracy@1
value: 0.78
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.9
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.94
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 1.0
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.78
name: Dot Precision@1
- type: dot_precision@3
value: 0.32
name: Dot Precision@3
- type: dot_precision@5
value: 0.19999999999999996
name: Dot Precision@5
- type: dot_precision@10
value: 0.10599999999999998
name: Dot Precision@10
- type: dot_recall@1
value: 0.7166666666666667
name: Dot Recall@1
- type: dot_recall@3
value: 0.8633333333333333
name: Dot Recall@3
- type: dot_recall@5
value: 0.9033333333333333
name: Dot Recall@5
- type: dot_recall@10
value: 0.9633333333333333
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.8531330669775704
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.8477777777777777
name: Dot Mrr@10
- type: dot_map@100
value: 0.8092375464632647
name: Dot Map@100
- type: query_active_dims
value: 147.24000549316406
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.995175938487217
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 403.611083984375
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9867763880484772
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoFiQA2018
type: NanoFiQA2018
metrics:
- type: dot_accuracy@1
value: 0.42
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.54
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.58
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.68
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.42
name: Dot Precision@1
- type: dot_precision@3
value: 0.26
name: Dot Precision@3
- type: dot_precision@5
value: 0.17600000000000002
name: Dot Precision@5
- type: dot_precision@10
value: 0.10800000000000001
name: Dot Precision@10
- type: dot_recall@1
value: 0.22341269841269842
name: Dot Recall@1
- type: dot_recall@3
value: 0.36168253968253966
name: Dot Recall@3
- type: dot_recall@5
value: 0.3871825396825397
name: Dot Recall@5
- type: dot_recall@10
value: 0.49990476190476196
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.4203007987534202
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.49455555555555547
name: Dot Mrr@10
- type: dot_map@100
value: 0.3593139757287158
name: Dot Map@100
- type: query_active_dims
value: 104.18000030517578
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.996586724319993
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 358.292724609375
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9882611649102493
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoHotpotQA
type: NanoHotpotQA
metrics:
- type: dot_accuracy@1
value: 0.86
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.92
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.94
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.94
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.86
name: Dot Precision@1
- type: dot_precision@3
value: 0.43999999999999995
name: Dot Precision@3
- type: dot_precision@5
value: 0.29599999999999993
name: Dot Precision@5
- type: dot_precision@10
value: 0.15599999999999997
name: Dot Precision@10
- type: dot_recall@1
value: 0.43
name: Dot Recall@1
- type: dot_recall@3
value: 0.66
name: Dot Recall@3
- type: dot_recall@5
value: 0.74
name: Dot Recall@5
- type: dot_recall@10
value: 0.78
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.758598735743758
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.8906666666666667
name: Dot Mrr@10
- type: dot_map@100
value: 0.6979999899439231
name: Dot Map@100
- type: query_active_dims
value: 138.0
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.995478671122469
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 333.8575744628906
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.989061739910134
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoNQ
type: NanoNQ
metrics:
- type: dot_accuracy@1
value: 0.48
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.68
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.74
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.82
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.48
name: Dot Precision@1
- type: dot_precision@3
value: 0.22666666666666666
name: Dot Precision@3
- type: dot_precision@5
value: 0.156
name: Dot Precision@5
- type: dot_precision@10
value: 0.086
name: Dot Precision@10
- type: dot_recall@1
value: 0.46
name: Dot Recall@1
- type: dot_recall@3
value: 0.63
name: Dot Recall@3
- type: dot_recall@5
value: 0.7
name: Dot Recall@5
- type: dot_recall@10
value: 0.78
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.6249114830342452
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.5917380952380952
name: Dot Mrr@10
- type: dot_map@100
value: 0.571780340853074
name: Dot Map@100
- type: query_active_dims
value: 97.18000030517578
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.996816067089143
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 359.4037780761719
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9882247631847136
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoQuoraRetrieval
type: NanoQuoraRetrieval
metrics:
- type: dot_accuracy@1
value: 0.88
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.98
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.98
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.98
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.88
name: Dot Precision@1
- type: dot_precision@3
value: 0.37999999999999995
name: Dot Precision@3
- type: dot_precision@5
value: 0.24399999999999994
name: Dot Precision@5
- type: dot_precision@10
value: 0.12999999999999998
name: Dot Precision@10
- type: dot_recall@1
value: 0.7873333333333332
name: Dot Recall@1
- type: dot_recall@3
value: 0.9186666666666667
name: Dot Recall@3
- type: dot_recall@5
value: 0.9453333333333334
name: Dot Recall@5
- type: dot_recall@10
value: 0.966
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.9174021126998152
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.9133333333333336
name: Dot Mrr@10
- type: dot_map@100
value: 0.8929260461760461
name: Dot Map@100
- type: query_active_dims
value: 98.04000091552734
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9967878906717932
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 103.37158203125
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.9966132107322179
name: Corpus Sparsity Ratio
- task:
type: sparse-information-retrieval
name: Sparse Information Retrieval
dataset:
name: NanoArguAna
type: NanoArguAna
metrics:
- type: dot_accuracy@1
value: 0.12
name: Dot Accuracy@1
- type: dot_accuracy@3
value: 0.44
name: Dot Accuracy@3
- type: dot_accuracy@5
value: 0.54
name: Dot Accuracy@5
- type: dot_accuracy@10
value: 0.64
name: Dot Accuracy@10
- type: dot_precision@1
value: 0.12
name: Dot Precision@1
- type: dot_precision@3
value: 0.14666666666666667
name: Dot Precision@3
- type: dot_precision@5
value: 0.10800000000000003
name: Dot Precision@5
- type: dot_precision@10
value: 0.064
name: Dot Precision@10
- type: dot_recall@1
value: 0.12
name: Dot Recall@1
- type: dot_recall@3
value: 0.44
name: Dot Recall@3
- type: dot_recall@5
value: 0.54
name: Dot Recall@5
- type: dot_recall@10
value: 0.64
name: Dot Recall@10
- type: dot_ndcg@10
value: 0.3757388113309908
name: Dot Ndcg@10
- type: dot_mrr@10
value: 0.29138095238095235
name: Dot Mrr@10
- type: dot_map@100
value: 0.3017868128905456
name: Dot Map@100
- type: query_active_dims
value: 462.05999755859375
name: Query Active Dims
- type: query_sparsity_ratio
value: 0.9848614115209162
name: Query Sparsity Ratio
- type: corpus_active_dims
value: 448.9967041015625
name: Corpus Active Dims
- type: corpus_sparsity_ratio
value: 0.98528940750601
name: Corpus Sparsity Ratio
---
# splade-NeoBERT-RetroMAE-pretrain trained on LightOn MS MARCO (triplets)
This is a [SPLADE Sparse Encoder](https://www.sbert.net/docs/sparse_encoder/usage/usage.html) model finetuned from [drexalt/NeoBERT-RetroMAE-pretrain](https://huggingface.co/drexalt/NeoBERT-RetroMAE-pretrain) on the [ms-marco-en-bge-gemma](https://huggingface.co/datasets/lightonai/ms-marco-en-bge-gemma) dataset using the [sentence-transformers](https://www.SBERT.net) library. It maps sentences & paragraphs to a 30522-dimensional sparse vector space and can be used for semantic search and sparse retrieval.
## Model Details
### Model Description
- **Model Type:** SPLADE Sparse Encoder
- **Base model:** [drexalt/NeoBERT-RetroMAE-pretrain](https://huggingface.co/drexalt/NeoBERT-RetroMAE-pretrain)
- **Maximum Sequence Length:** 256 tokens
- **Output Dimensionality:** 30522 dimensions
- **Similarity Function:** Dot Product
- **Training Dataset:**
- [ms-marco-en-bge-gemma](https://huggingface.co/datasets/lightonai/ms-marco-en-bge-gemma)
- **Language:** en
- **License:** apache-2.0
### Model Sources
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Documentation:** [Sparse Encoder Documentation](https://www.sbert.net/docs/sparse_encoder/usage/usage.html)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sparse Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=sparse-encoder)
### Full Model Architecture
```
SparseEncoder(
(0): MLMTransformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'NeoBERTLMHead'})
(1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': 30522})
)
```
## Usage
### Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
```bash
pip install -U sentence-transformers
```
Then you can load this model and run inference.
```python
from sentence_transformers import SparseEncoder
# Download from the 🤗 Hub
model = SparseEncoder("drexalt/splade-NeoBERT-RetroMAE-pretrain-lighton-msmarco-triplets-20250919-064009")
# Run inference
sentences = [
'The weather is lovely today.',
"It's so sunny outside!",
'He drove to the stadium.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 30522]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```
## Evaluation
### Metrics
#### Sparse Information Retrieval
* Datasets: `NanoSciFact`, `NanoMSMARCO`, `NanoTouche2020`, `NanoSCIDOCS`, `NanoNFCorpus`, `NanoClimateFEVER`, `NanoDBPedia`, `NanoFEVER`, `NanoFiQA2018`, `NanoHotpotQA`, `NanoMSMARCO`, `NanoNFCorpus`, `NanoNQ`, `NanoQuoraRetrieval`, `NanoSCIDOCS`, `NanoArguAna`, `NanoSciFact` and `NanoTouche2020`
* Evaluated with [SparseInformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseInformationRetrievalEvaluator)
| Metric | NanoSciFact | NanoMSMARCO | NanoTouche2020 | NanoSCIDOCS | NanoNFCorpus | NanoClimateFEVER | NanoDBPedia | NanoFEVER | NanoFiQA2018 | NanoHotpotQA | NanoNQ | NanoQuoraRetrieval | NanoArguAna |
|:----------------------|:------------|:------------|:---------------|:------------|:-------------|:-----------------|:------------|:-----------|:-------------|:-------------|:-----------|:-------------------|:------------|
| dot_accuracy@1 | 0.54 | 0.46 | 0.7143 | 0.36 | 0.44 | 0.26 | 0.78 | 0.78 | 0.42 | 0.86 | 0.48 | 0.88 | 0.12 |
| dot_accuracy@3 | 0.68 | 0.64 | 0.9184 | 0.56 | 0.6 | 0.46 | 0.92 | 0.9 | 0.54 | 0.92 | 0.68 | 0.98 | 0.44 |
| dot_accuracy@5 | 0.74 | 0.7 | 0.9592 | 0.66 | 0.64 | 0.52 | 0.94 | 0.94 | 0.58 | 0.94 | 0.74 | 0.98 | 0.54 |
| dot_accuracy@10 | 0.8 | 0.76 | 0.9796 | 0.8 | 0.64 | 0.62 | 0.94 | 1.0 | 0.68 | 0.94 | 0.82 | 0.98 | 0.64 |
| dot_precision@1 | 0.54 | 0.46 | 0.7143 | 0.36 | 0.44 | 0.26 | 0.78 | 0.78 | 0.42 | 0.86 | 0.48 | 0.88 | 0.12 |
| dot_precision@3 | 0.2467 | 0.2133 | 0.619 | 0.26 | 0.4 | 0.16 | 0.6133 | 0.32 | 0.26 | 0.44 | 0.2267 | 0.38 | 0.1467 |
| dot_precision@5 | 0.168 | 0.14 | 0.6163 | 0.216 | 0.34 | 0.116 | 0.588 | 0.2 | 0.176 | 0.296 | 0.156 | 0.244 | 0.108 |
| dot_precision@10 | 0.09 | 0.076 | 0.4898 | 0.146 | 0.272 | 0.074 | 0.502 | 0.106 | 0.108 | 0.156 | 0.086 | 0.13 | 0.064 |
| dot_recall@1 | 0.515 | 0.46 | 0.049 | 0.0757 | 0.0621 | 0.1433 | 0.1039 | 0.7167 | 0.2234 | 0.43 | 0.46 | 0.7873 | 0.12 |
| dot_recall@3 | 0.655 | 0.64 | 0.1284 | 0.1617 | 0.0997 | 0.2283 | 0.1658 | 0.8633 | 0.3617 | 0.66 | 0.63 | 0.9187 | 0.44 |
| dot_recall@5 | 0.73 | 0.7 | 0.202 | 0.2217 | 0.1159 | 0.26 | 0.2364 | 0.9033 | 0.3872 | 0.74 | 0.7 | 0.9453 | 0.54 |
| dot_recall@10 | 0.79 | 0.76 | 0.3157 | 0.2997 | 0.1399 | 0.3007 | 0.338 | 0.9633 | 0.4999 | 0.78 | 0.78 | 0.966 | 0.64 |
| **dot_ndcg@10** | **0.656** | **0.603** | **0.5537** | **0.2955** | **0.3541** | **0.2727** | **0.6253** | **0.8531** | **0.4203** | **0.7586** | **0.6249** | **0.9174** | **0.3757** |
| dot_mrr@10 | 0.6174 | 0.5534 | 0.8163 | 0.4941 | 0.52 | 0.3772 | 0.8407 | 0.8478 | 0.4946 | 0.8907 | 0.5917 | 0.9133 | 0.2914 |
| dot_map@100 | 0.6162 | 0.5666 | 0.401 | 0.2172 | 0.1637 | 0.2221 | 0.4622 | 0.8092 | 0.3593 | 0.698 | 0.5718 | 0.8929 | 0.3018 |
| query_active_dims | 215.0 | 99.8 | 93.6939 | 152.1 | 91.0 | 233.42 | 101.42 | 147.24 | 104.18 | 138.0 | 97.18 | 98.04 | 462.06 |
| query_sparsity_ratio | 0.993 | 0.9967 | 0.9969 | 0.995 | 0.997 | 0.9924 | 0.9967 | 0.9952 | 0.9966 | 0.9955 | 0.9968 | 0.9968 | 0.9849 |
| corpus_active_dims | 529.1915 | 296.6851 | 370.4649 | 498.6517 | 497.9546 | 401.8574 | 330.4877 | 403.6111 | 358.2927 | 333.8576 | 359.4038 | 103.3716 | 448.9967 |
| corpus_sparsity_ratio | 0.9827 | 0.9903 | 0.9879 | 0.9837 | 0.9837 | 0.9868 | 0.9892 | 0.9868 | 0.9883 | 0.9891 | 0.9882 | 0.9966 | 0.9853 |
#### Sparse Nano BEIR
* Dataset: `NanoBEIR_mean`
* Evaluated with [SparseNanoBEIREvaluator](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseNanoBEIREvaluator) with these parameters:
```json
{
"dataset_names": [
"scifact",
"msmarco",
"touche2020",
"scidocs",
"nfcorpus"
]
}
```
| Metric | Value |
|:----------------------|:-----------|
| dot_accuracy@1 | 0.5029 |
| dot_accuracy@3 | 0.6757 |
| dot_accuracy@5 | 0.7398 |
| dot_accuracy@10 | 0.7918 |
| dot_precision@1 | 0.5029 |
| dot_precision@3 | 0.3465 |
| dot_precision@5 | 0.2977 |
| dot_precision@10 | 0.2143 |
| dot_recall@1 | 0.2324 |
| dot_recall@3 | 0.3329 |
| dot_recall@5 | 0.3947 |
| dot_recall@10 | 0.4602 |
| **dot_ndcg@10** | **0.4922** |
| dot_mrr@10 | 0.5998 |
| dot_map@100 | 0.3929 |
| query_active_dims | 129.8795 |
| query_sparsity_ratio | 0.9957 |
| corpus_active_dims | 407.1837 |
| corpus_sparsity_ratio | 0.9867 |
#### Sparse Nano BEIR
* Dataset: `NanoBEIR_mean`
* Evaluated with [SparseNanoBEIREvaluator](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseNanoBEIREvaluator) with these parameters:
```json
{
"dataset_names": [
"climatefever",
"dbpedia",
"fever",
"fiqa2018",
"hotpotqa",
"msmarco",
"nfcorpus",
"nq",
"quoraretrieval",
"scidocs",
"arguana",
"scifact",
"touche2020"
]
}
```
| Metric | Value |
|:----------------------|:-----------|
| dot_accuracy@1 | 0.5457 |
| dot_accuracy@3 | 0.7106 |
| dot_accuracy@5 | 0.7599 |
| dot_accuracy@10 | 0.8154 |
| dot_precision@1 | 0.5457 |
| dot_precision@3 | 0.3297 |
| dot_precision@5 | 0.2588 |
| dot_precision@10 | 0.1769 |
| dot_recall@1 | 0.319 |
| dot_recall@3 | 0.4579 |
| dot_recall@5 | 0.514 |
| dot_recall@10 | 0.5826 |
| **dot_ndcg@10** | **0.5623** |
| dot_mrr@10 | 0.6345 |
| dot_map@100 | 0.4832 |
| query_active_dims | 156.4915 |
| query_sparsity_ratio | 0.9949 |
| corpus_active_dims | 359.4253 |
| corpus_sparsity_ratio | 0.9882 |
## Training Details
### Training Dataset
#### ms-marco-en-bge-gemma
* Dataset: [ms-marco-en-bge-gemma](https://huggingface.co/datasets/lightonai/ms-marco-en-bge-gemma) at [1a1ffe7](https://huggingface.co/datasets/lightonai/ms-marco-en-bge-gemma/tree/1a1ffe7cde403016be12ae532b249965b2293114)
* Size: 390,000 training samples
* Columns: query_id, document_ids, and scores
* Approximate statistics based on the first 1000 samples:
| | query_id | document_ids | scores |
|:--------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------|:------------------------------------|
| type | int | list | list |
| details |
401967 | [6705462, 6337046, 7109100, 3494181, 7044691, ...] | [27.25, 17.28125, 15.8359375, 16.046875, 18.03125, ...] |
| 720668 | [6444886, 5294662, 1672294, 7735669, 1950104, ...] | [28.9375, 14.84375, 13.3671875, 14.890625, 22.828125, ...] |
| 636966 | [2250068, 3211646, 4360683, 4515171, 2960232, ...] | [27.0625, 11.0234375, 11.4453125, 15.640625, 10.8359375, ...] |
* Loss: [SpladeLoss](https://sbert.net/docs/package_reference/sparse_encoder/losses.html#spladeloss) with these parameters:
```json
{
"loss": "SparseMultipleNegativesRankingLoss(scale=1.0, similarity_fct='dot_score', gather_across_devices=False)",
"document_regularizer_weight": 0.005,
"query_regularizer_weight": 0.001
}
```
### Evaluation Dataset
#### ms-marco-en-bge-gemma
* Dataset: [ms-marco-en-bge-gemma](https://huggingface.co/datasets/lightonai/ms-marco-en-bge-gemma) at [1a1ffe7](https://huggingface.co/datasets/lightonai/ms-marco-en-bge-gemma/tree/1a1ffe7cde403016be12ae532b249965b2293114)
* Size: 10,000 evaluation samples
* Columns: query_id, document_ids, and scores
* Approximate statistics based on the first 1000 samples:
| | query_id | document_ids | scores |
|:--------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------|:------------------------------------|
| type | int | list | list |
| details | 1031437 | [4421662, 6691457, 763928, 2409406, 3950402, ...] | [21.078125, 14.6328125, 15.1640625, 12.84375, 12.1328125, ...] |
| 578179 | [5025500, 3408725, 456079, 7559462, 290820, ...] | [23.09375, 20.328125, 22.71875, 22.296875, 17.921875, ...] |
| 307799 | [3998942, 3614389, 1644576, 6415293, 6655775, ...] | [24.171875, 13.8984375, 18.609375, 18.1875, 17.96875, ...] |
* Loss: [SpladeLoss](https://sbert.net/docs/package_reference/sparse_encoder/losses.html#spladeloss) with these parameters:
```json
{
"loss": "SparseMultipleNegativesRankingLoss(scale=1.0, similarity_fct='dot_score', gather_across_devices=False)",
"document_regularizer_weight": 0.005,
"query_regularizer_weight": 0.001
}
```
### Training Hyperparameters
#### Non-Default Hyperparameters
- `eval_strategy`: steps
- `per_device_train_batch_size`: 4
- `per_device_eval_batch_size`: 4
- `gradient_accumulation_steps`: 16
- `num_train_epochs`: 2
- `bf16`: True
- `dataloader_drop_last`: True
- `dataloader_num_workers`: 4
- `dataloader_prefetch_factor`: 2
- `load_best_model_at_end`: True
- `batch_sampler`: no_duplicates
#### All Hyperparameters