Upload README.md with huggingface_hub

README.md

@@ -4,14 +4,37 @@ library_name: llama.cpp
 license: apache-2.0
 language:
   - en
+  - zh
+  - multilingual
 tags:
   - gguf
   - embedding
   - multimodal
   - qwen2
   - qwen2.5-omni
+  - feature-extraction
+  - text-embedding
+  - image-embedding
+  - audio-embedding
+  - imatrix
 pipeline_tag: feature-extraction
+model_type: qwen2vl
 quantized_by: marksverdhei
+datasets:
+  - MIEB
+model_name: LCO-Embedding-Omni-7B-GGUF
+model-index:
+  - name: LCO-Embedding-Omni-7B-GGUF
+    results:
+      - task:
+          type: feature-extraction
+        metrics:
+          - name: Embedding Dimensions
+            type: embedding_dimensions
+            value: 3584
+          - name: Pooling Method
+            type: pooling
+            value: last-token
 ---
 
 # LCO-Embedding-Omni-7B-GGUF
@@ -22,10 +45,23 @@ Converted using [ht-llama.cpp](https://github.com/heiervang-technologies/ht-llam
 
 ## About the model
 
-LCO-Embedding-Omni-7B is a multimodal embedding model based on the Thinker component of [Qwen 2.5 Omni](https://huggingface.co/Qwen/Qwen2.5-Omni-7B), fine-tuned with contrastive learning to produce embeddings from text, images, audio, and video. It uses last-token pooling to extract embeddings from the final hidden state.
+[LCO-Embedding-Omni-7B](https://huggingface.co/LCO-Embedding/LCO-Embedding-Omni-7B) is a 9B-parameter multimodal embedding model based on the Thinker component of [Qwen 2.5 Omni](https://huggingface.co/Qwen/Qwen2.5-Omni-7B), fine-tuned with LoRA and contrastive learning to produce 3584-dimensional embeddings from text, images, audio, and video. It achieves **state-of-the-art on [MIEB-Lite](https://huggingface.co/spaces/MTEB/MIEB) (68.8 mean across 51 tasks)**, outperforming models trained on 21x more data. Uses last-token pooling.
+
+See [Scaling Language-Centric Omnimodal Representation Learning](https://arxiv.org/abs/2510.11693) (NeurIPS 2025) for details.
+
+### Original model benchmarks (MIEB-Lite, 51 tasks)
+
+| Model | Data | Mean |
+|-------|------|------|
+| GME (7B) | 8.0M pairs | 64.5 |
+| mmE5 (11B) | 2.1M pairs | 61.8 |
+| Voyage Multimodal 3 | -- | 58.1 |
+| **LCO-Emb-Omni (7B)** | **370k pairs** | **68.8** |
 
 ## Available files
 
+### Standard quantizations
+
 | File | Quant | Size | Description |
 |------|-------|------|-------------|
 | `LCO-Embedding-Omni-7B-BF16.gguf` | BF16 | 15 GB | Full precision, no quality loss |
@@ -33,10 +69,60 @@ LCO-Embedding-Omni-7B is a multimodal embedding model based on the Thinker compo
 | `LCO-Embedding-Omni-7B-Q4_K_M.gguf` | Q4_K_M | 4.4 GB | Good balance of quality and size |
 | `LCO-Embedding-Omni-7B-Q3_K_M.gguf` | Q3_K_M | 3.6 GB | Smaller, some quality loss |
 | `LCO-Embedding-Omni-7B-Q2_K.gguf` | Q2_K | 2.9 GB | Smallest, more quality loss |
-| `mmproj-LCO-Embedding-Omni-7b-F16.gguf` | F16 | 1.3 GB | Vision + audio projection (required for multimodal) |
+
+### Importance matrix (imatrix) quantizations
+
+Quantized with an importance matrix computed from WikiText-2 calibration data for improved quality at low bit widths.
+
+| File | Quant | Size | Description |
+|------|-------|------|-------------|
+| `LCO-Embedding-Omni-7B-IQ4_XS.gguf` | IQ4_XS | 4.0 GB | 4.25 bpw, imatrix-optimized |
+| `LCO-Embedding-Omni-7B-IQ3_M.gguf` | IQ3_M | 3.4 GB | 3.66 bpw, imatrix-optimized |
+| `LCO-Embedding-Omni-7B-IQ3_XS.gguf` | IQ3_XS | -- | 3.3 bpw, imatrix-optimized |
+| `LCO-Embedding-Omni-7B-IQ2_M.gguf` | IQ2_M | -- | 2.7 bpw, imatrix-optimized |
+| `LCO-Embedding-Omni-7B-IQ2_XS.gguf` | IQ2_XS | -- | 2.31 bpw, imatrix-optimized |
+
+### Multimodal projection
+
+| File | Quant | Size | Description |
+|------|-------|------|-------------|
+| `mmproj-LCO-Embedding-Omni-7b-F16.gguf` | F16 | 2.5 GB | Vision + audio projection (required for multimodal) |
 
 For text-only embedding, you only need one of the text model GGUFs. For multimodal (image/audio/video), you also need the `mmproj` file.
 
+## Quantization quality
+
+Measured on 8 diverse text sentences (3584-dim embeddings). BF16 is the reference.
+
+### Similarity matrix quality vs BF16
+
+| Quant | Speedup | Mean Abs Diff | Max Diff | Pearson r | Spearman rho |
+|-------|---------|---------------|----------|-----------|--------------|
+| Q8_0 | 1.7x | 0.0025 | 0.009 | 0.9997 | 0.9956 |
+| Q4_K_M | 2.4x | 0.0073 | 0.022 | 0.9974 | 0.9951 |
+| Q3_K_M | 2.8x | 0.0165 | 0.063 | 0.9839 | 0.9770 |
+| Q2_K | 3.1x | 0.0429 | 0.175 | 0.9126 | 0.8506 |
+
+### Embedding vector cosine similarity vs BF16
+
+| Quant | Mean | Min | Max |
+|-------|------|-----|-----|
+| Q8_0 | 0.9998 | 0.9997 | 0.9999 |
+| Q4_K_M | 0.9948 | 0.9908 | 0.9965 |
+| Q3_K_M | 0.9825 | 0.9646 | 0.9882 |
+| Q2_K | 0.9111 | 0.8620 | 0.9432 |
+
+### pgvector retrieval quality (query with quant, corpus in BF16)
+
+| Quant | Recall@1 | Recall@3 | Mean Drift | Max Drift |
+|-------|----------|----------|------------|-----------|
+| Q8_0 | 100% | 100% | 0.0002 | 0.0003 |
+| Q4_K_M | 100% | 100% | 0.0052 | 0.0092 |
+| Q3_K_M | 100% | 100% | 0.0175 | 0.0354 |
+| Q2_K | 100% | 100% | 0.0889 | 0.1380 |
+
+**Recommendation:** Q8_0 is essentially lossless for retrieval. Q4_K_M offers an excellent quality/size tradeoff. Q3_K_M is viable for constrained environments. Q2_K works but shows meaningful embedding drift (~9%).
+
 ## Usage
 
 ### Build llama.cpp
@@ -45,10 +131,10 @@ For text-only embedding, you only need one of the text model GGUFs. For multimod
 git clone https://github.com/heiervang-technologies/ht-llama.cpp
 cd ht-llama.cpp
 cmake -B build
-cmake --build build --target llama-embedding -j$(nproc)
+cmake --build build --target llama-embedding llama-server -j$(nproc)
 ```
 
-### Text embeddings
+### Text embeddings (CLI)
 
 ```bash
 ./build/bin/llama-embedding \
@@ -57,32 +143,76 @@ cmake --build build --target llama-embedding -j$(nproc)
   -p "Your text here"
 ```
 
-### Multimodal embeddings
+### Text embeddings (server)
 
-Multimodal embedding support (vision + audio) requires the mmproj file and the llama.cpp server:
+```bash
+./build/bin/llama-server \
+  -m LCO-Embedding-Omni-7B-Q8_0.gguf \
+  --embedding --pooling last
+
+curl -s http://localhost:8080/embeddings \
+  -d '{"content": "Your text here"}'
+```
+
+### Multimodal embeddings (vision + audio)
+
+Requires the mmproj file:
 
 ```bash
 ./build/bin/llama-server \
   -m LCO-Embedding-Omni-7B-Q8_0.gguf \
   --mmproj mmproj-LCO-Embedding-Omni-7b-F16.gguf \
-  --embedding \
-  --pooling last
+  --embedding --pooling last
 ```
 
-Then use the `/embeddings` endpoint:
-
 ```bash
-# Text embedding
+# Image embedding (base64-encoded image)
 curl -s http://localhost:8080/embeddings \
-  -d '{"content": "Your text here"}'
+  -d '{"content": [{"prompt_string": "<__media__>", "multimodal_data": ["<base64-image-data>"]}]}'
 
-# Image embedding (base64)
+# Audio embedding (base64-encoded WAV)
 curl -s http://localhost:8080/embeddings \
-  -d '{"content": [{"prompt_string": "Describe this image", "multimodal_data": [{"type": "image", "data": "<base64>"}]}]}'
+  -d '{"content": [{"prompt_string": "<__media__>", "multimodal_data": ["<base64-audio-data>"]}]}'
+```
+
+### JSON output (for programmatic use)
+
+```bash
+./build/bin/llama-embedding \
+  -m LCO-Embedding-Omni-7B-Q8_0.gguf \
+  --pooling last \
+  --embd-output-format json \
+  -p "Your text here"
 ```
 
 ## Notes
 
 - This is a quantization of [LCO-Embedding/LCO-Embedding-Omni-7B](https://huggingface.co/LCO-Embedding/LCO-Embedding-Omni-7B) -- see the original model card for benchmarks, training details, and licensing
-- Embedding inference has not been fully validated yet -- contributions and bug reports welcome at [ht-llama.cpp](https://github.com/heiervang-technologies/ht-llama.cpp/issues)
-- The `--pooling last` flag is important -- this model uses last-token pooling, not mean pooling
+- The `--pooling last` flag is required -- this model uses last-token pooling, not mean pooling
+- Embedding dimensions: 3584
+- All three modalities (text, vision, audio) have been tested and verified working
+- Contributions and bug reports welcome at [ht-llama.cpp](https://github.com/heiervang-technologies/ht-llama.cpp/issues)
+
+## Citations
+
+### LCO-Embedding
+
+```bibtex
+@article{xiao2025scaling,
+  title={Scaling Language-Centric Omnimodal Representation Learning},
+  author={Xiao, Chenghao and Chan, Hou Pong and Zhang, Hao and Xu, Weiwen and Aljunied, Mahani and Rong, Yu},
+  journal={arXiv preprint arXiv:2510.11693},
+  year={2025}
+}
+```
+
+### Qwen 2.5 Omni
+
+```bibtex
+@article{Qwen2.5-Omni,
+  title={Qwen2.5-Omni Technical Report},
+  author={Jin Xu and Zhifang Guo and Jinzheng He and Hangrui Hu and Ting He and Shuai Bai and Keqin Chen and Jialin Wang and Yang Fan and Kai Dang and Bin Zhang and Xiong Wang and Yunfei Chu and Junyang Lin},
+  journal={arXiv preprint arXiv:2503.20215},
+  year={2025}
+}
+```