Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,8 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+res/bf16_vs_fp8.png filter=lfs diff=lfs merge=lfs -text
+res/eu_cofunding.png filter=lfs diff=lfs merge=lfs -text
+res/overall_scores_by_model.png filter=lfs diff=lfs merge=lfs -text
+res/per_property_scores_by_model.png filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,410 @@
+---
+license: apache-2.0
+language:
+- eng
+- spa
+- ita
+- fra
+- deu
+- pol
+- ukr
+- nld
+- tha
+- jpn
+- heb
+- ell
+- kor
+- isl
+- dan
+- cat
+- slk
+- rus
+- kat
+- por
+- ben
+- fas
+- ekk
+- fin
+- tur
+- swe
+- ind
+- ces
+- lit
+- slv
+- vie
+- eus
+- bul
+- mlt
+- lvs
+- nob
+- hun
+- urd
+- ron
+- glg
+- gle
+- nno
+- ltg
+- yue
+- cmn
+- hrv
+- arb
+- bos
+- mkd
+- srp
+- hin
+- als
+- sqi
+- est
+- nor
+- lav
+- swa
+---
+
+<p align="center">
+  <img src="res/propella_logo.svg" alt="propella logo" width="150">
+</p>
+
+<h1 align="center">propella-1</h1>
+<p align="center"><em>propel your data curation to the next level. </em></p>
+
+propella-1 is a family of small multilingual LLMs that annotate text documents across six categories: core content, classification, quality & value, audience & purpose, safety & compliance, and geographic relevance. The annotations can be used to filter, select, and curate LLM training data at scale.
+
+
+---
+
+*Disclaimer: This is a research project and not an official ellamind product.*
+
+*Curate with propella. Evaluate with [elluminate](https://elluminate.de).*
+
+---
+
+## Highlights
+
+- **Annotate 18 properties**: Covers well-established dimensions like content quality and educational value, plus underexplored ones like reasoning indicators and time-sensitivity.
+- **Fast & accurate**: Small models (0.6B, 1.7B, 4B) that punch above their weight. Trained in fp8, ready for high-throughput inference.
+- **Any text, any format**: Handles web pages, PDFs, code, math, post-training data and more.
+- **Highly multilingual**: Supports 57 languages.
+
+<p align="center">
+  <img src="res/overall_scores_by_model.png" alt="overall-performance-plot">
+</p>
+
+## The propella-1 family of models
+| Model | Parameters | Performance| Docs/s (A100/H100) |
+|-------|:----------:|:-------------:|:------------------:|
+| [propella-1-4b](https://huggingface.co/ellamind/propella-1-4b) | 4B | 0.779 | 10.3 / 27.0 |
+| [propella-1-1.7b](https://huggingface.co/ellamind/propella-1-1.7b) | 1.7B | 0.737 | 17.8 / 39.1 |
+| [propella-1-0.6b](https://huggingface.co/ellamind/propella-1-0.6b) | 0.6B | 0.729 | 21.5 / 39.9 |
+
+
+## Properties
+propella-1 models evaluate documents across 18 properties organized into six categories:
+| Category | Property | Short Description |
+|----------|----------|-------------|
+| **Core Content** | Content Integrity | Completeness and technical quality of the content |
+| | Content Ratio | Proportion of content vs. navigation/UI elements |
+| | Content Length | Amount of substantive content |
+| **Classification** | One-Sentence Description | Ultra-short neutral description of the document |
+| | Content Type | Functional structure and purpose |
+| | Business Sector | Industry domain relevance |
+| | Technical Content | Type and intensity of specialized knowledge |
+| **Quality & Value** | Content Quality | Overall writing and presentation quality |
+| | Information Density | Ratio of valuable information to redundancy |
+| | Educational Value | Potential for teaching and learning |
+| | Reasoning Indicators | Presence of logical reasoning and analysis |
+| **Audience & Purpose** | Audience Level | Target sophistication level |
+| | Commercial Bias | Commercial influence on objectivity |
+| | Time-Sensitivity | How content value changes over time |
+| **Safety & Compliance** | Content Safety | Presence of inappropriate or harmful content |
+| | PII Presence | Contains personally identifiable information |
+| **Geographic** | Regional Relevance | Primary regional/cultural context |
+| | Country Relevance | Specific country relevance |
+
+Read the [property reference](property_descriptions.md) for detailed definitions and enum values.
+
+
+## Datasets annotated with propella-1
+TBA. Stay tuned.
+
+
+## Input
+A text document in any of the [57 supported languages](#language-support).
+
+### Output
+
+A JSON object containing annotations. The output strictly conforms to a predefined schema with enumerated values for categorical properties.
+
+<details>
+<summary><strong>Example output</strong></summary>
+
+```json
+{
+  "content_integrity": "complete",
+  "content_ratio": "mostly_content",
+  "content_length": "moderate",
+  "one_sentence_description": "Technical documentation explaining how to define and evaluate structured LLM output schemas using elluminate's Python client.",
+  "content_type": [
+    "technical_documentation",
+    "instructional",
+    "source_code"
+  ],
+  "business_sector": [
+    "technology_software"
+  ],
+  "technical_content": [
+    "code_heavy"
+  ],
+  "information_density": "dense",
+  "content_quality": "excellent",
+  "audience_level": "advanced",
+  "commercial_bias": "minimal",
+  "time_sensitivity": "slowly_changing",
+  "content_safety": "safe",
+  "educational_value": "high",
+  "reasoning_indicators": "explanatory",
+  "pii_presence": "no_pii",
+  "regional_relevance": [
+    "global"
+  ],
+  "country_relevance": [
+    "none"
+  ]
+}
+```
+</details>
+
+## Usage
+See `propella.py` for prompts and schemas. We recommend enforcing a strict json schema without any whitespace for error-free generation.
+
+### Serving
+We recommend serving propella models with [SGLang](https://github.com/sgl-project/sglang) and the [llguidance](https://github.com/guidance-ai/llguidance) structured output backend:
+```bash
+python -m sglang.launch_server \
+    --model-path outputs/propella-1-4b \
+    --host 0.0.0.0 \
+    --port 8000 \
+    --context-length 65536 \
+    --max-running-requests 256 \
+    --chunked-prefill-size 8192 \
+    --enable-mixed-chunk \
+    --num-continuous-decode-steps 8 \
+    --grammar-backend llguidance \
+    --mem-fraction-static 0.7
+```
+
+<details>
+<summary>fp8 on H100</summary>
+
+```bash
+python -m sglang.launch_server \
+    --model-path outputs/propella-1-4b \
+    --quantization w8a8_fp8 \
+    --kv-cache-dtype fp8_e4m3 \
+    --host 0.0.0.0 \
+    --port 8000 \
+    --context-length 65536 \
+    --max-running-requests 256 \
+    --chunked-prefill-size 8192 \
+    --enable-mixed-chunk \
+    --num-continuous-decode-steps 8 \
+    --grammar-backend llguidance \
+    --mem-fraction-static 0.7
+```
+
+</details>
+
+For single-node multi-GPU we recommend increasing `data-parallel-size`.
+For large scale offline inference on SLURM clusters we use [inference-hive](https://github.com/ellamind/inference-hive).
+
+### Sending request via OpenAI SDK
+```python
+from openai import OpenAI
+from propella import (
+    create_messages,
+    AnnotationResponse,
+    get_annotation_response_schema,
+)
+
+document = "Hi, its me Max."
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
+
+response = client.chat.completions.create(
+    model="ellamind/propella-1-4b",
+    messages=create_messages(document),
+    response_format={
+        "type": "json_schema",
+        "json_schema": {
+            "name": "AnnotationResponse",
+            "schema": get_annotation_response_schema(flatten=True, compact_whitespace=True),
+            "strict": True,
+        }
+    },
+)
+response_content = response.choices[0].message.content
+result = AnnotationResponse.model_validate_json(response_content)
+print(result.model_dump_json(indent=4))
+```
+<details>
+<summary>Result</summary>
+
+```json
+{
+    "content_integrity": "complete",
+    "content_ratio": "complete_content",
+    "content_length": "minimal",
+    "one_sentence_description": "A short personal greeting introducing someone named Max.",
+    "content_type": [
+        "conversational"
+    ],
+    "business_sector": [
+        "general_interest"
+    ],
+    "technical_content": [
+        "non_technical"
+    ],
+    "information_density": "dense",
+    "content_quality": "good",
+    "audience_level": "general",
+    "commercial_bias": "none",
+    "time_sensitivity": "evergreen",
+    "content_safety": "safe",
+    "educational_value": "none",
+    "reasoning_indicators": "none",
+    "pii_presence": "contains_pii",
+    "regional_relevance": [
+        "culturally_neutral"
+    ],
+    "country_relevance": [
+        "none"
+    ]
+}
+```
+</details>
+
+## Throughput
+
+The throughput results below provide a rough estimate for GPU-hours required to annotate 1M documents. After a short warmup, we run inference for 5k documents, sending 1k concurrent requests to the SGLang server.
+
+| Model | GPU | Docs/s | hours-per-1M docs | Prompt TPS | Output TPS | Total TPS |
+|-------|-----|--------|-------------------|------------|------------|-----------|
+| propella-1-4b | A100 80GB | 10.3 | 27.0 | 19.1k | 1.5k | 20.5k |
+| propella-1-4b | H100 96GB | 22.4 | 12.4 | 41.6k | 3.2k | 44.8k |
+| propella-1-4b (fp8) | H100 96GB | 27.0 | 10.3 | 50.1k | 3.9k | 54.0k |
+| propella-1-1.7b | A100 80GB | 17.8 | 15.6 | 33.0k | 2.6k | 35.6k |
+| propella-1-1.7b | H100 96GB | 35.8 | 7.8 | 66.5k | 5.2k | 71.8k |
+| propella-1-1.7b (fp8) | H100 96GB | 39.1 | 7.1 | 72.7k | 5.7k | 78.4k |
+| propella-1-0.6b | H100 96GB | 39.9 | 7.0 | 74.2k | 5.7k | 79.9k |
+| propella-1-0.6b | A100 80GB | 21.5 | 12.9 | 40.0k | 3.1k | 43.1k |
+
+
+## Evaluation
+
+We evaluate the propella-1 models on a test set containing 3k documents. For these documents we obtain annotations from Gemini-3-Pro (reasoning_effort: high), which we consider as groundtruth labels under the assumption that they represent the upper limit in terms of annotation quality.
+
+All baseline models use the detailed annotator system- and user-prompts as defined in `propella.py`. For throughput reasons, the propella-1 models use a very short, propella-1 specific prompt. We also tested some baseline models with the propella-1 prompt, always leading to worse performance as the prompt lacks details.
+
+### Metrics by Property Type
+
+Properties are grouped into three categories, each evaluated with an appropriate metric:
+
+- **Ordinal Properties** (11 properties): **QWK** (Quadratic Weighted Kappa), which measures agreement while accounting for the ordinal nature of labels. It penalizes larger disagreements more heavily.
+- **Binary Properties** (1 property): **F1**, the harmonic mean of precision and recall.
+- **Multi-select Properties** (5 properties): **IoU** (Jaccard Index), intersection-over-union averaged across samples.
+- **Free-text Properties** (1 property): The `one_sentence_description` property is excluded from quantitative evaluation.
+
+<p align="center">
+  <a href="res/per_property_scores_by_model.png">
+    <img src="res/per_property_scores_by_model.png" alt="per-property-performance-plot">
+  </a>
+</p>
+
+### Overall Score
+
+The overall score is a weighted average of the primary metric for each property type:
+
+```
+overall = (11/17 × avg_QWK) + (1/17 × avg_F1) + (5/17 × avg_IoU)
+```
+
+<p align="center">
+  <a href="res/overall_scores_by_model.png">
+    <img src="res/overall_scores_by_model.png" alt="overall-performance-plot">
+  </a>
+</p>
+
+## Language Support
+The training data for propella-1 contains documents in 57 languages:
+| lang_script | percent |
+|-------------|---------|
+| eng_Latn | 35.08 |
+| spa_Latn | 3.98 |
+| ita_Latn | 3.97 |
+| fra_Latn | 3.95 |
+| deu_Latn | 3.86 |
+| pol_Latn | 3.81 |
+| code | 2.82 |
+| math | 2.77 |
+| sft | 2.41 |
+| ukr_Cyrl | 0.95 |
+| nld_Latn | 0.95 |
+| tha_Thai | 0.95 |
+| jpn_Jpan | 0.94 |
+| heb_Hebr | 0.94 |
+| ell_Grek | 0.93 |
+| kor_Hang | 0.93 |
+| isl_Latn | 0.93 |
+| dan_Latn | 0.92 |
+| cat_Latn | 0.92 |
+| slk_Latn | 0.92 |
+| rus_Cyrl | 0.91 |
+| kat_Geor | 0.9 |
+| por_Latn | 0.9 |
+| ben_Beng | 0.9 |
+| fas_Arab | 0.89 |
+| ekk_Latn | 0.89 |
+| fin_Latn | 0.89 |
+| tur_Latn | 0.89 |
+| swe_Latn | 0.88 |
+| ind_Latn | 0.88 |
+| ces_Latn | 0.88 |
+| lit_Latn | 0.88 |
+| slv_Latn | 0.87 |
+| vie_Latn | 0.87 |
+| eus_Latn | 0.87 |
+| bul_Cyrl | 0.86 |
+| mlt_Latn | 0.86 |
+| lvs_Latn | 0.86 |
+| nob_Latn | 0.86 |
+| hun_Latn | 0.85 |
+| urd_Arab | 0.85 |
+| ron_Latn | 0.84 |
+| glg_Latn | 0.83 |
+| gle_Latn | 0.83 |
+| nno_Latn | 0.83 |
+| ltg_Latn | 0.77 |
+| yue_Hant | 0.49 |
+| cmn_Hant | 0.48 |
+| hrv_Latn | 0.43 |
+| arb_Arab | 0.39 |
+| bos_Latn | 0.39 |
+| mkd_Cyrl | 0.39 |
+| srp_Latn | 0.37 |
+| cmn_Hani | 0.37 |
+| hin_Deva | 0.36 |
+| srp_Cyrl | 0.36 |
+| als_Latn | 0.35 |
+| sqi_Latn | 0.03 |
+| est_Latn | 0.02 |
+| nor_Latn | 0.02 |
+| lav_Latn | 0.02 |
+| swa_Latn | 0.02 |
+
+## Acknowledgements
+* This project is supported by the OpenEuroLLM project, co-funded by the Digital Europe Programme under GA no. 101195233. For more information see [openeurollm.eu](openeurollm.eu).
+* This project is supported by the LLMs4EU project, co-funded by the Digital Europe Programme under GA no. 101198470. For more information see [LLMs4EU website](https://www.alt-edic.eu/projects/llms4eu/).
+* This project is supported by the German Federal Ministry for Economic Affairs and Energy (BMWE) under the soofi (Sovereign Open Source Foundation Models for European Intelligence) project.
+* We acknowledge the EuroHPC Joint Undertaking for supporting this project through access to the EuroHPC supercomputer LEONARDO, hosted by CINECA (Italy) and the LEONARDO consortium, through an EuroHPC AI Factory Large Scale Access call.
+* We thank the AI Service Center for Sensitive and Critical Infrastructures (KISSKI), hosted by GWDG, for additional compute access.
+
+<img src="res/eu_cofunding.png" alt="eu-cofunding-logo" width="300" style="vertical-align: middle;">

added_tokens.json

@@ -0,0 +1,28 @@
+{
+  "</think>": 151668,
+  "</tool_call>": 151658,
+  "</tool_response>": 151666,
+  "<think>": 151667,
+  "<tool_call>": 151657,
+  "<tool_response>": 151665,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

chat_template.jinja

@@ -0,0 +1,13 @@
+{%- if messages[0].role == 'system' %}
+    {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+{%- endif %}
+{%- for message in messages %}
+    {%- if message.role == "user" or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>assistant\n' + message.content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

config.json

@@ -0,0 +1,68 @@
+{
+  "architectures": [
+    "Qwen3ForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "dtype": "bfloat16",
+  "eos_token_id": 151645,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 2560,
+  "initializer_range": 0.02,
+  "intermediate_size": 9728,
+  "layer_types": [
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention"
+  ],
+  "max_position_embeddings": 262144,
+  "max_window_layers": 36,
+  "model_type": "qwen3",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 36,
+  "num_key_value_heads": 8,
+  "pad_token_id": 151643,
+  "rms_norm_eps": 1e-06,
+  "rope_scaling": null,
+  "rope_theta": 5000000,
+  "sliding_window": null,
+  "tie_word_embeddings": true,
+  "transformers_version": "4.57.1",
+  "use_cache": false,
+  "use_sliding_window": false,
+  "vocab_size": 151936
+}

generation_config.json

@@ -0,0 +1,12 @@
+{
+  "do_sample": true,
+  "eos_token_id": [
+    151645,
+    151643
+  ],
+  "pad_token_id": 151643,
+  "temperature": 0.7,
+  "top_k": 20,
+  "top_p": 0.8,
+  "transformers_version": "4.57.1"
+}

inference_example.py

@@ -0,0 +1,26 @@
+from openai import OpenAI
+from propella import (
+    create_messages,
+    AnnotationResponse,
+    get_annotation_response_schema,
+)
+
+document = "Hi, its me Max."
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
+
+response = client.chat.completions.create(
+    model="ellamind/propella-1-4b",
+    messages=create_messages(document),
+    response_format={
+        "type": "json_schema",
+        "json_schema": {
+            "name": "AnnotationResponse",
+            "schema": get_annotation_response_schema(flatten=True, compact_whitespace=True),
+            "strict": True,
+        }
+    },
+)
+response_content = response.choices[0].message.content
+result = AnnotationResponse.model_validate_json(response_content)
+print(result.model_dump_json(indent=4))

model-00001-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:593a0cf241a56e7f0e830365bcc9e72f128b3a2cc56a8994773467cc573ff02b
+size 4967215360

model-00002-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:787b95bc9da1801e0078afa670576b9b9701c420574b2c65af16101a28a94eb9
+size 3855679144

model.safetensors.index.json

@@ -0,0 +1,406 @@
+{
+  "metadata": {
+    "total_size": 8822848512
+  },
+  "weight_map": {
+    "lm_head.weight": "model-00002-of-00002.safetensors",
+    "model.embed_tokens.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.18.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.19.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.20.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.20.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.20.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.21.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.21.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.21.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.21.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.21.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.21.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.21.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.21.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.21.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.21.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.21.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.22.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.22.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.22.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.22.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.22.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.22.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.22.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.22.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.22.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.22.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.22.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.23.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.23.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.23.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.23.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.23.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.23.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.23.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.23.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.23.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.23.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.23.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.24.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.24.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.24.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.24.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.24.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.24.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.24.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.24.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.24.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.24.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.24.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.25.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.25.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.25.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.25.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.25.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.25.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.25.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.25.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.25.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.25.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.25.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.26.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.27.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.28.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.29.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.3.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.30.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.30.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.30.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.30.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.30.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.30.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.30.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.30.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.30.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.30.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.30.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.31.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.32.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.32.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.32.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.32.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.32.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.32.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.32.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.32.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.32.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.32.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.32.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.33.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.33.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.33.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.33.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.33.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.33.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.33.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.33.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.33.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.33.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.33.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.34.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.34.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.34.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.34.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.34.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.34.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.34.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.34.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.34.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.34.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.34.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.35.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.35.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.35.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.35.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.35.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.35.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.35.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.35.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.35.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.35.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.35.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.4.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.norm.weight": "model-00002-of-00002.safetensors"
+  }
+}

propella.py

@@ -0,0 +1,916 @@
+import json
+from copy import deepcopy
+from enum import Enum
+from pathlib import Path
+from typing import List, Type, Union
+
+from pydantic import BaseModel, ConfigDict, Field
+
+SYSTEM_PROMPT = """Annotate the document. Any language; assess quality within its linguistic norms. Respond with a JSON object:
+content_integrity: technical completeness (complete|mostly_complete|fragment|severely_degraded)
+content_ratio: content vs navigation/boilerplate ratio (complete_content|mostly_content|mixed_content|mostly_navigation|minimal_content)
+content_length: substantive words (substantial 2k+|moderate 500-2k|brief 100-500|minimal <100)
+one_sentence_description: neutral ~10 word summary in English
+content_type[]: functional purpose (analytical|instructional|reference|procedural|qa_structured|conversational|creative|transactional|boilerplate|news_report|opinion_editorial|review_critique|technical_documentation|specification_standard|legal_document|press_release|structured_data|source_code)
+business_sector[]: industry domain (academic_research|education_sector|technology_software|hardware_electronics|healthcare_medical|pharmaceutical_biotech|financial_services|legal_services|government_public|manufacturing_industrial|mining_resources|chemicals_materials|energy_utilities|retail_commerce|wholesale_distribution|real_estate_construction|transportation_logistics|automotive_industry|telecommunications|media_entertainment|advertising_marketing|hospitality_tourism|agriculture_food|environmental_services|aerospace_defense|insurance_industry|nonprofit_ngo|consulting_professional|human_resources|security_cyber|gaming_industry|gambling_betting|travel_aviation|food_beverage_hospitality|consumer_goods|general_interest|other)
+technical_content[]: specialized knowledge (code_heavy|math_heavy|scientific|data_heavy|engineering|basic_technical|non_technical)
+content_quality: writing/presentation quality (excellent|good|adequate|poor|unacceptable)
+information_density: signal vs padding (dense|adequate|moderate|thin|empty)
+educational_value: teaching potential (high|moderate|basic|minimal|none)
+reasoning_indicators: logical analysis depth (analytical|explanatory|basic_reasoning|minimal|none)
+audience_level: assumed background (expert|advanced|general|beginner|youth|children)
+commercial_bias: promotional influence (none|minimal|moderate|heavy|pure_marketing)
+time_sensitivity: temporal decay (evergreen|slowly_changing|regularly_updating|time_sensitive)
+content_safety: harmful content (safe|mild_concerns|nsfw|harmful|illegal)
+pii_presence: private individual data (no_pii|contains_pii)
+regional_relevance[]: geographic/cultural context (european|north_american|east_asian|south_asian|southeast_asian|middle_eastern|sub_saharan_african|latin_american|oceanian|central_asian|russian_sphere|global|culturally_neutral|indeterminate)
+country_relevance[]: specific countries as ISO names, or supranational|none
+"""
+
+USER_PROMPT = """<start_of_document>
+{content}
+<end_of_document>
+"""
+
+ANNOTATOR_SYSTEM_PROMPT = """You are an expert content analysis assistant specializing in document annotations for LLM pretraining data. Your team is curating a multilingual dataset for language model training. Your task is to analyze documents and annotate them with specific properties that will later on be used to filter the dataset. The user will provide a document inside of "<start_of_document>" and "<end_of_document>" tags. Analyze the content of the document systematically and objectively. Respond with your annotations in JSON format, following the annotation framework below.
+
+# Annotation Framework
+## Output Requirements
+- You must respond with a JSON object that matches the specified schema.
+- Use the exact enum values provided in the property descriptions.
+- Ensure all fields are included.
+- For multi-select properties, always return arrays (even if only one value applies). Multi-select fields: content_type, business_sector, technical_content, regional_relevance, country_relevance. All other properties are single-select strings.
+- Do not include any explanatory text, comments, or additional formatting.
+
+## Key Principles
+* Objective assessment: Base decisions on clear criteria, not subjective preferences.
+* Completeness: Address all properties for every document.
+* Consistency: Apply the same standards across all documents.
+* Multilinguality: The user provided document can be in any language, the language itself should not influence the annotations.
+
+## Properties to Annotate
+The annotation framework evaluates documents across 18 key properties organized into six main categories:
+
+**Core Content Properties:**
+- Content Integrity: Completeness and technical quality (complete, mostly_complete, fragment, severely_degraded)
+- Content Ratio: Proportion of meaningful content vs navigation/UI elements (complete_content, mostly_content, mixed_content, mostly_navigation, minimal_content)
+- Content Length: Amount of substantive content (substantial, moderate, brief, minimal)
+
+**Content Classification:**
+- One-Sentence Description: Ultra-short neutral description; exactly one sentence; target 8–15 words (soft max 20)
+- Content Type: Functional structure and purpose (analytical, instructional, reference, procedural, qa_structured, conversational, creative, transactional, boilerplate, news_report, opinion_editorial, review_critique, technical_documentation, specification_standard, legal_document, press_release, structured_data, source_code)
+- Business Sector: Industry domain relevance (see Detailed Property Descriptions for exact enum values)
+- Technical Content: Type and intensity of specialized knowledge (code_heavy, math_heavy, scientific, data_heavy, engineering, basic_technical, non_technical)
+
+**Quality and Value Assessment:**
+- Content Quality: Overall writing and presentation quality (excellent, good, adequate, poor, unacceptable)
+- Information Density: Ratio of valuable information to redundancy (dense, adequate, moderate, thin, empty)
+- Educational Value: Potential for teaching and learning (high, moderate, basic, minimal, none)
+- Reasoning Indicators: Presence of logical reasoning and analysis (analytical, explanatory, basic_reasoning, minimal, none)
+
+**Audience and Purpose:**
+- Audience Level: Target sophistication level (expert, advanced, general, beginner, youth, children)
+- Commercial Bias: Commercial influence on objectivity (none, minimal, moderate, heavy, pure_marketing)
+- Time-Sensitivity: How content value changes over time (evergreen, slowly_changing, regularly_updating, time_sensitive)
+
+**Safety and Compliance:**
+- Content Safety: Presence of inappropriate or harmful content (safe, mild_concerns, nsfw, harmful, illegal)
+- PII Presence: Contains personally identifiable information (no_pii, contains_pii)
+
+**Geographic Relevance:**
+- Regional Relevance: Primary regional context (european, north_american, east_asian, south_asian, southeast_asian, middle_eastern, sub_saharan_african, latin_american, oceanian, central_asian, russian_sphere, global, culturally_neutral, indeterminate)
+- Country Relevance: Specific country relevance (array of country names or special values: "supranational", "none")
+
+
+{property_descriptions}
+
+## JSON Schema for the Response
+Return a single JSON object that strictly conforms to the following JSON Schema:
+```json
+{json_schema}
+```
+
+## Multilingual Annotation Guidelines
+
+### Universal Principles
+1. **Evaluate content quality within language context** - Don't penalize non-English content for being non-English
+2. **Consider linguistic norms** - Writing styles, sentence lengths, and paragraph structures vary by language
+3. **Respect script directionality** - RTL languages (Arabic, Hebrew) may have different navigation patterns
+4. **Account for morphological complexity** - Agglutinative/polysynthetic languages pack more information per word
+
+### Language-Specific Considerations
+
+Here are some examples of language-specific considerations:
+
+**Chinese/Japanese:**
+- Character count more relevant than word count
+- Lack of spaces between words is normal
+- Mixed script usage (especially Japanese) is standard
+
+**Arabic/Hebrew/Persian:**
+- RTL text direction affects layout assessment
+- Diacritical marks may be absent in informal content
+- Mixed Arabic/English is common in technical content
+
+**Indian Languages (Hindi, Bengali, Tamil, etc.):**
+- Code-mixing with English is extremely common and acceptable
+- Technical terms often borrowed from English
+- Multiple scripts may appear in same document
+
+**European Languages:**
+- Formal/informal distinctions (tu/vous, du/Sie) indicate audience
+- Compound words affect word count metrics
+- Regional variants (Brazilian vs European Portuguese, Spanish vs Catalan, etc.) are both valid
+
+# Annotation Workflow
+- The user will provide a document in "<start_of_document>" and "<end_of_document>" tags. Analyze the content of the document systematically and objectively
+- You must respond with a valid JSON object that matches the schema above.
+- Use the exact enum values provided in the property descriptions
+- Ensure all required fields are included
+- For multi-select properties, always return arrays, even if only one value applies (content_type, business_sector, technical_content, regional_relevance, country_relevance). All other properties are single-select strings.
+- Do not include any explanatory text, comments, or formatting
+"""
+
+ANNOTATOR_USER_PROMPT = """Analyze the following document and provide annotations in JSON format according to the annotation framework. Return only the JSON object.
+<start_of_document>
+{content}
+<end_of_document>"""
+
+
+# Default max length for one_sentence_description field
+ONE_SENTENCE_DESCRIPTION_MAX_LENGTH = 200
+
+
+class ContentIntegrity(str, Enum):
+    """Content completeness and technical quality"""
+    COMPLETE = "complete"
+    MOSTLY_COMPLETE = "mostly_complete"
+    FRAGMENT = "fragment"
+    SEVERELY_DEGRADED = "severely_degraded"
+
+
+class ContentRatio(str, Enum):
+    """Ratio of meaningful content vs navigation/UI elements"""
+    COMPLETE_CONTENT = "complete_content"
+    MOSTLY_CONTENT = "mostly_content"
+    MIXED_CONTENT = "mixed_content"
+    MOSTLY_NAVIGATION = "mostly_navigation"
+    MINIMAL_CONTENT = "minimal_content"
+
+
+class ContentLength(str, Enum):
+    """Amount of substantive content"""
+    SUBSTANTIAL = "substantial"  # 500+ words
+    MODERATE = "moderate"        # 100-500 words
+    BRIEF = "brief"             # 20-100 words
+    MINIMAL = "minimal"         # <20 words
+
+
+class ContentType(str, Enum):
+    """Primary purpose and type of content"""
+    ANALYTICAL = "analytical"
+    INSTRUCTIONAL = "instructional"
+    REFERENCE = "reference"
+    PROCEDURAL = "procedural"
+    QA_STRUCTURED = "qa_structured"
+    CONVERSATIONAL = "conversational"
+    CREATIVE = "creative"
+    TRANSACTIONAL = "transactional"
+    BOILERPLATE = "boilerplate"
+    NEWS_REPORT = "news_report"
+    OPINION_EDITORIAL = "opinion_editorial"
+    REVIEW_CRITIQUE = "review_critique"
+    TECHNICAL_DOCUMENTATION = "technical_documentation"
+    SPECIFICATION_STANDARD = "specification_standard"
+    LEGAL_DOCUMENT = "legal_document"
+    PRESS_RELEASE = "press_release"
+    STRUCTURED_DATA = "structured_data"
+    SOURCE_CODE = "source_code"
+
+
+class BusinessSector(str, Enum):
+    """Industry domain(s) for sector classification (multi-select)"""
+    ACADEMIC_RESEARCH = "academic_research"
+    EDUCATION_SECTOR = "education_sector"
+    TECHNOLOGY_SOFTWARE = "technology_software"
+    HARDWARE_ELECTRONICS = "hardware_electronics"
+    HEALTHCARE_MEDICAL = "healthcare_medical"
+    PHARMACEUTICAL_BIOTECH = "pharmaceutical_biotech"
+    FINANCIAL_SERVICES = "financial_services"
+    LEGAL_SERVICES = "legal_services"
+    GOVERNMENT_PUBLIC = "government_public"
+    MANUFACTURING_INDUSTRIAL = "manufacturing_industrial"
+    MINING_RESOURCES = "mining_resources"
+    CHEMICALS_MATERIALS = "chemicals_materials"
+    ENERGY_UTILITIES = "energy_utilities"
+    RETAIL_COMMERCE = "retail_commerce"
+    WHOLESALE_DISTRIBUTION = "wholesale_distribution"
+    REAL_ESTATE_CONSTRUCTION = "real_estate_construction"
+    TRANSPORTATION_LOGISTICS = "transportation_logistics"
+    AUTOMOTIVE_INDUSTRY = "automotive_industry"
+    TELECOMMUNICATIONS = "telecommunications"
+    MEDIA_ENTERTAINMENT = "media_entertainment"
+    ADVERTISING_MARKETING = "advertising_marketing"
+    HOSPITALITY_TOURISM = "hospitality_tourism"
+    AGRICULTURE_FOOD = "agriculture_food"
+    ENVIRONMENTAL_SERVICES = "environmental_services"
+    AEROSPACE_DEFENSE = "aerospace_defense"
+    INSURANCE_INDUSTRY = "insurance_industry"
+    NONPROFIT_NGO = "nonprofit_ngo"
+    CONSULTING_PROFESSIONAL = "consulting_professional"
+    HUMAN_RESOURCES = "human_resources"
+    SECURITY_CYBER = "security_cyber"
+    GAMING_INDUSTRY = "gaming_industry"
+    GAMBLING_BETTING = "gambling_betting"
+    TRAVEL_AVIATION = "travel_aviation"
+    FOOD_BEVERAGE_HOSPITALITY = "food_beverage_hospitality"
+    CONSUMER_GOODS = "consumer_goods"
+    GENERAL_INTEREST = "general_interest"
+    OTHER = "other"
+
+
+class TechnicalContent(str, Enum):
+    """Type and intensity of specialized technical knowledge"""
+    CODE_HEAVY = "code_heavy"
+    MATH_HEAVY = "math_heavy"
+    SCIENTIFIC = "scientific"
+    DATA_HEAVY = "data_heavy"
+    ENGINEERING = "engineering"
+    BASIC_TECHNICAL = "basic_technical"
+    NON_TECHNICAL = "non_technical"
+
+
+class InformationDensity(str, Enum):
+    """Ratio of valuable information to redundancy and padding"""
+    DENSE = "dense"
+    ADEQUATE = "adequate"
+    MODERATE = "moderate"
+    THIN = "thin"
+    EMPTY = "empty"
+
+
+class ContentQuality(str, Enum):
+    """Overall quality considering writing, value, and presentation"""
+    EXCELLENT = "excellent"
+    GOOD = "good"
+    ADEQUATE = "adequate"
+    POOR = "poor"
+    UNACCEPTABLE = "unacceptable"
+
+
+class AudienceLevel(str, Enum):
+    """Intended sophistication level and background knowledge assumptions"""
+    EXPERT = "expert"
+    ADVANCED = "advanced"
+    GENERAL = "general"
+    BEGINNER = "beginner"
+    YOUTH = "youth"
+    CHILDREN = "children"
+
+
+class CommercialBias(str, Enum):
+    """Commercial influence on objectivity and informational value"""
+    NONE = "none"
+    MINIMAL = "minimal"
+    MODERATE = "moderate"
+    HEAVY = "heavy"
+    PURE_MARKETING = "pure_marketing"
+
+
+class ContentSafety(str, Enum):
+    """Presence of inappropriate, harmful, or legally problematic content"""
+    SAFE = "safe"
+    MILD_CONCERNS = "mild_concerns"
+    NSFW = "nsfw"
+    HARMFUL = "harmful"
+    ILLEGAL = "illegal"
+
+
+class EducationalValue(str, Enum):
+    """Potential for teaching, learning, and knowledge transfer"""
+    HIGH = "high"
+    MODERATE = "moderate"
+    BASIC = "basic"
+    MINIMAL = "minimal"
+    NONE = "none"
+
+
+class ReasoningIndicators(str, Enum):
+    """Presence and quality of logical reasoning and analysis"""
+    ANALYTICAL = "analytical"
+    EXPLANATORY = "explanatory"
+    BASIC_REASONING = "basic_reasoning"
+    MINIMAL = "minimal"
+    NONE = "none"
+
+
+class RegionalRelevance(str, Enum):
+    """Primary regional, cultural, or geopolitical sphere(s)"""
+    EUROPEAN = "european"
+    NORTH_AMERICAN = "north_american"
+    EAST_ASIAN = "east_asian"
+    SOUTH_ASIAN = "south_asian"
+    SOUTHEAST_ASIAN = "southeast_asian"
+    MIDDLE_EASTERN = "middle_eastern"
+    SUB_SAHARAN_AFRICAN = "sub_saharan_african"
+    LATIN_AMERICAN = "latin_american"
+    OCEANIAN = "oceanian"
+    CENTRAL_ASIAN = "central_asian"
+    RUSSIAN_SPHERE = "russian_sphere"
+    GLOBAL = "global"
+    CULTURALLY_NEUTRAL = "culturally_neutral"
+    INDETERMINATE = "indeterminate"
+
+
+class TimeSensitivity(str, Enum):
+    """How time-sensitive the content is"""
+    EVERGREEN = "evergreen"
+    SLOWLY_CHANGING = "slowly_changing"
+    REGULARLY_UPDATING = "regularly_updating"
+    TIME_SENSITIVE = "time_sensitive"
+
+
+class PiiPresence(str, Enum):
+    """Presence of personally identifiable information"""
+    NO_PII = "no_pii"
+    CONTAINS_PII = "contains_pii"
+
+
+class Country(str, Enum):
+    """
+    Country names for country relevance classification.
+    Based on ISO 3166-1 standard - the authoritative international standard 
+    for country codes maintained by the International Organization for Standardization.
+    
+    Includes all 249 entities from ISO 3166-1: 193 UN member states, 
+    2 UN observer states, plus dependent territories and special areas.
+    
+    References:
+    - https://en.wikipedia.org/wiki/List_of_ISO_3166_country_codes
+    - https://en.wikipedia.org/wiki/List_of_countries_and_dependencies_by_area
+    """
+    
+    # UN Member States (193 total) and UN Observer States (2 total)
+    AFGHANISTAN = "afghanistan"
+    ALBANIA = "albania"
+    ALGERIA = "algeria"
+    ANDORRA = "andorra"
+    ANGOLA = "angola"
+    ANTIGUA_AND_BARBUDA = "antigua_and_barbuda"
+    ARGENTINA = "argentina"
+    ARMENIA = "armenia"
+    AUSTRALIA = "australia"
+    AUSTRIA = "austria"
+    AZERBAIJAN = "azerbaijan"
+    BAHAMAS = "bahamas"
+    BAHRAIN = "bahrain"
+    BANGLADESH = "bangladesh"
+    BARBADOS = "barbados"
+    BELARUS = "belarus"
+    BELGIUM = "belgium"
+    BELIZE = "belize"
+    BENIN = "benin"
+    BHUTAN = "bhutan"
+    BOLIVIA = "bolivia"
+    BOSNIA_AND_HERZEGOVINA = "bosnia_and_herzegovina"
+    BOTSWANA = "botswana"
+    BRAZIL = "brazil"
+    BRUNEI = "brunei"
+    BULGARIA = "bulgaria"
+    BURKINA_FASO = "burkina_faso"
+    BURUNDI = "burundi"
+    CABO_VERDE = "cabo_verde"
+    CAMBODIA = "cambodia"
+    CAMEROON = "cameroon"
+    CANADA = "canada"
+    CENTRAL_AFRICAN_REPUBLIC = "central_african_republic"
+    CHAD = "chad"
+    CHILE = "chile"
+    CHINA = "china"
+    COLOMBIA = "colombia"
+    COMOROS = "comoros"
+    CONGO = "congo"
+    CONGO_DEMOCRATIC_REPUBLIC = "congo_democratic_republic"
+    COOK_ISLANDS = "cook_islands"
+    COSTA_RICA = "costa_rica"
+    CROATIA = "croatia"
+    CUBA = "cuba"
+    CYPRUS = "cyprus"
+    CZECH_REPUBLIC = "czech_republic"
+    DENMARK = "denmark"
+    DJIBOUTI = "djibouti"
+    DOMINICA = "dominica"
+    DOMINICAN_REPUBLIC = "dominican_republic"
+    ECUADOR = "ecuador"
+    EGYPT = "egypt"
+    EL_SALVADOR = "el_salvador"
+    EQUATORIAL_GUINEA = "equatorial_guinea"
+    ERITREA = "eritrea"
+    ESTONIA = "estonia"
+    ESWATINI = "eswatini"
+    ETHIOPIA = "ethiopia"
+    FIJI = "fiji"
+    FINLAND = "finland"
+    FRANCE = "france"
+    GABON = "gabon"
+    GAMBIA = "gambia"
+    GEORGIA = "georgia"
+    GERMANY = "germany"
+    GHANA = "ghana"
+    GREECE = "greece"
+    GRENADA = "grenada"
+    GUATEMALA = "guatemala"
+    GUINEA = "guinea"
+    GUINEA_BISSAU = "guinea_bissau"
+    GUYANA = "guyana"
+    HAITI = "haiti"
+    HONDURAS = "honduras"
+    HUNGARY = "hungary"
+    ICELAND = "iceland"
+    INDIA = "india"
+    INDONESIA = "indonesia"
+    IRAN = "iran"
+    IRAQ = "iraq"
+    IRELAND = "ireland"
+    ISRAEL = "israel"
+    ITALY = "italy"
+    IVORY_COAST = "ivory_coast"
+    JAMAICA = "jamaica"
+    JAPAN = "japan"
+    JORDAN = "jordan"
+    KAZAKHSTAN = "kazakhstan"
+    KENYA = "kenya"
+    KIRIBATI = "kiribati"
+    NORTH_KOREA = "north_korea"
+    SOUTH_KOREA = "south_korea"
+    KOSOVO = "kosovo"
+    KUWAIT = "kuwait"
+    KYRGYZSTAN = "kyrgyzstan"
+    LAOS = "laos"
+    LATVIA = "latvia"
+    LEBANON = "lebanon"
+    LESOTHO = "lesotho"
+    LIBERIA = "liberia"
+    LIBYA = "libya"
+    LIECHTENSTEIN = "liechtenstein"
+    LITHUANIA = "lithuania"
+    LUXEMBOURG = "luxembourg"
+    MADAGASCAR = "madagascar"
+    MALAWI = "malawi"
+    MALAYSIA = "malaysia"
+    MALDIVES = "maldives"
+    MALI = "mali"
+    MALTA = "malta"
+    MARSHALL_ISLANDS = "marshall_islands"
+    MAURITANIA = "mauritania"
+    MAURITIUS = "mauritius"
+    MEXICO = "mexico"
+    MICRONESIA = "micronesia"
+    MOLDOVA = "moldova"
+    MONACO = "monaco"
+    MONGOLIA = "mongolia"
+    MONTENEGRO = "montenegro"
+    MOROCCO = "morocco"
+    MOZAMBIQUE = "mozambique"
+    MYANMAR = "myanmar"
+    NAMIBIA = "namibia"
+    NAURU = "nauru"
+    NEPAL = "nepal"
+    NETHERLANDS = "netherlands"
+    NEW_ZEALAND = "new_zealand"
+    NICARAGUA = "nicaragua"
+    NIGER = "niger"
+    NIGERIA = "nigeria"
+    NIUE = "niue"
+    NORTH_MACEDONIA = "north_macedonia"
+    NORWAY = "norway"
+    OMAN = "oman"
+    PAKISTAN = "pakistan"
+    PALAU = "palau"
+    PALESTINE = "palestine"  # UN Observer State
+    PANAMA = "panama"
+    PAPUA_NEW_GUINEA = "papua_new_guinea"
+    PARAGUAY = "paraguay"
+    PERU = "peru"
+    PHILIPPINES = "philippines"
+    POLAND = "poland"
+    PORTUGAL = "portugal"
+    QATAR = "qatar"
+    ROMANIA = "romania"
+    RUSSIA = "russia"
+    RWANDA = "rwanda"
+    SAINT_KITTS_AND_NEVIS = "saint_kitts_and_nevis"
+    SAINT_LUCIA = "saint_lucia"
+    SAINT_VINCENT_AND_THE_GRENADINES = "saint_vincent_and_the_grenadines"
+    SAMOA = "samoa"
+    SAN_MARINO = "san_marino"
+    SAO_TOME_AND_PRINCIPE = "sao_tome_and_principe"
+    SAUDI_ARABIA = "saudi_arabia"
+    SENEGAL = "senegal"
+    SERBIA = "serbia"
+    SEYCHELLES = "seychelles"
+    SIERRA_LEONE = "sierra_leone"
+    SINGAPORE = "singapore"
+    SLOVAKIA = "slovakia"
+    SLOVENIA = "slovenia"
+    SOLOMON_ISLANDS = "solomon_islands"
+    SOMALIA = "somalia"
+    SOUTH_AFRICA = "south_africa"
+    SOUTH_SUDAN = "south_sudan"
+    SPAIN = "spain"
+    SRI_LANKA = "sri_lanka"
+    SUDAN = "sudan"
+    SURINAME = "suriname"
+    SWEDEN = "sweden"
+    SWITZERLAND = "switzerland"
+    SYRIA = "syria"
+    TAJIKISTAN = "tajikistan"
+    TANZANIA = "tanzania"
+    THAILAND = "thailand"
+    TIMOR_LESTE = "timor_leste"
+    TOGO = "togo"
+    TONGA = "tonga"
+    TRINIDAD_AND_TOBAGO = "trinidad_and_tobago"
+    TUNISIA = "tunisia"
+    TURKEY = "turkey"
+    TURKMENISTAN = "turkmenistan"
+    TUVALU = "tuvalu"
+    UGANDA = "uganda"
+    UKRAINE = "ukraine"
+    UNITED_ARAB_EMIRATES = "united_arab_emirates"
+    UNITED_KINGDOM = "united_kingdom"
+    UNITED_STATES = "united_states"
+    URUGUAY = "uruguay"
+    UZBEKISTAN = "uzbekistan"
+    VANUATU = "vanuatu"
+    VATICAN_CITY = "vatican_city"  # UN Observer State
+    VENEZUELA = "venezuela"
+    VIETNAM = "vietnam"
+    YEMEN = "yemen"
+    ZAMBIA = "zambia"
+    ZIMBABWE = "zimbabwe"
+    
+    # # Dependent Territories and Special Administrative Regions (from ISO 3166-1)
+    ALAND_ISLANDS = "aland_islands"  # Finland
+    AMERICAN_SAMOA = "american_samoa"  # United States
+    ANGUILLA = "anguilla"  # United Kingdom
+    ANTARCTICA = "antarctica"  # Antarctic Treaty
+    ARUBA = "aruba"  # Netherlands
+    ASCENSION_ISLAND = "ascension_island"  # United Kingdom
+    BERMUDA = "bermuda"  # United Kingdom
+    BRITISH_VIRGIN_ISLANDS = "british_virgin_islands"  # United Kingdom
+    CAYMAN_ISLANDS = "cayman_islands"  # United Kingdom
+    CHRISTMAS_ISLAND = "christmas_island"  # Australia
+    COCOS_ISLANDS = "cocos_islands"  # Australia
+    CURACAO = "curacao"  # Netherlands
+    FALKLAND_ISLANDS = "falkland_islands"  # United Kingdom
+    FAROE_ISLANDS = "faroe_islands"  # Denmark
+    FRENCH_GUIANA = "french_guiana"  # France
+    FRENCH_POLYNESIA = "french_polynesia"  # France
+    GIBRALTAR = "gibraltar"  # United Kingdom
+    GREENLAND = "greenland"  # Denmark
+    GUADELOUPE = "guadeloupe"  # France
+    GUAM = "guam"  # United States
+    GUERNSEY = "guernsey"  # United Kingdom
+    HONG_KONG = "hong_kong"  # China
+    ISLE_OF_MAN = "isle_of_man"  # United Kingdom
+    JERSEY = "jersey"  # United Kingdom
+    MACAU = "macau"  # China
+    MARTINIQUE = "martinique"  # France
+    MAYOTTE = "mayotte"  # France
+    MONTSERRAT = "montserrat"  # United Kingdom
+    NEW_CALEDONIA = "new_caledonia"  # France
+    NORFOLK_ISLAND = "norfolk_island"  # Australia
+    NORTHERN_MARIANA_ISLANDS = "northern_mariana_islands"  # United States
+    PITCAIRN_ISLANDS = "pitcairn_islands"  # United Kingdom
+    PUERTO_RICO = "puerto_rico"  # United States
+    REUNION = "reunion"  # France
+    SAINT_BARTHELEMY = "saint_barthelemy"  # France
+    SAINT_HELENA = "saint_helena"  # United Kingdom
+    SAINT_MARTIN = "saint_martin"  # France
+    SAINT_PIERRE_AND_MIQUELON = "saint_pierre_and_miquelon"  # France
+    SINT_MAARTEN = "sint_maarten"  # Netherlands
+    SVALBARD_AND_JAN_MAYEN = "svalbard_and_jan_mayen"  # Norway
+    TAIWAN = "taiwan"  # China (disputed)
+    TOKELAU = "tokelau"  # New Zealand
+    TRISTAN_DA_CUNHA = "tristan_da_cunha"  # United Kingdom
+    TURKS_AND_CAICOS_ISLANDS = "turks_and_caicos_islands"  # United Kingdom
+    US_VIRGIN_ISLANDS = "us_virgin_islands"  # United States
+    WALLIS_AND_FUTUNA = "wallis_and_futuna"  # France
+    WESTERN_SAHARA = "western_sahara"  # Disputed
+
+
+class CountryRelevanceSpecial(str, Enum):
+    """
+    Special values for country relevance classification from annotation guidelines.
+    These are used when content doesn't relate to specific countries.
+    """
+    SUPRANATIONAL = "supranational"
+    NONE = "none"
+
+
+def create_annotation_response_model(
+    one_sentence_description_max_length: int = ONE_SENTENCE_DESCRIPTION_MAX_LENGTH,
+) -> Type[BaseModel]:
+    """
+    Factory function to create an AnnotationResponse model with configurable max_length
+    for the one_sentence_description field.
+    
+    Args:
+        one_sentence_description_max_length: Maximum length for the one_sentence_description field.
+                                             Defaults to ONE_SENTENCE_DESCRIPTION_MAX_LENGTH (200).
+    
+    Returns:
+        A Pydantic model class with the specified configuration.
+    """
+    
+    class _AnnotationResponse(BaseModel):
+        """
+        Property annotation pydantic model for LLM pretraining data.
+        It captures all 18 properties as defined in the annotation guidelines for consistently identifying high-value content for language model training.
+        """
+        
+        # Property 1: Content Integrity
+        content_integrity: ContentIntegrity = Field(
+            ...,
+            description="Completeness and technical quality of the content itself"
+        )
+        
+        # Property 2: Content Ratio  
+        content_ratio: ContentRatio = Field(
+            ...,
+            description="Ratio of meaningful content vs navigation/UI elements"
+        )
+        
+        # Property 3: Content Length
+        content_length: ContentLength = Field(
+            ...,
+            description="Amount of substantive content, ignoring navigation and boilerplate"
+        )
+        
+        # Property 4: One-Sentence Description
+        one_sentence_description: str = Field(
+            ...,
+            description="Ultra-short neutral description of the document. Exactly one sentence. Target 8–15 words (soft max 20). Neutral tone; avoid boilerplate intros and calls to action.",
+            max_length=one_sentence_description_max_length,
+        )
+        
+        # Property 5: Content Type (multi-select)
+        content_type: List[ContentType] = Field(
+            ...,
+            description="Primary purpose and type of content - always return an array (one or more types)",
+            min_length=1,
+            max_length=5
+        )
+        
+        # Property 6: Business Sector (multi-select)
+        business_sector: List[BusinessSector] = Field(
+            ...,
+            description="Industry sector(s) - always return an array (one or more sectors)",
+            min_length=1,
+            max_length=10
+        )
+        
+        # Property 7: Technical Content (multi-select)
+        technical_content: List[TechnicalContent] = Field(
+            ...,
+            description="Type and intensity of specialized technical knowledge - always return an array (one or more types)",
+            min_length=1
+        )
+        
+        # Property 8: Information Density
+        information_density: InformationDensity = Field(
+            ...,
+            description="Ratio of valuable information to redundancy, padding, and repetition"
+        )
+        
+        # Property 9: Content Quality
+        content_quality: ContentQuality = Field(
+            ...,
+            description="Overall quality considering writing excellence, substantive value, and presentation"
+        )
+        
+        # Property 10: Audience Level
+        audience_level: AudienceLevel = Field(
+            ...,
+            description="Intended sophistication level and background knowledge assumptions"
+        )
+        
+        # Property 11: Commercial Bias
+        commercial_bias: CommercialBias = Field(
+            ...,
+            description="How much commercial interests influence objectivity and informational value"
+        )
+        
+        # Property 12: Time Sensitivity
+        time_sensitivity: TimeSensitivity = Field(
+            ...,
+            description="How time-sensitive the content is"
+        )
+        
+        # Property 13: Content Safety
+        content_safety: ContentSafety = Field(
+            ...,
+            description="Presence of inappropriate, harmful, or legally problematic content"
+        )
+        
+        # Property 14: Educational Value
+        educational_value: EducationalValue = Field(
+            ...,
+            description="Potential for teaching, learning, and knowledge transfer"
+        )
+        
+        # Property 15: Reasoning Indicators
+        reasoning_indicators: ReasoningIndicators = Field(
+            ...,
+            description="Presence and quality of logical reasoning, analysis, and explanatory content"
+        )
+        
+        # Property 16: PII Presence
+        pii_presence: PiiPresence = Field(
+            ...,
+            description="Whether the content contains personally identifiable information"
+        )
+        
+        # Property 17: Regional Relevance (multi-select)
+        regional_relevance: List[RegionalRelevance] = Field(
+            ...,
+            description="Primary regional, cultural, or geopolitical sphere(s) - always return an array (one or multiple regions)",
+            min_length=1,
+            max_length=3
+        )
+
+        # Property 18: Country Relevance (multi-select)
+        country_relevance: List[Union[Country, CountryRelevanceSpecial]] = Field(
+            ...,
+            description="Specific country/countries the content mentions or is relevant for (or special values for supranational/non-country-specific) - always return an array (one or more countries/special values)",
+            min_length=1,
+            max_length=10
+        )
+        
+        model_config = ConfigDict(
+            validate_assignment=True,
+            extra="forbid",  # Don't allow extra fields
+            json_schema_extra={
+                "example": {
+                    "content_integrity": "complete",
+                    "content_ratio": "mostly_content",
+                    "content_length": "substantial",
+                    "one_sentence_description": "API reference for payment endpoints and error codes.",
+                    "content_type": ["analytical", "instructional"],
+                    "business_sector": ["academic_research", "technology_software"],
+                    "technical_content": ["scientific", "data_heavy"],
+                    "information_density": "dense",
+                    "content_quality": "excellent",
+                    "audience_level": "expert",
+                    "commercial_bias": "none",
+                    "time_sensitivity": "slowly_changing",
+                    "content_safety": "safe",
+                    "educational_value": "high",
+                    "reasoning_indicators": "analytical",
+                    "pii_presence": "no_pii",
+                    "regional_relevance": ["european"],
+                    "country_relevance": ["germany"]
+                }
+            },
+        )
+    
+    return _AnnotationResponse
+
+
+def flatten_model_json_schema(schema: dict) -> dict:
+    """Inline all #/$defs/... references and remove $defs from a Pydantic JSON Schema.
+
+    - Recursively resolves $ref entries that point into local $defs
+    - Preserves sibling constraints next to $ref by shallow-merging into the resolved target
+    - Drops any nested $defs occurrences
+    """
+    schema_copy = deepcopy(schema)
+    defs = schema_copy.pop("$defs", {})
+
+    def resolve(node):
+        if isinstance(node, dict):
+            if "$ref" in node:
+                ref = node.get("$ref")
+                extra = {k: v for k, v in node.items() if k != "$ref" and k != "$defs"}
+                if isinstance(ref, str) and ref.startswith("#/$defs/"):
+                    name = ref.split("/")[-1]
+                    replacement = deepcopy(defs.get(name, {}))
+                    resolved_replacement = resolve(replacement)
+                    resolved_extra = resolve(extra)
+                    if isinstance(resolved_replacement, dict) and isinstance(resolved_extra, dict):
+                        return {**resolved_replacement, **resolved_extra}
+                    return resolved_replacement
+                resolved_extra = resolve(extra)
+                return {**({"$ref": ref}), **(resolved_extra if isinstance(resolved_extra, dict) else {})}
+            return {k: resolve(v) for k, v in node.items() if k != "$defs"}
+        if isinstance(node, list):
+            return [resolve(item) for item in node]
+        return node
+
+    return resolve(schema_copy)
+
+
+def get_annotation_response_schema(
+    use_country_enum: bool = True,
+    flatten: bool = True,
+    as_string: bool = False,
+    minify: bool = True,
+    one_sentence_description_max_length=ONE_SENTENCE_DESCRIPTION_MAX_LENGTH,
+    compact_whitespace: bool = True
+) -> Union[dict, str]:
+    """
+    Build the JSON Schema for `AnnotationResponse` with an option to avoid large country enums.
+
+    - If `use_country_enum` is True (default), the schema uses enum definitions for
+      `country_relevance` items as generated by Pydantic.
+    - If `use_country_enum` is False, `country_relevance` becomes a list of strings
+      (no enum) while the property's description still contains the full list of
+      valid values. This avoids very large enum blocks for APIs that do not support them.
+    - If `flatten` is True (default), inline all local $defs via `flatten_model_json_schema`.
+    - If `as_string` is True, return the schema as a JSON string. When `as_string`
+      is True and `minify` is True (default), emit compact JSON with no extra
+      whitespace to reduce token usage. If `minify` is False, pretty-print with indentation.
+    - If `compact_whitespace` is True (default), adds x-guidance directive to enforce
+      compact JSON output with no tabs, newlines, or extra whitespace between tokens.
+      This prevents models from generating whitespace-heavy malformed JSON.
+    """
+    schema = create_annotation_response_model(one_sentence_description_max_length).model_json_schema()
+    
+    # Add x-guidance directive for llguidance to enforce compact JSON (no tabs/newlines/whitespace)
+    if compact_whitespace:
+        schema["x-guidance"] = {"whitespace_flexible": False}
+
+    if not use_country_enum:
+        # Construct the list of valid values from the enums but do not emit them as enum types
+        valid_values = [e.value for e in Country] + [e.value for e in CountryRelevanceSpecial]
+
+        country_prop = schema.get("properties", {}).get("country_relevance")
+        if isinstance(country_prop, dict):
+            existing_description = country_prop.get("description", "")
+
+            # Ensure the property is an array of strings without duplicating the long values list
+            country_prop["type"] = "array"
+            country_prop["items"] = {"type": "string"}
+
+            # Retain minItems and other constraints already present on the property
+
+            # Put the full list of valid values only in the property description (not in items)
+            values_text = f" Valid values: {', '.join(valid_values)}"
+            if existing_description and "Valid values:" not in existing_description:
+                country_prop["description"] = existing_description.rstrip() + values_text
+            elif not existing_description:
+                country_prop["description"] = values_text.strip()
+
+    if flatten:
+        schema = flatten_model_json_schema(schema)
+
+    if as_string:
+        if minify:
+            return json.dumps(schema, separators=(",", ":"), ensure_ascii=False)
+        return json.dumps(schema, indent=2, ensure_ascii=False)
+
+    return schema
+
+
+# Default AnnotationResponse model with default max_length
+AnnotationResponse = create_annotation_response_model()
+
+
+TRUNCATION_TAG = "<truncated_content>"
+
+
+def truncate_content(content: str, max_content_chars: int) -> str:
+    if max_content_chars > 0 and len(content) > max_content_chars:
+        return f"{content[:max_content_chars]}\n{TRUNCATION_TAG}"
+    return content
+
+
+with open(Path("property_descriptions.md"), "r") as f:
+    property_descriptions = f.read()
+
+
+def create_messages(document_text: str, max_content_chars: int = 50_000) -> list[dict]:
+    document_text = truncate_content(document_text, max_content_chars)
+    user_prompt = USER_PROMPT.format(content=document_text)
+
+    messages = [
+        {"role": "system", "content": SYSTEM_PROMPT},
+        {"role": "user", "content": user_prompt},
+    ]
+    return messages
+
+
+schema_str = get_annotation_response_schema(as_string=True, one_sentence_description_max_length=150)
+annotator_system_prompt = ANNOTATOR_SYSTEM_PROMPT.format(json_schema=schema_str, property_descriptions=property_descriptions)
+
+
+def create_annotator_messages(document_text: str, max_content_chars: int = 50_000) -> list[dict]:
+    document_text = truncate_content(document_text, max_content_chars)
+    user_prompt = ANNOTATOR_USER_PROMPT.format(content=document_text)
+
+    messages = [
+        {"role": "system", "content": annotator_system_prompt},
+        {"role": "user", "content": user_prompt},
+    ]
+    return messages

property_descriptions.md

@@ -0,0 +1,1182 @@
+## Detailed Property Descriptions & Annotation Guidelines
+
+### Core Content Properties
+
+#### 1. Content Integrity
+
+**What we're measuring**: Completeness and technical quality of the content itself, regardless of navigation ratio.
+
+##### Values & Criteria:
+
+**`complete`** - Full, intact content as intended
+- Content appears complete with proper beginning, middle, and end
+- All essential elements present (introduction, body, conclusion where appropriate)
+- No obvious truncation or missing sections
+- Example: Complete articles, full tutorials, intact documents
+
+**`mostly_complete`** - Minor elements missing but core content intact
+- Core content is complete but some secondary elements may be missing
+- Minor truncation that doesn't affect main message
+- Example: Article with truncated comments, missing sidebar content, partial author bio
+
+**`fragment`** - Incomplete content, missing significant portions
+- Missing introduction, conclusion, or substantial middle sections
+- Truncated mid-sentence or mid-paragraph
+- Content feels incomplete or cut off
+- Example: Search result snippets, article excerpts, broken crawls, partial downloads
+
+**`severely_degraded`** - Broken, unreadable, or corrupted content
+- Encoding errors, scrambled text, missing characters
+- Severely malformed HTML rendering as gibberish
+- Technical corruption making content unreadable
+- Example: �&$^%*@# characters, completely broken formatting, corrupted files
+
+##### Key Decision Points:
+- **Content completeness**: Does the content feel like a complete unit of information?
+- **Technical integrity**: Is the content technically readable and properly formatted?
+- **Fragment vs. complete**: Independent of navigation - is the actual content complete?
+- **Degraded vs. fragment**: Degraded has technical issues; fragment is just incomplete
+
+**Note**: Documents may end with the special tag `<content_truncated>`, indicating upstream length-based truncation due to processing constraints. Do not penalize Content Integrity due to this truncation signal; assess integrity based on the visible content's coherence and technical readability, ignoring the artificial cutoff.
+
+---
+
+#### 2. Content Ratio
+
+**What we're measuring**: How much of the document is actual content vs. navigation, UI elements, and structural markup.
+
+##### Values & Criteria:
+
+**`complete_content`** - 90-100% meaningful content
+- Full articles, papers, tutorials with minimal navigation
+- Clean text with proper paragraphs and structure
+- Example: A Wikipedia article, academic paper, complete blog post
+
+**`mostly_content`** - 70-89% meaningful content  
+- Complete documents with some navigation elements (header, footer, sidebar)
+- Minor UI elements that don't disrupt reading
+- Example: News articles with standard website navigation
+
+**`mixed_content`** - 40-69% meaningful content
+- Significant navigation mixed throughout content
+- Multiple sidebars, ads, or UI elements interrupting text
+- Example: E-commerce product pages with reviews mixed with purchase options
+
+**`mostly_navigation`** - 10-39% meaningful content
+- Predominantly menus, links, headers, footers
+- Content overwhelmed by structural elements
+- Example: Site maps, navigation pages, heavily UI-focused pages
+
+**`minimal_content`** - 0-9% meaningful content
+- Almost entirely navigation, UI elements, or structural markup
+- Very little readable content present
+- Example: Empty pages, pure navigation menus, error pages with minimal text
+
+##### Key Decision Points:
+- Focus on the **ratio of readable text to navigation/UI elements**
+- **Count only substantive content**, ignore boilerplate and structural elements
+- **Mixed vs. mostly_navigation**: Can you read it as coherent content despite distractions?
+
+---
+
+#### 3. Content Length
+
+**What we're measuring**: Amount of substantive content, ignoring navigation and boilerplate.
+
+##### Values & Criteria:
+
+**`substantial`** - 2,000+ words of meaningful content
+- Long-form, comprehensive content that provides in-depth coverage of a topic
+- Typically includes detailed analysis, multiple sections or chapters, extensive research, or thorough exploration of complex subjects
+- Examples: White papers, research reports, e-books, long-form journalism
+
+**`moderate`** - 500–2,000 words of meaningful content
+- Standard-length content that offers meaningful coverage while remaining focused and digestible
+- Balances depth with accessibility; provides enough detail to be informative without overwhelming readers
+- Examples: Typical blog posts, news articles, product reviews, how-to guides
+
+**`brief`** - 100–500 words of meaningful content
+- Short, focused content that delivers key information quickly and efficiently
+- Gets straight to the point while still providing value and context
+- Examples: News briefs, product descriptions, FAQs, short blog posts
+
+**`minimal`** - Under 100 words of meaningful content
+- Very short content that provides only essential information or serves as a quick reference
+- Designed for rapid consumption or specific micro-purposes
+- Examples: Social media posts, announcements, abstracts, snippets, navigation pages
+
+##### Measurement Tips:
+- **Count only readable content of value**: include article body and substantive headings/captions; exclude headers/footers, menus/sidebars, related links, share/consent UI, pagination, ads, and boilerplate.
+- **Focus on substantive information**, not filler words
+- **Complete thoughts matter more than exact word counts**
+- **Contextual adjustment**: Thresholds are guidelines and can be adjusted based on specific use cases and typical content. Academic contexts may shift ranges upward, while social media contexts may shift them downward.
+
+---
+
+### Content Classification
+
+#### 4. One-Sentence Description
+
+**What we're looking for**: A very short, neutral description of what the document contains.
+
+##### Field:
+
+**`one_sentence_description`**
+- Ultra-short neutral description of the document
+- Exactly one sentence
+- Target length: <100 characters
+- Focus on the main topic and, if useful, the document’s function
+- Examples of functions: tutorial, policy, news report, product page, navigation page
+- Neutral, descriptive tone (no hype or marketing language)
+
+##### To Avoid:
+- Boilerplate intros: "This document...", "This article...", "In this guide..."
+- Calls to action: "Learn how to...", "Discover...", "Find out..."
+- User-facing phrasing: "You will learn...", "How do I..."
+- Non-essential details (dates, numbers) unless central to the topic
+
+##### Examples:
+- "Beginner tutorial on React hooks and basic state management."
+- "News report on European Central Bank interest rate decisions."
+- "Internal policy for customer data retention and deletion."
+- "API reference for payment processing endpoints and error codes."
+- "Research paper analyzing housing price trends in major US cities."
+- "FAQ answering common questions about employee parental leave."
+- "Opinion essay arguing for stricter international climate change legislation."
+
+##### Examples for low-quality or problematic documents (still annotate):
+- "Fragment of article discussing proposed changes to European data privacy laws."
+- "Keyword-stuffed promotional page about cheap car insurance quotes."
+- "Website navigation page listing links to product categories and help pages."
+- "Error page explaining that the requested resource could not be found."
+- "Affiliate landing page promoting multiple online casino bonus offers."
+- "Corrupted text with no identifiable topic or meaningful content."
+
+
+#### 5. Content Type
+
+**What we're measuring**: The functional structure and purpose of content.
+
+**Multi-type content**: Content can be assigned multiple type labels if it genuinely serves multiple purposes. Choose ALL applicable types rather than forcing a single primary choice. Always output an array for this property, even if only one type applies.
+
+##### Values & Criteria:
+
+**`analytical`** - In-depth analysis, research, and critical examination
+- Provides detailed analysis or research on a topic
+- Develops arguments, evaluates evidence, or presents findings
+- Example: Research analysis, investigative reports, academic articles, expert commentary
+
+**`instructional`** - Teaching and how-to content
+- Explicitly teaches skills, concepts, or procedures
+- Step-by-step guidance or educational explanations
+- Example: Tutorials, how-to guides, educational content, training materials
+
+**`reference`** - Lookup materials, definitions, specifications
+- Designed for looking up specific information rather than reading through
+- Often organized alphabetically, categorically, or as lists
+- Example: Dictionaries, encyclopedias, API references, product catalogs
+
+**`procedural`** - Step-by-step processes and procedures
+- Sequential instructions or workflows
+- Process documentation with clear steps
+- Example: Recipes, installation guides, standard operating procedures, workflows
+
+**`qa_structured`** - Structured question-answer content
+- Formal Q&A format with clear questions and answers
+- Often expert responses to specific questions
+- Example: Stack Overflow, FAQ sections, structured Q&A sites
+
+**`conversational`** - Multi-party or turn-based dialogues (humans, bots, or both)
+- Casual or structured conversations between two or more participants
+- May include human–AI chats, forum threads, or comment chains
+- Example: Reddit threads, forum discussions, support chats, assistant chat logs
+
+**`creative`** - Entertainment, artistic, fictional content
+- Primary purpose is entertainment or artistic expression
+- Not primarily informational or instructional
+- Example: Short stories, poems, movie reviews, game content, fiction
+
+**`transactional`** - Commercial, shopping, service-oriented
+- Primary purpose is to facilitate a transaction or service
+- Focuses on products, services, or business processes
+- Example: Product listings, service descriptions, checkout pages
+
+**`boilerplate`** - Legal, policy, standard template text
+- Standard legal or policy language
+- Often repeated across multiple sites with minimal variation
+- Example: Terms of service, privacy policies, disclaimers, cookie banners, standard notices
+
+**`news_report`** - Straight reporting of events with minimal analysis
+- Describes events or facts in a neutral, descriptive tone
+- Time-bound news, updates, or reports
+- Example: Wire-service news articles, breaking-news updates
+
+**`opinion_editorial`** - Persuasive/opinionated commentary or editorials
+- Expresses a stance or argument; aims to persuade
+- May cite evidence but prioritizes viewpoint
+- Example: Op-eds, opinion columns, personal essays with clear stance
+
+**`review_critique`** - Evaluative reviews of products, media, or services
+- Provides judgments, ratings, or critiques
+- May include pros/cons, scoring systems
+- Example: Product reviews, film/book critiques, app store reviews (long-form)
+
+**`technical_documentation`** - Manuals, API docs, developer guides, READMEs
+- Primary goal is to instruct usage of software/hardware/APIs
+- Includes reference sections, examples, parameters, version notes
+- Example: API reference, library README, user manual
+
+**`specification_standard`** - Normative standards and formal specifications
+- Defines requirements, must/shall language, compliance criteria
+- Maintained by standards bodies or authoritative groups
+- Example: RFCs, ISO standards, formal protocol specs
+
+**`legal_document`** - Statutes, case law, contracts, regulatory texts
+- Binding or authoritative legal content
+- Formal legal language and structure
+- Example: Court opinions, legislation, contracts, regulatory rules
+
+**`press_release`** - Organization-issued announcements and PR materials
+- Promotional announcements framed as information
+- Quotes from executives, product/service announcements
+- Example: Company press releases, launch announcements
+
+
+**`structured_data`** - Tables, datasets, indices, catalogs with minimal prose
+- Predominantly tabular/listed data meant for lookup
+- Minimal narrative or explanatory text
+- Example: Product catalogs, schedules, statistical tables
+
+**`source_code`** - Code listings as primary content
+- Dominant content is program source code or scripts
+- May include lightweight comments or snippets without narrative
+- Example: Code files, gist-like pages, competitive programming solutions
+
+
+##### Multi-Type Examples:
+- **Tutorial that analyzes different approaches** → `["instructional", "analytical"]`
+- **Educational reference manual** → `["instructional", "reference"]`
+- **Research paper with step-by-step methodology** → `["analytical", "procedural"]`
+- **Q&A site with analytical responses** → `["qa_structured", "analytical"]`
+- **API guide with examples** → `["technical_documentation", "reference", "instructional"]`
+- **RFC with rationale** → `["specification_standard", "analytical"]`
+- **Film review with interview snippets** → `["review_critique", "conversational"]`
+- **Helpdesk chat with an AI** → `["conversational", "transactional"]`
+- **Breaking news explainer** → `["news_report", "explanatory"]`
+
+---
+
+#### 6. Business Sector
+
+**What we're measuring**: Business sector(s) or industry domain(s) for training sector-specific LLMs.
+
+**Multi-sector content**: Content can be assigned multiple sector labels if it genuinely spans multiple industries. Choose ALL applicable sectors rather than forcing a single primary choice or using "other". Always output an array for this property, even if only one sector applies.
+
+#### Values & Criteria:
+
+**`academic_research`** - Scholarly and research content
+- Peer-reviewed publications, academic papers
+- University-affiliated research and scholarship
+- Formal academic discourse and methodology
+- Example: Journal articles, conference papers, academic books, dissertations
+
+**`education_sector`** - Educational institutions and pedagogy
+- K-12 education, higher education administration
+- Educational technology, curriculum development
+- Teaching methodologies and educational policy
+- Example: School curricula, educational policy papers, teaching resources, edtech content
+
+**`technology_software`** - Software and information technology
+- Software development, programming, IT services
+- Digital products, platforms, and technology companies
+- Computer science and software engineering
+- Example: Software documentation, tech company content, programming guides, IT industry analysis
+
+**`hardware_electronics`** - Hardware devices and electronics industry
+- Semiconductors, consumer electronics, embedded systems, hardware design
+- Electronics manufacturing and supply chains
+- Example: Chip design docs, hardware datasheets, device manuals
+
+**`healthcare_medical`** - Healthcare and medical sector
+- Medical research, clinical practice, healthcare delivery
+- Hospitals, medical devices, healthcare policy
+- Public health and wellness
+- Example: Medical journals, clinical guidelines, healthcare administration, wellness content
+
+**`pharmaceutical_biotech`** - Pharmaceutical and biotechnology
+- Drug development, clinical trials, biotech research
+- Pharmaceutical industry, biotechnology companies
+- Life sciences and molecular biology applications
+- Example: Drug research papers, clinical trial reports, biotech industry analysis
+
+**`financial_services`** - Banking and financial services
+- Banking, investment, insurance, financial planning
+- Financial markets, fintech, payment systems
+- Asset management and financial advisory
+- Example: Financial analysis, banking documentation, investment guides
+
+**`legal_services`** - Legal sector and jurisprudence
+- Law firms, legal practice, court systems
+- Legal education, regulatory compliance
+- Litigation, contracts, legal advisory
+- Example: Legal briefs, court opinions, legal analysis, compliance guides
+
+**`government_public`** - Government and public administration
+- Government agencies, public policy, civic services
+- Regulatory bodies, public administration
+- Political institutions and governance
+- Example: Government reports, policy documents, regulatory filings, civic information
+
+**`manufacturing_industrial`** - Manufacturing and heavy industry
+- Industrial production, manufacturing processes
+- Supply chain, logistics, industrial equipment
+- Factory operations and industrial engineering
+- Example: Manufacturing specs, industrial reports, supply chain analysis, production guides
+
+**`mining_resources`** - Mining and natural resources
+- Exploration, extraction, and processing of minerals and resources
+- Resource markets and operations (metals, rare earths)
+- Example: Mining reports, resource exploration docs, commodity operations
+
+**`chemicals_materials`** - Chemicals and advanced materials
+- Petrochemicals, specialty chemicals, polymers, composites, advanced materials
+- Safety data sheets (SDS), process chemistry, materials science
+- Example: Material datasheets, REACH documentation, chemical process guides
+
+**`energy_utilities`** - Energy and utilities sector
+- Power generation, renewable energy, oil and gas
+- Electric utilities, water services, waste management
+- Energy infrastructure and grid management
+- Example: Energy industry reports, utility regulations, renewable energy research
+
+**`retail_commerce`** - Retail and e-commerce
+- Retail operations, e-commerce platforms
+- Consumer goods distribution, merchandising
+- Retail technology and customer experience
+- Example: Retail industry analysis, e-commerce guides, merchandising strategies
+
+**`wholesale_distribution`** - Wholesale trade and distribution
+- B2B wholesale, distributors, procurement, inventory and fulfillment
+- Supply relationships between manufacturers and retailers
+- Example: Distributor catalogs, wholesale operations, procurement guides
+
+**`real_estate_construction`** - Real estate and construction
+- Property development, construction industry
+- Real estate markets, property management
+- Architecture and building services
+- Example: Real estate analysis, construction specifications, property guides
+
+**`transportation_logistics`** - Transportation and logistics
+- Airlines, shipping, freight, public transit
+- Logistics operations, supply chain transportation
+- Vehicle fleet management, transportation infrastructure
+- Example: Logistics guides, transportation planning, shipping documentation
+
+**`travel_aviation`** - Travel industry and commercial aviation
+- Airlines, airports, OTA platforms, hospitality travel operations
+- Route planning, airline commercial, loyalty, IATA regulations
+- Example: Airline scheduling, fare rules, OTA partner docs
+
+**`automotive_industry`** - Automotive manufacturing and services
+- Vehicle manufacturers, automotive suppliers
+- Automotive technology, electric vehicles
+- Dealerships and automotive services
+- Example: Automotive engineering docs, vehicle technology papers, industry analysis
+
+**`telecommunications`** - Telecommunications industry
+- Telecom operators, network infrastructure
+- Mobile services, broadband, satellite communications
+- Telecommunications equipment and technology
+- Example: Telecom industry reports, network specifications, 5G technology papers
+
+**`media_entertainment`** - Media and entertainment industry
+- Film, television, music, gaming industries
+- Publishing, news media, content creation
+- Streaming services and digital media
+- Example: Entertainment industry analysis, media studies, content strategy
+
+**`gaming_industry`** - Video games and interactive entertainment
+- Game development, studios, engines, esports, live ops
+- Monetization models, community management, platform ecosystems
+- Example: Patch notes, game design docs, esports operations
+
+**`gambling_betting`** - Gambling, betting, and online casinos
+- Sportsbooks, casino games, lotteries, poker rooms
+- Affiliate landing pages, bonus/promotions, tipster content
+- Often high commercial bias and promotional framing
+
+**`advertising_marketing`** - Advertising, marketing, and PR
+- Brand strategy, campaign planning, performance marketing, martech
+- Agencies, in-house marketing, PR communications
+- Example: Campaign briefs, media plans, PR strategies
+
+**`hospitality_tourism`** - Hospitality and tourism sector
+- Hotels, restaurants, travel services
+- Tourism industry, destination management
+- Event planning and hospitality services
+- Example: Tourism studies, hospitality management, travel industry reports
+
+**`food_beverage_hospitality`** - Food & beverage and restaurant operations
+- Restaurant ops, menu engineering, supply chain, QSR/fast casual
+- Food safety, compliance, procurement for F&B
+- Example: Restaurant training manuals, HACCP docs, vendor specs
+
+**`agriculture_food`** - Agriculture and food production
+- Farming, agricultural technology, food processing
+- Agricultural supply chain, food safety
+- Agribusiness and agricultural policy
+- Example: Agricultural research, food industry reports, farming guides
+
+**`environmental_services`** - Environmental and sustainability services
+- Environmental consulting, ESG reporting, sustainability programs
+- Waste management services, remediation, impact assessments
+- Example: ESG reports, environmental impact assessments, sustainability frameworks
+
+**`aerospace_defense`** - Aerospace and defense industry
+- Aircraft manufacturing, space technology
+- Defense contractors, military systems
+- Aviation and space exploration
+- Example: Aerospace engineering papers, defense industry analysis, aviation guides
+
+**`insurance_industry`** - Insurance sector
+- Life, health, property, and casualty insurance
+- Reinsurance, actuarial science, risk assessment
+- Insurance technology and underwriting
+- Example: Actuarial studies, insurance policy analysis, risk management guides
+
+**`nonprofit_ngo`** - Nonprofit and NGO sector
+- Charitable organizations, international development
+- Social services, humanitarian organizations
+- Foundations and philanthropic institutions
+- Example: NGO reports, nonprofit management, development studies
+
+**`consulting_professional`** - Professional services and consulting
+- Management consulting, accounting firms
+- Business advisory, professional services firms
+- Corporate strategy and business transformation
+- Example: Consulting reports, professional services guides, business strategy papers
+
+**`human_resources`** - HR and people operations
+- Talent acquisition, compensation & benefits, performance management, L&D
+- HR tech, workforce planning, organizational development
+- Example: HR policy docs, job frameworks, talent strategy
+
+**`security_cyber`** - Security and cybersecurity
+- Information security, threat intelligence, risk management, compliance (e.g., SOC2)
+- Physical security operations and incident response
+- Example: Security guidelines, incident playbooks, vulnerability reports
+
+**`consumer_goods`** - Consumer products and CPG
+- Fast-moving consumer goods, household products
+- Personal care, food and beverage brands
+- Consumer product development and marketing
+- Example: CPG industry analysis, product development docs, consumer research
+
+**`general_interest`** - General audience content
+- Content for broad audiences without sector focus
+- General knowledge and miscellaneous topics
+- Cross-sector or sector-agnostic content
+- Example: General magazines, broad interest content, lifestyle articles
+
+**`other`** - Highly specialized or unclassifiable
+- Highly specialized niches not covered by existing sectors
+- Content with genuinely unclear sector classification
+- Unique content types that don't map to any defined sector
+- Example: Highly specialized technical niches, unique content formats
+
+##### Multi-Sector Examples:
+- **Medical device regulations** → `healthcare_medical` + `pharmaceutical_biotech` + `government_public`
+- **Fintech software documentation** → `financial_services` + `technology_software`
+- **Agricultural biotechnology research** → `agriculture_food` + `pharmaceutical_biotech`
+
+---
+
+#### 7. Technical Content
+
+**What we're measuring**: Type and intensity of specialized technical knowledge.
+
+**Multi-technical content**: Content can be assigned multiple technical content labels if it genuinely combines multiple technical domains. Choose ALL applicable technical types rather than forcing a single primary choice. Always output an array for this property, even if only one technical type applies.
+
+##### Values & Criteria:
+
+**`code_heavy`** - Significant programming content
+- Multiple code examples, algorithms, or implementations
+- Technical programming concepts and methodologies
+- Software development focus
+- Example: Programming tutorials, API documentation, software guides
+
+**`math_heavy`** - Substantial mathematical content
+- Mathematical equations, proofs, or statistical analysis
+- Quantitative analysis and mathematical reasoning
+- Mathematical concepts and methodologies  
+- Example: Mathematical papers, statistical analysis, quantitative research
+
+**`scientific`** - Research and scientific methodology content
+- Scientific research findings, experimental data
+- Scientific methodology and analysis
+- Peer-reviewed research content
+- Example: Research papers, scientific studies, experimental reports
+
+**`data_heavy`** - Substantial datasets, tables, and data analysis
+- Contains significant data tables, charts, or datasets
+- Focus on data interpretation and analysis
+- Statistical content with data presentations
+- Example: Research data, statistical reports, data analysis, survey results
+
+**`engineering`** - Engineering and applied technical content
+- Engineering design, systems, and applied technical solutions
+- Technical specifications for physical systems
+- Non-software engineering disciplines
+- Example: Mechanical engineering, civil engineering, technical specifications, design documents
+
+**`basic_technical`** - Some technical elements but not dominant
+- Light technical content mixed with general explanations
+- Technical concepts explained for general audience
+- Example: Technology articles for general audience, basic technical explanations
+
+**`non_technical`** - No significant technical content
+- General audience content without specialized technical knowledge
+- No programming, mathematical, engineering, or scientific focus
+- Example: General articles, humanities content, basic informational content
+
+##### Multi-Technical Examples:
+- **Data science tutorial with code examples** → `["code_heavy", "math_heavy", "data_heavy"]`
+- **Engineering research with statistical analysis** → `["engineering", "scientific", "data_heavy"]`
+- **Computational biology paper** → `["code_heavy", "scientific"]`
+
+---
+
+### Quality and Value Assessment
+
+#### 8. Content Quality
+
+**What we're measuring**: Overall quality of content considering writing excellence, substantive value, and presentation quality regardless of authorship origin.
+
+#### Values & Criteria:
+
+**`excellent`** - Outstanding quality across all dimensions
+- Sophisticated writing with varied sentence structures and engaging style
+- Rich, appropriate vocabulary with error-free grammar and punctuation
+- High substantive value with clear insights or information
+- Professional presentation and formatting
+- Natural flow and logical organization
+- Example: High-quality publications, expert analyses, polished educational content, well-crafted professional documents
+
+**`good`** - High quality with minor imperfections
+- Grammatically correct with proper sentence structure
+- Appropriate vocabulary and tone for content type
+- Solid substantive value and clear information
+- Good organization and readable flow
+- Only occasional minor issues (1-2 typos per section)
+- Example: Quality journalism, professional websites, well-written blog posts, solid educational materials
+
+**`adequate`** - Acceptable quality for most purposes
+- Generally clear and understandable writing
+- Some grammatical errors but meaning remains clear
+- Reasonable substantive value though may lack depth
+- Basic organization and structure present
+- Minor formatting or presentation issues
+- Example: Casual blogs, user reviews, basic informational content, simple guides
+
+**`poor`** - Significant quality issues impacting utility
+- Multiple errors affecting comprehension or credibility
+- Unclear expression, confusing organization, or awkward phrasing
+- Limited substantive value or questionable information
+- Major formatting problems or unprofessional presentation
+- Difficult to extract reliable information
+- Example: Low-quality web content, poorly edited materials, confusing instructions
+
+**`unacceptable`** - Quality too low for productive use
+- Severely impaired communication with major errors
+- Incoherent, nonsensical, or corrupted content
+- No reliable substantive value
+- Broken formatting or technical corruption
+- Cannot determine intended meaning or extract useful information
+- Example: Corrupted text, severe translation errors, spam content, SEO content, completely broken formatting
+
+##### Quality Assessment Guidelines:
+- **Comprehension**: Can the intended message be clearly understood?
+- **Substantive value**: Does the content provide useful information or insights?
+- **Technical presentation**: Is the content properly formatted and readable?
+- **Error impact**: Do errors significantly impede understanding or credibility?
+- **Professional standards**: Does the content meet basic standards for its intended purpose?
+
+**Language-Specific Quality Indicators:**
+- For non-Latin scripts (Arabic, Chinese, Japanese): Check for proper character encoding
+- For agglutinative languages (Turkish, Finnish): Adjust expectations for word count/density
+- For languages with different formality levels (Japanese, Korean): Assess appropriate register
+- Mixed-language documents: Evaluate code-switching quality and appropriateness
+---
+
+#### 9. Information Density
+
+**What we're measuring**: Ratio of valuable information to redundancy, padding, and repetition.
+
+##### Values & Criteria:
+
+**`dense`** - Efficient, information-packed content
+- Every sentence adds new information or insight
+- Minimal redundancy or unnecessary elaboration
+- Little to no repetition of the same concepts
+- Example: Technical specifications, concise academic writing, quality reference material
+
+**`adequate`** - Good information content with reasonable elaboration
+- Most content adds value with some acceptable elaboration
+- Minimal repetition within the document
+- Good balance of information and explanation
+- Example: Well-written articles, good tutorials with examples
+
+**`moderate`** - Mixed substantive content with noticeable padding
+- Some valuable information mixed with unnecessary elaboration
+- Noticeable repetition of key points for emphasis
+- Some sections feel padded or verbose
+- Example: Blog posts with some fluff, articles with repetitive conclusions
+
+**`thin`** - Low information content with significant problems
+- Much content doesn't add new information
+- High internal repetition and excessive redundancy  
+- Significant padding to reach desired length
+- Example: SEO-optimized content, poorly edited writing
+
+**`empty`** - Dominated by repetition and meaningless content
+- Minimal actual information value
+- Dominated by repetition and copy-paste artifacts
+- Same ideas repeated multiple times without development
+- Example: Spam content, template-filled pages, keyword-stuffed articles
+
+##### Common Repetition Patterns to Watch For:
+- **Same phrases repeated throughout** (especially in SEO content)
+- **Identical paragraphs** or sections (copy-paste errors)
+- **Circular reasoning** (saying the same thing in different ways)
+- **Template artifacts** (repeated boilerplate mixed with content)
+
+---
+
+#### 10. Educational Value
+
+**What we're measuring**: Potential for teaching, learning, and knowledge transfer.
+
+##### Values & Criteria:
+
+**`high`** - Clear instructional design and learning objectives
+- Explicitly teaches concepts or skills
+- Progressive skill building from basic to advanced
+- Clear learning objectives and outcomes
+- Comprehensive explanations with examples
+- Example: Quality tutorials, textbooks, structured courses, educational guides
+
+**`moderate`** - Good instructional value with some learning potential
+- Some instructional elements present
+- Explanations help build understanding
+- Transferable knowledge to other contexts
+- Good examples or illustrations
+- Example: How-to articles, explanatory content, informative guides
+
+**`basic`** - Limited educational content
+- Some explanations but not systematically instructional
+- Basic explanations of concepts
+- Limited learning potential or skill building
+- Example: Basic explanations, simple informational content
+
+**`minimal`** - Little educational value
+- Primarily informational rather than instructional
+- No clear learning objectives or skill building
+- Entertainment or commercial focus
+- Example: Entertainment content, basic news, commercial content
+
+**`none`** - No educational content
+- No instructional value or learning potential
+- Purely transactional, entertainment, or administrative
+- No knowledge transfer potential
+- Example: Pure entertainment, transactions, legal boilerplate
+
+##### Disambiguation tips
+- Explanatory vs Educational: explanations alone ≠ educational design; require intent to teach plus scaffolding for Basic+
+- Reference docs: typically Minimal; promote to Basic/Moderate when guided “how-to” segments or curated examples exist
+- Reviews/op-eds: None/Minimal unless they include actionable how-to guidance designed for learning
+
+##### Automation heuristics
+- Keywords: Objectives/Outcomes, Lesson, Exercise/Quiz, Homework, Assessment, Syllabus, Module, Unit, Learning Goals
+- Structure: numbered steps + prerequisites/requirements → Basic; add practice tasks/solutions → Moderate; syllabus/modules/assessments → High
+- Signals of non-edu mix: heavy CTAs/ads or product pitches → cap at Minimal unless clear instructional scaffolding
+
+##### Quick decision tree
+- Are there explicit learning goals or a syllabus? → High
+- Else, are there step-by-step instructions with examples/exercises? → Moderate
+- Else, are there explanatory sections intended to teach basics? → Basic
+- Else, is there any minor instructional element? → Minimal
+- Otherwise → None
+
+##### Borderline examples
+- API reference with examples but no guidance → Minimal to Basic (depending on clarity/examples)
+- Blog post explaining concept with analogies and one example → Basic
+- Tutorial with tasks, checkpoints, and solutions → High
+- Product documentation with “Getting Started” and “How-To” flows → Moderate
+
+##### Educational Indicators:
+- **Learning objectives**: Clear goals for what reader should learn
+- **Skill progression**: Builds from basic to advanced concepts
+- **Examples and practice**: Provides concrete examples or exercises
+- **Knowledge transfer**: Concepts applicable beyond immediate context
+
+---
+
+#### 11. Reasoning Indicators
+
+**What we're measuring**: Presence and quality of logical reasoning, analysis, and explanatory content.
+
+##### Values & Criteria:
+
+**`analytical`** - Complex reasoning and systematic analysis
+- Multi-step arguments with logical progression
+- Cause-effect analysis and systematic thinking
+- Considers multiple perspectives or variables
+- Draws conclusions from evidence and reasoning
+- Example: Research analysis, complex problem-solving, systematic evaluations
+
+**`explanatory`** - Clear explanations with logical flow
+- Explains how or why things work
+- Shows cause-effect relationships clearly
+- Educational reasoning that builds understanding
+- Logical connections between concepts
+- Example: Good tutorials, educational content, how-to explanations
+
+**`basic_reasoning`** - Simple logical connections
+- Some logical connections between ideas
+- Basic explanations of concepts or processes
+- Elementary analytical thinking
+- Simple cause-effect relationships
+- Example: Basic explanations, simple arguments, elementary analysis
+
+**`minimal`** - Limited reasoning, mostly descriptive
+- Primarily describes what rather than why or how
+- Few logical connections between ideas
+- Mostly factual statements without analysis
+- Little explanatory content
+- Example: Basic descriptions, simple factual content, minimal analysis
+
+**`none`** - No clear reasoning present
+- Purely descriptive content
+- Simple factual listing without connections
+- Narrative content without analysis
+- No logical argumentation or explanation
+- Example: Simple lists, basic narratives, pure description
+
+##### Thinking-trace signals (what to look for)
+- Stepwise structure: numbered steps in proofs/derivations/solutions; “First… therefore… hence… so…”
+- Hypothesis and test: assumptions, intermediate results, counterexamples, sanity checks
+- Tool- or method-calls: named algorithms, theorems, lemmas, or procedures invoked and justified
+- Error analysis or reflection: “we tried X, failed because Y, so we…”, “limitations,” “edge cases”
+- Intermediate artifacts: scratch calculations, partial code reasoning, sub-problems and sub-claims
+
+##### Disambiguation rules
+- Explanatory vs Analytical: explanations tell how; analytical shows multi-step inference with evidence and intermediate claims
+- Worked example vs Mere answer: worked examples expose steps and justification; mere answers without steps are not reasoning-rich
+- Procedural vs Reasoning: procedural lists actions; reasoning links actions via logic, evidence, or constraints
+
+##### Automation heuristics
+- Lexical cues: because, therefore, thus, hence, suppose/assume, we conclude, by induction, lemma/theorem/proof, O(n), hypothesis, counterexample
+- Structure cues: presence of proof blocks, derivations (e.g., “Proof.”, “QED”, TeX environments), multi-step numeric calculations
+- Program reasoning: code comments like “// invariant”, “// complexity”, pre/post-conditions, test reasoning
+- Thresholding: count reasoning cues per 1k tokens; with ≥2 structural cues or ≥5 lexical cues → at least explanatory; proofs/derivations → analytical
+
+##### Quick decision tree
+- Is there a proof/derivation or multi-step argument with intermediate claims? → analytical
+- Else, does it explain why/how with cause-effect and logical links? → explanatory
+- Else, are there simple logical connections or one-step justifications? → basic_reasoning
+- Else, does it mostly describe without connecting ideas? → minimal/none
+
+##### Borderline examples
+- Answer-only solutions (final numeric result without steps) → minimal
+- Step-by-step math solution with intermediate equations → analytical
+- “How it works” article connecting 2–3 causal steps without data → explanatory
+- Troubleshooting log with attempts and justifications → analytical if causal chain is explicit; otherwise explanatory
+
+##### Key Reasoning Patterns to Identify:
+- **Cause-effect**: "Because X, therefore Y"
+- **Problem-solution**: Identifies problems and proposes solutions
+- **Comparison**: Analyzes similarities and differences
+- **Logical progression**: Ideas build on previous ideas
+- **Evidence-based conclusions**: Draws conclusions from presented evidence
+
+---
+
+### Audience and Purpose
+
+#### 12. Audience Level
+
+**What we're measuring**: Intended sophistication level and background knowledge assumptions of the target audience.
+
+##### Values & Criteria:
+
+**`expert`** - Highly specialized professional/academic content
+- Assumes deep domain expertise and advanced training
+- Uses technical terminology without explanation
+- Content for practitioners actively working in specialized fields
+- Example: Climate modeling methodology in Nature Climate Change, research papers, technical specifications, expert-to-expert communications
+
+**`advanced`** - Educated adult audience with analytical skills
+- Assumes higher education and critical thinking ability
+- Explains specialized concepts but uses sophisticated language
+- Intellectually challenging but accessible to educated generalists
+- Example: Complex climate change analysis in The Atlantic, quality journalism, policy analysis, advanced general interest content
+
+**`general`** - General adult audience
+- Accessible to most educated adults without specialized background
+- Explains technical concepts when introduced
+- Uses clear language while maintaining intellectual substance
+- Example: Quality journalism, general interest articles, accessible explanations of complex topics
+
+**`beginner`** - Introductory level with minimal prerequisites
+- Explains basic concepts and terminology
+- Builds up from fundamental principles
+- Assumes minimal prior knowledge of the subject area
+- Example: Introductory tutorials, beginner guides, basic explanations, getting-started content
+
+**`youth`** - Targeted at teenagers and young adults (ages 13-19)
+- Age-appropriate complexity with contemporary cultural references
+- Sophisticated enough for developing critical thinking but accessible
+- May address topics relevant to adolescent experiences and concerns
+- Example: High school educational content, young adult literature, teen-focused explanations, college prep materials
+
+**`children`** - Designed specifically for children
+- Simple language and concepts appropriate for young readers
+- Educational content designed for elementary/middle school levels
+- Age-appropriate topics and complexity
+- Example: Children's educational content, elementary school materials, simple explanations for young learners
+
+##### Assessment Guidelines:
+- **Professional context**: Is this content designed for workplace use vs. general learning?
+- **Terminology density**: How much specialized vocabulary is used without explanation?
+- **Concept complexity**: How sophisticated are the ideas and their development?
+- **Background assumptions**: What education level and domain knowledge does the author assume?
+
+**Cross-Linguistic Considerations:**
+- Expert terminology density varies by language (German allows more compound terms)
+- Formality markers differ across cultures
+- Educational level assumptions vary by country's education system
+- Age-appropriate content differs across cultures
+
+---
+
+#### 13. Commercial Bias
+
+**What we're measuring**: How much commercial interests influence the objectivity and informational value of content.
+
+##### Values & Criteria:
+
+**`none`** - No commercial influence detected  
+- Objective, informational presentation
+- No promotional language or commercial agenda
+- Focus purely on informing or educating
+- Example: Academic papers, objective journalism, educational content
+
+**`minimal`** - Slight commercial context but maintains objectivity
+- May mention products/services but in informational context
+- Maintains balanced, objective tone
+- Commercial mentions serve informational purpose
+- Example: Product reviews with balanced analysis, informational articles mentioning relevant products
+
+**`moderate`** - Some commercial influence on content
+- Mix of informational and promotional content
+- Some promotional language but still provides useful information
+- Commercial interests somewhat visible but not dominant
+- Example: Company blogs with useful information, sponsored content with actual value
+
+**`heavy`** - Strong commercial bias throughout
+- Primarily promotional with some informational elements
+- Heavy use of marketing language and persuasive techniques
+- Clear commercial agenda affects content objectivity
+- Example: Marketing articles disguised as information, heavily biased product comparisons
+
+**`pure_marketing`** - Entirely commercial/promotional content
+- No genuine informational value beyond promotion
+- Pure marketing copy or advertising material
+- Designed solely to drive sales or conversions
+- Example: Sales pages, pure advertising copy, promotional brochures
+
+##### Key Indicators:
+- **Language tone**: Objective vs. promotional language
+- **Primary purpose**: Inform vs. persuade/sell
+- **Balance**: Are alternatives/drawbacks mentioned?
+- **Call-to-action**: Subtle information vs. obvious sales pitch
+
+---
+
+#### 14. Time-Sensitivity
+
+**What we're measuring**: How time-sensitive the content is - whether its value degrades over time or remains stable.
+
+##### Values & Criteria:
+
+**`evergreen`** - Content remains valuable indefinitely
+- Fundamental concepts, principles, theories
+- Historical information and established facts
+- Skills and techniques that don't change
+- Reference materials with lasting value
+- Example: Mathematical proofs, language grammar guides, classical literature analysis, basic cooking techniques
+
+**`slowly_changing`** - Content remains valuable for years
+- Best practices that evolve slowly
+- Technical content that updates every few years
+- Cultural and social topics with gradual change
+- Example: Programming language tutorials, academic textbooks, industry standards, educational curricula
+
+**`regularly_updating`** - Content valuable for months to a year
+- Industry trends and market analysis
+- Technology reviews and comparisons
+- Policy discussions and current research
+- Example: Software framework guides, business strategies, product reviews, research summaries
+
+**`time_sensitive`** - Content value degrades quickly
+- News and current events
+- Time-bound information (prices, schedules, availability)
+- Temporary situations or short-term trends
+- Real-time data and statistics
+- Example: Stock prices, weather reports, breaking news, event announcements, sales/promotions
+
+##### Key Decision Points:
+- **Core question**: If someone reads this in 2 years, will it still be valuable?
+- **Update frequency**: How often does this type of information typically change?
+- **Temporal references**: Does the content heavily reference "now," "recently," "currently"?
+- **Subject matter stability**: Is this about unchanging principles or evolving situations?
+
+---
+
+### Safety and Compliance
+
+#### 15. Content Safety
+
+**What we're measuring**: Presence of inappropriate, harmful, or legally problematic content.
+
+##### Values & Criteria:
+
+**`safe`** - Appropriate for all contexts
+- No concerning content of any type
+- Professional, appropriate language throughout
+- Suitable for general audiences including workplace settings
+
+**`mild_concerns`** - Minor issues that don't constitute major problems
+- Occasional mild profanity in context
+- Brief mentions of sensitive topics handled appropriately  
+- Minor concerns that don't affect overall suitability
+- Example: Historical discussions of sensitive topics, professional content with mild language
+
+**`nsfw`** - Not safe for work or general audiences
+- Explicit sexual content or graphic descriptions
+- Adult themes requiring content warnings
+- Graphic violence or disturbing imagery descriptions
+- Example: Adult content, graphic medical descriptions, explicit violence
+
+**`harmful`** - Potentially harmful content requiring careful handling
+- Content promoting dangerous activities or self-harm
+- Hate speech targeting individuals or groups
+- Violent content glorifying harm to others
+- Example: Self-harm content, hate speech, dangerous "how-to" guides
+
+**`illegal`** - Illegal content requiring immediate rejection
+- Content promoting clearly illegal activities
+- Material that violates laws in major jurisdictions
+- Example: Terrorist content, child exploitation
+
+##### Safety Assessment Guidelines:
+- **Context matters**: Medical/educational discussions of sensitive topics may be appropriate
+- **Intent matters**: Discussing harmful topics for educational purposes vs. promoting them
+- **Audience consideration**: Content appropriate for experts may not be safe for general audiences
+
+---
+
+#### 16. PII Presence
+
+**What we're measuring**: Whether the content contains personally identifiable information that could identify private individuals.
+
+##### Values & Criteria:
+
+**`no_pii`** - No personal information detected
+- No names of private individuals
+- No contact information (emails, phones, addresses)
+- No identification numbers
+- Public figures and officials mentioned by name are acceptable
+- Example: News articles about politicians, technical documentation, general information
+
+**`contains_pii`** - Contains potentially identifiable information
+- Names of private individuals (non-public figures)
+- Email addresses, phone numbers, physical addresses
+- ID numbers (SSN, passport, driver's license, employee IDs)
+- Medical information about identifiable individuals
+- Financial account information
+- Example: Personal blogs with full names, leaked databases, medical case studies with identifying info
+
+##### Key Decision Points:
+- **Public vs. Private figures**: Politicians, celebrities, CEOs = public (no PII flag); private citizens = PII
+- **Context matters**: Academic paper authors and their institutional emails = typically no PII; personal emails in forums = PII
+- **Aggregated vs. Individual**: Statistical data = no PII; individual records = PII
+
+---
+
+### Geographic Relevance
+
+#### 17. Regional Relevance
+
+**What we're measuring**: Primary regional, cultural, or geopolitical sphere(s) that the content relates to, regardless of language used.
+
+**Multi-regional content**: Content can be assigned multiple regional labels if it genuinely spans multiple regions. Choose ALL applicable regions rather than forcing a single primary choice. Always output an array for this property, even if only one region applies.
+
+##### Values & Criteria:
+
+**`european`** - European context (EU and broader Europe)
+- Content about European countries, EU policies, or pan-European topics
+- European cultural perspectives, social systems, or business practices
+- References to European cities, institutions, companies, or regulations
+- Includes: EU member states, UK, Switzerland, Norway, Balkans, etc.
+- Example: GDPR compliance, European Parliament elections, Schengen area travel, European football leagues
+
+**`north_american`** - North American context
+- Content about US, Canada, or Mexico
+- North American cultural perspectives, USMCA/NAFTA region topics
+- References to North American institutions, companies, or issues
+- Example: FDA regulations, Silicon Valley tech, NHL, US constitutional law, Canadian healthcare
+
+**`east_asian`** - East Asian context
+- Content about China, Japan, Korea (North/South), Taiwan, Mongolia
+- East Asian cultural perspectives, Confucian-influenced societies
+- References to East Asian economic models, companies, or social systems
+- Example: Gaokao exams, K-pop, Shenzhen tech hub, Japanese work culture, Taiwan semiconductor industry
+
+**`south_asian`** - South Asian context  
+- Content about India, Pakistan, Bangladesh, Sri Lanka, Nepal, Bhutan, Afghanistan, Maldives
+- South Asian cultural perspectives, subcontinental issues
+- References to South Asian institutions, economies, or social structures
+- Example: IIT entrance exams, Bollywood, cricket leagues, monsoon impacts, caste system discussions
+
+**`southeast_asian`** - Southeast Asian context
+- Content about ASEAN countries (Indonesia, Thailand, Vietnam, Philippines, Malaysia, Singapore, etc.)
+- Southeast Asian regional perspectives and economic integration
+- References to ASEAN policies, regional companies, or cultural phenomena
+- Example: ASEAN economic community, Indonesian elections, Singapore financial sector, Thai tourism
+
+**`middle_eastern`** - Middle Eastern and North African context
+- Content about Arab states, Iran, Turkey, Israel, North Africa (MENA region)
+- Middle Eastern cultural perspectives, Islamic finance, regional conflicts
+- References to Middle Eastern institutions, oil economies, or geopolitics
+- Example: Gulf Cooperation Council, OPEC decisions, Middle East peace process, Islamic banking
+
+**`sub_saharan_african`** - Sub-Saharan African context
+- Content about African countries south of the Sahara
+- African Union topics, sub-Saharan development issues
+- References to African institutions, economies, or cultural topics
+- Example: M-Pesa mobile banking, African Union policies, safari tourism, ubuntu philosophy
+
+**`latin_american`** - Latin American context
+- Content about Central and South America, Caribbean
+- Latin American cultural perspectives, regional integration (Mercosur, etc.)
+- References to Latin American institutions, economies, or social movements
+- Example: Mercosur trade, telenovelas, Amazon rainforest, Latin American revolutions
+
+**`oceanian`** - Oceanian context
+- Content about Australia, New Zealand, Pacific Island nations
+- Oceanian perspectives, Pacific regional issues
+- References to Oceanian institutions, companies, or cultural topics
+- Example: ANZAC relations, Pacific Island climate change, Australian mining, Māori culture
+
+**`central_asian`** - Central Asian context
+- Content about Kazakhstan, Uzbekistan, Turkmenistan, Tajikistan, Kyrgyzstan
+- Central Asian perspectives, post-Soviet regional dynamics
+- Silk Road region, resource economies, nomadic heritage
+- Example: Silk Road initiatives, Caspian Sea resources, post-Soviet transitions
+
+**`russian_sphere`** - Russian/Post-Soviet context
+- Content about Russia, Belarus, and strong Russian influence areas
+- Post-Soviet perspectives, CIS (Commonwealth of Independent States) topics
+- Russian language content about regional (not global) topics
+- Example: Russian federal politics, CIS integration, post-Soviet economic transitions
+
+**`global`** - Genuinely international or universal
+- Content with truly global scope or application
+- International organizations, worldwide phenomena, global comparisons
+- Topics that transcend regional boundaries
+- Example: UN reports, climate change (global perspective), international standards, pandemic response
+
+**`culturally_neutral`** - No clear regional focus
+- Abstract, theoretical, or technical content without regional markers
+- Universal scientific, mathematical, or philosophical content
+- Content that could apply equally anywhere without modification
+- Example: Mathematical proofs, chemical formulas, abstract philosophy, programming concepts
+
+**`indeterminate`** - Cannot determine regional relevance
+- Insufficient content to identify regional focus
+- Mixed or contradictory regional signals
+- Fragment or corrupted content lacking regional context
+- Example: Technical specifications without context, isolated data tables
+
+##### Multi-Regional Examples:
+- **EU-China trade relations** → `["european", "east_asian"]`
+- **NAFTA/USMCA impact on Mexican agriculture** → `["north_american", "latin_american"]`
+- **Indian diaspora in the Gulf states** → `["south_asian", "middle_eastern"]`
+- **Comparative study of healthcare systems globally** → `["global"]`
+
+##### Regional Identification Guidelines:
+
+**Primary indicators:**
+- **Geographic references**: Countries, cities, regions, landmarks mentioned
+- **Institutional references**: Governments, companies, universities, organizations specific to region
+- **Cultural markers**: Holidays, customs, cultural phenomena, sports, entertainment
+- **Political/economic systems**: References to regional political structures, economic blocs
+- **Legal/regulatory frameworks**: Region-specific laws, regulations, standards
+- **Language context**: While not determinative, language can provide regional hints
+
+**Important distinctions:**
+- **Language ≠ Region**: Spanish content about Asian markets = `["east_asian"]`, not `["latin_american"]`
+- **Company origin vs. topic**: Apple (US company) operating in India = consider actual content focus
+- **Historical vs. current**: Historical content about ancient Rome = `["european"]` if discussing modern implications
+- **Diaspora content**: Content about diaspora communities should include both origin and current regions
+
+**Quality checks:**
+- If content is in a non-English language but discusses global topics → still mark as `["global"]`
+- If content compares multiple regions → mark all regions discussed substantially
+- If content is about a specific place but has universal applications → consider both regional and global tags
+---
+
+#### 18. Country Relevance
+
+**What we're measuring**: Which specific country or countries (if any) the content is relevant to, globally.
+
+**Note**: Always output an array of country names for this property (even when only a single country applies). Use standard country names from any region worldwide (e.g., "germany", "france", "united_states", "united_kingdom", "china", "japan", "brazil", "india", "south_africa", "australia", "canada", etc.). The array may also contain the special values `supranational` or `none`.
+
+##### Values & Criteria:
+
+**`{COUNTRY_NAME}`** - Content specifically relevant to a single country
+- Content explicitly about that country's politics, culture, institutions, or regulations
+- Content written from that country's cultural perspective
+- Content addressing that country's specific issues, regulations, or cultural phenomena
+- Content about that country's cities, companies, institutions, or country-specific topics
+- Example: For "germany" → German election coverage, Bundesliga content, German legal analysis
+- Example: For "united_states" → US election coverage, NFL content, US legal analysis
+- Example: For "japan" → Japanese politics, J-League content, Japanese cultural analysis
+- Only use country names listed in ISO-3166. Use "united_kingdom" instead of "england", "wales", etc.
+
+**`supranational`** - For content focused on supranational entities or regions
+- International organizations, regional blocs, global institutions
+- Content about supranational policies, international organizations, global governance
+- Pan-regional analysis that transcends individual countries
+- Multi-continental or global institutional content
+- Example: UN resolutions, NATO discussions, EU policy analysis, ASEAN agreements, WTO trade rules
+
+**`none`** - For content not specifically relevant to any country
+- Abstract, theoretical, or universal content without geographic specificity
+- Technical/scientific content that applies globally without country focus
+- Content that doesn't reference specific countries, cultures, or national contexts
+- Example: Mathematical proofs, universal scientific principles, abstract philosophical discussions
+
+
+##### Country Identification Criteria:
+- **Political content**: Elections, government policies, political parties, political figures specific to the country
+- **Cultural content**: National traditions, cultural phenomena, historical events specific to the country
+- **Institutional references**: Government bodies, national companies, universities specific to the country
+- **Geographic focus**: Cities, regions, landmarks within the country as primary subjects
+- **Legal/regulatory**: Laws, regulations, legal frameworks specific to the country
+- **Economic content**: National economic policies, country-specific market analysis
+- **Sports/media**: National sports leagues, national teams, country-specific media outlets
+- **Social issues**: Social policies, demographic topics, social movements specific to the country
+
+---

special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aeb13307a71acd8fe81861d94ad54ab689df773318809eed3cbe794b4492dae4
+size 11422654

tokenizer_config.json

@@ -0,0 +1,239 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151665": {
+      "content": "<tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151666": {
+      "content": "</tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151667": {
+      "content": "<think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151668": {
+      "content": "</think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 1010000,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}