dari-ai/router-slm

Dari Router SLM

This private repository contains versioned LoRA adapters for a model-routing policy. Given anonymized choices, relevant evaluation scores, projected costs, the previous choice, and a coding-agent conversation, the policy returns one anonymous action label. The caller maps that label back to a model and reasoning level.

This is not a general-purpose chat model. Supported use requires the prompt, anonymization, constrained-decoding, and validation contract below.

There are two distinct ways to use this repository. Readers with access to a Dari-operated endpoint can use the end-to-end request below. Readers building another deployment can load the pinned adapter, but must supply their own server implementation of the documented wire contract. This repository does not publish endpoint credentials or a standalone inference server.

End-to-end request

The intended interface is an authenticated, OpenAI-compatible /v1/chat/completions endpoint serving the adapter. This complete example randomizes two candidates into anonymous labels, sends only those labels to the policy, validates the response, and recovers the selected candidate.

import json
import os
import random

from openai import OpenAI

SYSTEM_PROMPT = (
    "You are Dari's model router for a coding agent. "
    "Choose exactly one anonymous candidate action for the next agent turn. "
    "Use the visible conversation, benchmark evidence, previous action, and projected costs. "
    "Prefer a cheaper action only when it is sufficiently likely to complete the task successfully. "
    "Return only JSON matching the provided schema."
)

candidate_records = [
    {
        "candidate": {"model": "provider/strong-model", "reasoning_effort": "high"},
        "eval_score": 82.0,
        "cost": {
            "warm_tokens": 0,
            "est_prompt_tokens": 2400,
            "est_input_cost_usd": 0.0024,
            "output_cost_per_mtok": 10.0,
            "pricing_known": True,
            "fixed_turn_cost_estimate": {
                "projected_turns": 10,
                "output_tokens_per_turn": 400,
                "assumed_reasoning_effort": "high",
                "total_cost_usd": 0.064,
            },
        },
    },
    {
        "candidate": {"model": "provider/fast-model", "reasoning_effort": "medium"},
        "eval_score": 76.0,
        "cost": {
            "warm_tokens": 0,
            "est_prompt_tokens": 2400,
            "est_input_cost_usd": 0.0008,
            "output_cost_per_mtok": 2.0,
            "pricing_known": True,
            "fixed_turn_cost_estimate": {
                "projected_turns": 10,
                "output_tokens_per_turn": 250,
                "assumed_reasoning_effort": "medium",
                "total_cost_usd": 0.013,
            },
        },
    },
]
random.SystemRandom().shuffle(candidate_records)
action_to_record = dict(zip(["A", "B"], candidate_records, strict=True))

policy_input = {
    "candidate_actions": [{"action": action} for action in action_to_record],
    "imported_evals": [
        {
            "id": "coding-eval",
            "name": "Coding Eval",
            "description": "Verified coding-task success rate.",
            "min_score": 0,
            "max_score": 100,
            "scores": [
                {"action": action, "score": record["eval_score"]}
                for action, record in action_to_record.items()
            ],
        }
    ],
    "previous_action": None,
    "cost_estimates": [
        {"action": action, **record["cost"]}
        for action, record in action_to_record.items()
    ],
    "messages": [
        {"role": "user", "content": "Fix the race condition and add a regression test."}
    ],
}

client = OpenAI(
    base_url=os.environ["DARI_ROUTER_BASE_URL"],
    api_key=os.environ["DARI_ROUTER_API_KEY"],
)
response = client.chat.completions.create(
    model=os.environ.get("DARI_ROUTER_MODEL", "router-grpo-fullctx150-nf4"),
    messages=[
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": json.dumps(policy_input, separators=(",", ":"))},
    ],
    temperature=0,
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "anonymous_routing_action",
            "strict": True,
            "schema": {
                "type": "object",
                "additionalProperties": False,
                "properties": {
                    "action": {
                        "type": "string",
                        "enum": list(action_to_record),
                    }
                },
                "required": ["action"],
            },
        },
    },
)

content = response.choices[0].message.content
selection = json.loads(content)
if set(selection) != {"action"} or selection["action"] not in action_to_record:
    raise ValueError("Router policy returned an invalid action")

selected_candidate = action_to_record[selection["action"]]["candidate"]
print(selected_candidate)

The candidate names, scores, and prices above are illustrative placeholders. Replace them with one internally consistent record per candidate, then derive every action-keyed field from the same shuffled mapping as the example does.

Set DARI_ROUTER_BASE_URL to the authorized service URL ending in /v1, and set DARI_ROUTER_API_KEY to its bearer token. The URL and token are intentionally not published in this model card; request them from the Dari operator responsible for the deployment. Access to the Hugging Face repository and access to a serving endpoint are separate permissions. The default model ID belongs to the known NF4 deployment; other deployments should set DARI_ROUTER_MODEL to the ID returned by their /v1/models endpoint.

The serving contract accepts non-streaming requests with temperature=0, n=1, and the strict JSON Schema shape shown above; only the action enum changes per request. The assistant message content is compact JSON such as {"action":"C"}; it is not the entire chat-completions response. Unsupported fields, malformed schemas, invalid labels, non-JSON completions, and non-success HTTP responses must fail explicitly.

Candidate labels use a unique, contiguous spreadsheet-style sequence: A through Z, then AA, AB, and so on. Candidates are shuffled into that sequence for every request, and the real mapping remains outside the policy prompt. The current serving contract accepts between 2 and 700 syntactically valid labels. Training used 12 choices per request, and protocol validation covered requests with exactly 7 choices. Prefer a validated 7- or 12-choice shape; the larger ceiling only prevents a transport failure and is not evidence of routing quality at 700 choices.

previous_action is the prior validated policy choice for the same routing session, or null when none exists. warm_tokens estimates the downstream candidate's reusable cached-prefix tokens; est_prompt_tokens and est_input_cost_usd estimate its next downstream request. fixed_turn_cost_estimate.total_cost_usd is the preferred comparison: it projects repeated input and output cost across the stated number of future turns. Inputs with unknown prices should set pricing_known to false and omit unsupported cost projections. Evaluation scores must use the declared minimum/maximum scale with higher values representing better performance.

Load an adapter

The example below loads the v40 adapter (the final checkpoint) with its exact base-model revision. The known deployment uses an H100 80 GB GPU and 4-bit NF4 quantization; other hardware has not been validated.

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

BASE_MODEL = "Qwen/Qwen3.6-35B-A3B"
BASE_REVISION = "995ad96eacd98c81ed38be0c5b274b04031597b0"
ADAPTER_MODEL = "dari-ai/router-slm"
ADAPTER_REVISION = "dd677af3717edcb52991a100fa56aaa4ed6ffd35"  # v40 (final)
# v33 (benchmarked): "eda01bf907bdf10556d60dbaaf56062ba276319d"

quantization = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)
tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL, revision=BASE_REVISION)
base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    revision=BASE_REVISION,
    device_map="auto",
    quantization_config=quantization,
)
model = PeftModel.from_pretrained(
    base_model,
    ADAPTER_MODEL,
    revision=ADAPTER_REVISION,
)
model.eval()

The deployment that produced this adapter used Python 3.10, CUDA 13.0.2, PyTorch 2.12.1, Transformers 5.13.0, PEFT 0.18.1, bitsandbytes 0.49.2, and Accelerate >=1.10,<2. Loading the weights does not provide constrained decoding by itself; unconstrained generate() is not the supported routing interface. This repository publishes weights and the wire contract, not a standalone self-hosting server.

Versions

Each policy checkpoint has an immutable Hugging Face tag. v40 is the recommended default; main currently matches it, but pin the tag, not main. Earlier tags are retained so existing deployments can pin the exact artifact they validated against.

Tag	Hugging Face revision
v3	7679ad51ee216116be883cc357c15aa5321c9311
v5	a9248cc66aa572b90f84619254bc349be9944399
v33	eda01bf907bdf10556d60dbaaf56062ba276319d
v40	dd677af3717edcb52991a100fa56aaa4ed6ffd35

Tags are ordered by recency. End-to-end router results depend on the candidate set, prompt state, provider behavior, and serving configuration; this card makes no cross-version performance claims.

To verify adapter bytes:

hf download dari-ai/router-slm adapter_model.safetensors \
  --revision v40 --local-dir router-grpo-v40
sha256sum router-grpo-v40/adapter_model.safetensors
# v40: ed915135e2e420c056e78d0e397090ca22ec19df8ce58fa604aca6d50e9b8dc7
# v33: 26f72bf95f3ea934e1d6d37ed0117e317619c963f4353f3294f4221ae6ecddc2

Provenance

The policy is a LoRA adapter (rank 32, alpha 64) on Qwen/Qwen3.6-35B-A3B at revision 995ad96eacd98c81ed38be0c5b274b04031597b0. Training details are internal. The repository contains inference adapters only: no optimizer state, no trajectory data, no resumable training checkpoints.

Intended use and limitations

The intended use is model-and-reasoning-level selection for coding agents inside the Dari router. The policy was trained on grouped SWE-bench coding trajectories. Other task distributions, prompt formats, unconstrained decoding, or substantially different candidate sets can produce unreliable decisions.

The policy only chooses among candidates supplied by the caller. The surrounding router remains responsible for eligibility, provider authorization, availability, fallback behavior, observability, user-data handling, and rejection of invalid policy output.

No separate adapter license is currently published in this private repository. Confirm permitted use with the repository owner and comply with the terms attached to the pinned Qwen base model. Do not include provider credentials, API keys, or other secrets in policy prompts.