Upload 12 files

.gitattributes

@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+axl_reasoning-q4_k_m_v2.gguf filter=lfs diff=lfs merge=lfs -text
+axl-reasoning-f16.gguf filter=lfs diff=lfs merge=lfs -text
+axl-reasoning-q4_k_m.gguf filter=lfs diff=lfs merge=lfs -text

Modelfile

@@ -0,0 +1,14 @@
+FROM ./axl_reasoning_70m-f16.gguf
+
+TEMPLATE """{{ .System }}
+
+User: {{ .Prompt }}
+Assistant: """
+
+SYSTEM """You are AXL-Reasoning-70M, a code generation assistant built by Koinic."""
+
+PARAMETER temperature 0.8
+PARAMETER top_k 40
+PARAMETER top_p 0.9
+PARAMETER repeat_penalty 1.1
+PARAMETER num_ctx 256

README.md

@@ -1,3 +1,189 @@
 ---
 license: apache-2.0
+language:
+  - code
+tags:
+  - code-generation
+  - multi-scale-transformer
+  - cpu-optimized
+  - koinic
+  - pytorch
+  - llama
+  - gguf
+  - byte-level
+pipeline_tag: text-generation
+library_name: transformers
+datasets:
+  - bigcode/starcoderdata
+  - theblackcat102/evol-codealpaca-v1
+widget:
+  - text: "# Find GCD of two numbers\ndef gcd(a, b):"
+model-index:
+  - name: AXL-Reasoning-70M
+    results:
+      - task:
+          type: text-generation
+        metrics:
+          - name: Perplexity (byte-level)
+            type: perplexity
+            value: 1.93
 ---
+
+# AXL-Reasoning-70M
+
+CoT reasoning (SGD). 70M params. PPL 1.93 Part of the AXL model family by [KoinicLabs](https://huggingface.co/KoinicLabs).
+
+## Model Details
+
+| Property | Value |
+|----------|-------|
+| Developed by | [KoinicLabs](https://huggingface.co/KoinicLabs) |
+| Architecture | Multi-Scale Transformer |
+| Parameters | 70M |
+| Optimizer | SGD |
+| Attention | SDPA |
+| Vocab Size | 258 (byte-level) |
+| Context Window | 512 bytes |
+| d_model | 512 |
+| Attention Heads | 4 |
+| Layers per Scale | 5 |
+| Downsample Factors | [1, 2, 4] |
+| License | Apache 2.0 |
+
+### Sources
+
+- **Repository:** [GitHub](https://github.com/Koinic/AXL)
+- **Organization:** [KoinicLabs](https://huggingface.co/KoinicLabs)
+
+## Uses
+
+### Direct Use
+
+Chain-of-thought reasoning (SGD baseline).
+
+```python
+import torch
+from multiscale_transformer.model.model import MultiScaleTransformer
+from multiscale_transformer.training.tokenizer import ByteTokenizer
+ckpt = torch.load("axl_reasoning_70m.pt", map_location="cpu")
+model = MultiScaleTransformer(config)
+model.load_state_dict(ckpt["model_state_dict"])
+model.eval()
+tokenizer = ByteTokenizer()
+ids = torch.tensor([tokenizer.encode("def hello():")], dtype=torch.long)
+with torch.no_grad():
+    out = model.generate(ids, max_new_tokens=50, temperature=0.8)
+print(tokenizer.decode(out[0].tolist()))
+```
+
+### Out-of-Scope Use
+
+Not for production code generation. Use the Lion version for better results. For integration with tools like Continue.dev, LlamaIndex, or LangChain, use the Python API server which provides OpenAI-compatible endpoints.
+
+## Bias, Risks, and Limitations
+
+Byte-level perplexity is not comparable to BPE-level perplexity. SGD baseline. Use AXL-Reasoning-Lion for better results. Note: GGUF files for Ollama use a simplified single-stack encoder. For full AXL quality, use the Python API server.
+
+### Recommendations
+
+- Use for prototyping and experimentation, not production code generation.
+- Byte-level perplexity (258 vocab) is not comparable to BPE-level perplexity (32K vocab).
+- For better results, use the Lion-optimized version if available.
+
+## Training Details
+
+### Training Data
+
+SGD 60 min. 1478 steps. Lion version achieves PPL 1.93 in 20 min.
+
+### Preprocessing
+
+Byte-level tokenization with vocabulary size 258 (256 bytes + BOS + EOS). No vocabulary training required.
+
+### Speeds, Sizes, Times
+
+| Metric | Value |
+|--------|-------|
+| Training Steps | 1478 |
+| Training Time | 60 min |
+| Final Loss | 0.6384 |
+
+## Evaluation
+
+### Metrics
+
+Perplexity on held-out Python code using byte-level tokenization.
+
+### Results
+
+| Metric | Value |
+|--------|-------|
+| Perplexity (byte-level) | 1.93 |
+| Final Loss | 0.6384 |
+| Training Steps | 1478 |
+| Training Time | 60 min |
+
+**Summary:** SGD reasoning baseline. Lion version is significantly better.
+
+## Environmental Impact
+
+| Property | Value |
+|----------|-------|
+| Hardware | AMD Ryzen 5 5600G |
+| Hours Used | 1.001 |
+| Carbon Emitted | 0.0420 kg CO2 |
+| Cloud Provider | None (local CPU) |
+
+## Technical Specifications
+
+### Model Architecture
+
+Multi-Scale Transformer with three parallel encoder stacks at resolution scales 1x, 2x, and 4x. Cross-scale attention connects all scale pairs. Adaptive gating fusion. SwiGLU feed-forward. RoPE positional encoding.
+
+### Compute Infrastructure
+
+| Property | Value |
+|----------|-------|
+| Hardware | AMD Ryzen 5 5600G (6 cores, 12 threads) |
+| RAM | 16 GB |
+| GPU | None (CPU-only) |
+
+## Citation
+
+```bibtex
+@misc{axl_2026,
+  title={AXL: AXL-Reasoning-70M - Multi-Scale Transformer for CPU Code Generation},
+  author={Koinic},
+  year={2026},
+  url={https://huggingface.co/KoinicLabs}
+}
+```
+
+## How to Get Started
+
+### With Ollama
+
+```bash
+ollama create axl-reasoning-70m -f Modelfile
+ollama run axl-reasoning-70m "def fibonacci():"
+```
+
+### With Python
+
+```python
+import torch
+from multiscale_transformer.model.config import load_config
+from multiscale_transformer.model.model import MultiScaleTransformer
+from multiscale_transformer.training.tokenizer import ByteTokenizer
+config = load_config("config.json")
+model = MultiScaleTransformer(config)
+ckpt = torch.load("axl_reasoning_70m.pt", map_location="cpu")
+model.load_state_dict(ckpt["model_state_dict"])
+model.eval()
+tokenizer = ByteTokenizer()
+prompt = "def fibonacci():"
+ids = torch.tensor([tokenizer.encode(prompt)], dtype=torch.long)
+with torch.no_grad():
+    out = model.generate(ids, max_new_tokens=100, temperature=0.8, top_k=40)
+print(tokenizer.decode(out[0].tolist()))
+```

axl-reasoning-f16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ae4aa1771a77c666d0ace8f22ff85e73a20c96235c3735c73cba5e35fe5e9032
+size 140299104

axl-reasoning-q4_k_m.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ae4aa1771a77c666d0ace8f22ff85e73a20c96235c3735c73cba5e35fe5e9032
+size 140299104

axl_reasoning-q4_k_m_v2.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f786213021cddfb83bfc7dd96cec5aaa27e2606aa34352f2c937cb04512cad3a
+size 43637536

axl_reasoning.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bdfc52009fdec562225bd4e338902387649c450f396bbf83713e685ad01a3509
+size 280121339

config.json

@@ -0,0 +1,30 @@
+{
+  "model_type": "multiscale_transformer",
+  "architectures": [
+    "MultiScaleForCausalLM"
+  ],
+  "vocab_size": 258,
+  "d_model": 512,
+  "n_heads": 4,
+  "d_ff": 1376,
+  "n_layers_per_scale": 5,
+  "n_cross_attn_layers": 1,
+  "max_seq_len": 512,
+  "dropout": 0.0,
+  "bias": false,
+  "rope_theta": 10000.0,
+  "downsample_factors": [
+    1,
+    2,
+    4
+  ],
+  "num_parameters": 66831872,
+  "training_results": {
+    "model": "AXL-Reasoning",
+    "params": 70004224,
+    "steps": 1478,
+    "time": 3602.1849868297577,
+    "final_loss": 0.6384159326553345,
+    "perplexity": 1.93
+  }
+}

generation_config.json

@@ -0,0 +1,8 @@
+{
+  "max_new_tokens": 256,
+  "temperature": 0.8,
+  "top_k": 40,
+  "top_p": 0.9,
+  "repetition_penalty": 1.1,
+  "do_sample": true
+}

index.html

@@ -0,0 +1,95 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<meta charset="UTF-8">
+<meta name="viewport" content="width=device-width,initial-scale=1.0">
+<title>AXL-Reasoning-70M - AXL</title>
+<style>*{margin:0;padding:0;box-sizing:border-box}
+body{font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,sans-serif;background:#0d1117;color:#c9d1d9;line-height:1.6}
+a{color:#58a6ff;text-decoration:none}a:hover{text-decoration:underline}
+.hero{padding:40px 20px;text-align:center;border-bottom:1px solid #30363d;background:linear-gradient(135deg,#0d1117,#161b22,#0d1117)}
+.hero h1{font-size:2.2rem;color:#fff;letter-spacing:-1px}
+.cat{display:inline-block;padding:3px 12px;border-radius:12px;font-size:.75rem;font-weight:600;margin-bottom:12px}
+.cat.Lion{background:#1f3a5f;color:#4285f4}
+.cat.SGD{background:#3d1f1f;color:#f85149}
+.cat.Specialized{background:#2d1b69;color:#bb86fc}
+.desc{color:#8b949e;font-size:.95rem;max-width:600px;margin:12px auto 0}
+.ms{display:flex;flex-wrap:wrap;gap:12px;justify-content:center;padding:24px 20px}
+.mc{background:#161b22;border:1px solid #30363d;border-radius:10px;padding:16px 24px;text-align:center;min-width:120px}
+.v{font-size:1.5rem;font-weight:700;color:#fff}.l{font-size:.75rem;color:#8b949e;margin-top:2px}
+.tabs{max-width:800px;margin:0 auto;padding:0 20px}
+.tabs>input[type=radio]{display:none}
+.tl{display:inline-block;background:#21262d;border:1px solid #30363d;color:#8b949e;padding:7px 16px;border-radius:8px;cursor:pointer;font-size:.85rem;margin:0 4px 16px;transition:all .2s}
+.tl:hover{background:#30363d;color:#c9d1d9}
+.p{display:none;background:#161b22;border:1px solid #30363d;border-radius:12px;padding:24px;margin-bottom:24px}
+#t1:checked~.p1,#t2:checked~.p2,#t3:checked~.p3,#t4:checked~.p4{display:block}
+#t1:checked+label[for=t1],#t2:checked+label[for=t2],#t3:checked+label[for=t3],#t4:checked+label[for=t4]{background:#f85149;color:#fff;border-color:#f85149}
+table{width:100%;border-collapse:collapse}
+th{text-align:left;color:#8b949e;font-size:.8rem;padding:8px 12px;border-bottom:1px solid #21262d;font-weight:600}
+td{padding:8px 12px;font-size:.9rem;border-bottom:1px solid #21262d}
+pre{background:#0d1117;padding:14px;border-radius:8px;overflow-x:auto;margin:12px 0}
+code{color:#c9d1d9;font-size:.82rem;line-height:1.5}
+.note{background:#21262d;border-left:3px solid #f85149;padding:12px 16px;border-radius:0 8px 8px 0;margin:12px 0;font-size:.85rem;color:#8b949e}
+.story{font-size:.9rem;color:#8b949e;line-height:1.6;margin:8px 0}
+.back{text-align:center;padding:24px 20px 40px}
+.back a{color:#58a6ff;font-size:.9rem}
+@media(max-width:768px){.hero h1{font-size:1.6rem}.ms{flex-direction:column;align-items:center}.mc{min-width:200px}}</style>
+</head>
+<body>
+<div class="hero">
+<div class="cat SGD">SGD Optimized</div>
+<h1>AXL-Reasoning-70M</h1>
+<p class="desc">CoT reasoning (SGD). 70M params. PPL 1.93.</p>
+</div>
+<div class="ms">
+<div class="mc"><div class="v">70M</div><div class="l">Parameters</div></div>
+<div class="mc"><div class="v">1.93</div><div class="l">Perplexity</div></div>
+<div class="mc"><div class="v">60 min</div><div class="l">Training</div></div>
+<div class="mc"><div class="v">140 MB</div><div class="l">GGUF</div></div>
+</div>
+<div class="tabs">
+<input type="radio" name="t" id="t1" checked><label for="t1" class="tl">Specs</label>
+<input type="radio" name="t" id="t2"><label for="t2" class="tl">Training</label>
+<input type="radio" name="t" id="t3"><label for="t3" class="tl">Usage</label>
+<input type="radio" name="t" id="t4"><label for="t4" class="tl">Download</label>
+<div class="p p1">
+<table>
+<tr><th>Property</th><th>Value</th></tr>
+<tr><td>Architecture</td><td>Multi-Scale Transformer</td></tr>
+<tr><td>d_model</td><td>?</td></tr>
+<tr><td>Attention Heads</td><td>?</td></tr>
+<tr><td>Layers per Scale</td><td>?</td></tr>
+<tr><td>Context Window</td><td>256 bytes</td></tr>
+<tr><td>Downsample Factors</td><td>[1, 2, 4]</td></tr>
+<tr><td>Vocab Size</td><td>258 (byte-level)</td></tr>
+<tr><td>Optimizer</td><td>SGD</td></tr>
+</table>
+</div>
+<div class="p p2">
+<div class="story">SGD 60 min. 1478 steps. Lion version achieves PPL 1.79 in 20 min.</div>
+<table>
+<tr><th>Metric</th><th>Value</th></tr>
+<tr><td>Final Loss</td><td>0.6384</td></tr>
+<tr><td>Perplexity</td><td>1.93</td></tr>
+<tr><td>Training Steps</td><td>1478</td></tr>
+<tr><td>Training Time</td><td>60 min</td></tr>
+</table>
+</div>
+<div class="p p3">
+<h3 style="color:#fff;margin-bottom:12px">Usage</h3>
+<pre><code>ollama create axl-reasoning-70m -f Modelfile
+ollama run axl-reasoning-70m "def fibonacci():"</code></pre>
+<div class="note">SGD reasoning baseline. Lion version is significantly better.</div>
+</div>
+<div class="p p4">
+<table>
+<tr><th>File</th><th>Size</th><th>Format</th></tr>
+<tr><td>F16 GGUF</td><td>140 MB</td><td>Full precision</td></tr>
+<tr><td>Q4_K_M GGUF</td><td>140 MB</td><td>4-bit quantized</td></tr>
+</table>
+<div class="note" style="margin-top:16px">GGUF files work with Ollama and llama.cpp. Q4_K_M is about 3x smaller than F16.</div>
+</div>
+</div>
+<div class="back"><a href="../">← All AXL Models</a></div>
+</body>
+</html>

results.json

@@ -0,0 +1,8 @@
+{
+  "model": "AXL-Reasoning",
+  "params": 70004224,
+  "steps": 1478,
+  "time": 3602.1849868297577,
+  "final_loss": 0.6384159326553345,
+  "perplexity": 1.93
+}

special_tokens_map.json

@@ -0,0 +1,6 @@
+{
+  "pad_token": "[PAD]",
+  "bos_token": "[BOS]",
+  "eos_token": "[EOS]",
+  "unk_token": "[UNK]"
+}

tokenizer_config.json

@@ -0,0 +1,8 @@
+{
+  "tokenizer_class": "ByteTokenizer",
+  "vocab_size": 258,
+  "pad_token": "[PAD]",
+  "bos_token": "[BOS]",
+  "eos_token": "[EOS]",
+  "unk_token": "[UNK]"
+}