Add model card

README.md

@@ -0,0 +1,188 @@
+---
+license: apache-2.0
+base_model: Qwen/Qwen2.5-Coder-1.5B-Instruct
+tags:
+  - text-to-sql
+  - nl2sql
+  - sql
+  - qlora
+  - gguf
+  - llama-cpp
+  - edge
+  - on-prem
+language:
+  - en
+library_name: gguf
+pipeline_tag: text-generation
+datasets:
+  - xlangai/spider
+---
+
+# TinySQL-1.5B
+
+**A private, on-prem Natural-Language-to-SQL model that runs on a laptop CPU.**
+
+TinySQL is [Qwen2.5-Coder-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct)
+fine-tuned with QLoRA for text-to-SQL and quantized to 4-bit GGUF (Q4_K_M) so it
+runs offline via `llama.cpp` — no GPU, no cloud, no per-query cost. It converts an
+English question + a database schema into a **validated, read-only SQL SELECT**.
+
+The point is **not** to beat frontier models on raw accuracy. It is to be the
+*compliant, $0/query, offline* option for regulated data (finance, health, legal,
+government) where the schema + data **cannot** leave the premises.
+
+---
+
+## Measured results
+
+All numbers below are **measured**, not estimated. Evaluation is **execution
+accuracy** on the full **Spider dev split (1,034 examples)**: run the predicted SQL
+and the gold SQL against the real SQLite database and compare returned rows
+(order-insensitive), over a read-only connection.
+
+### Fine-tuning lift (apples-to-apples)
+
+Identical base model, identical Q4_K_M quantization, identical prompt template and
+SELECT-only guardrail — **only the QLoRA adapter differs**.
+
+| Model | Spider dev exec. acc | Valid (SELECT-only + executes) | Malformed outputs |
+|-------|----------------------|--------------------------------|-------------------|
+| Base Qwen2.5-Coder-1.5B | 50.87% | 76.9% | 54 |
+| **TinySQL-1.5B** | **62.86%** | **87.99%** | **0** |
+
+- **+11.99 points** execution accuracy from fine-tuning.
+- **54 → 0** malformed outputs: the base model emitted non-SQL chatter and
+  degenerate repetition loops; the fine-tune produces clean, parseable SELECTs.
+
+### Performance (laptop CPU, Intel Core Ultra 5 235U)
+
+| Metric | Value |
+|--------|-------|
+| Mean latency | 0.57 s / query |
+| p95 latency | 0.75 s |
+| Peak RAM | ~1.73 GB |
+| Cost | $0 / query (self-hosted) |
+
+> Larger models and cloud APIs achieve higher accuracy, but require GPU/cloud and
+> send your schema + data off-premises. TinySQL trades peak accuracy for privacy,
+> $0 cost, and offline operation — the axes that matter for regulated data.
+
+---
+
+## Intended use
+
+- Private/on-prem "text-to-SQL copilot" for non-technical users to query a database
+  in plain English.
+- Embedded/offline analytics (edge devices, desktop apps) with no network.
+- A cheap first-pass layer that handles routine queries locally, escalating only
+  hard ones to a larger model.
+
+**Read-only by design.** Generated SQL is validated to be a single SELECT before it
+is shown or executed. It cannot INSERT/UPDATE/DELETE/DROP.
+
+## Out of scope / limitations
+
+- **Not for autonomous critical decisions** — ~63% accuracy means a human should
+  verify before acting on results.
+- **No writes** — SELECT-only.
+- **Schema size** — trained/served at 2048-token context; very large schemas are
+  pruned and accuracy drops.
+- **SQLite dialect** — targets SQLite SQL.
+
+---
+
+## How to use
+
+### With `llama-cpp-python`
+
+```python
+from llama_cpp import Llama
+
+llm = Llama(model_path="tinysql-1.5b-q4_k_m.gguf", n_ctx=2048, verbose=False)
+
+INSTRUCTION = ("You are a SQL expert. Given the database schema, write a single "
+               "SQLite SELECT query that answers the question. Return ONLY the SQL.")
+
+schema = """CREATE TABLE orders (
+  id INTEGER PRIMARY KEY,
+  status TEXT,
+  customer_id INTEGER
+);"""
+question = "How many orders are completed?"
+
+prompt = (f"### Instruction:\n{INSTRUCTION}\n\n"
+          f"### Schema:\n{schema}\n\n"
+          f"### Question:\n{question}\n\n"
+          f"### SQL:\n")
+
+out = llm(prompt, max_tokens=256, temperature=0.0, stop=["###"])
+print(out["choices"][0]["text"].strip())
+# -> SELECT count(*) FROM orders WHERE status = 'completed';
+```
+
+### With `llama.cpp` CLI
+
+```bash
+llama-cli -m tinysql-1.5b-q4_k_m.gguf -p "### Instruction:..." -n 256 --temp 0
+```
+
+**Always enforce SELECT-only + run against a read-only DB connection** before
+executing generated SQL. Do not run model output with write permissions.
+
+---
+
+## Prompt format
+
+The model was trained with (and expects) this exact template:
+
+```
+### Instruction:
+You are a SQL expert. Given the database schema, write a single SQLite
+SELECT query that answers the question. Return ONLY the SQL.
+
+### Schema:
+{CREATE TABLE statements}
+
+### Question:
+{natural-language question}
+
+### Evidence:        # optional external-knowledge hint (BIRD-style)
+{hint}
+
+### SQL:
+```
+
+---
+
+## Training
+
+- **Base:** Qwen/Qwen2.5-Coder-1.5B-Instruct
+- **Method:** QLoRA (4-bit), LoRA r=16, α=16
+- **Data:** Spider + BIRD, ~15.4k instruction examples, SELECT-only, FK-aware schema
+  pruning to fit 2048 tokens
+- **Shipped checkpoint:** 500 steps (~0.26 epoch). A training-length ablation found
+  accuracy peaks early: 500 steps = 62.86%, 1000 = 61.22%, 1800 = 40.81% (overfit).
+  **Less was more.**
+- **Export:** merged to 16-bit → GGUF → quantized Q4_K_M for CPU/edge.
+
+---
+
+## Datasets & attribution
+
+- **Spider** (Yu et al., 2018) — CC BY-SA 4.0
+- **BIRD** (Li et al., 2023) — CC BY-SA 4.0
+
+Base model **Qwen2.5-Coder-1.5B-Instruct** is Apache-2.0. This fine-tune is released
+under **Apache-2.0**; please also honor the CC BY-SA 4.0 attribution for Spider/BIRD.
+
+## Citation
+
+```bibtex
+@misc{tinysql2026,
+  title  = {TinySQL: Private On-Prem NL-to-SQL on a Laptop CPU},
+  author = {Indirakumar},
+  year   = {2026},
+  howpublished = {\url{https://huggingface.co/Indirakumar01/tinysql-1.5b}},
+  note   = {Fine-tuned Qwen2.5-Coder-1.5B, GGUF Q4_K_M}
+}
+```