Instructions to use FINAL-Bench/POCKET-35B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use FINAL-Bench/POCKET-35B-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf FINAL-Bench/POCKET-35B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf FINAL-Bench/POCKET-35B-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf FINAL-Bench/POCKET-35B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf FINAL-Bench/POCKET-35B-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf FINAL-Bench/POCKET-35B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama cli -hf FINAL-Bench/POCKET-35B-GGUF:Q4_K_M

Use Docker

docker model run hf.co/FINAL-Bench/POCKET-35B-GGUF:Q4_K_M

Llama new
LM Studio
Jan

vLLM

How to use FINAL-Bench/POCKET-35B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "FINAL-Bench/POCKET-35B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FINAL-Bench/POCKET-35B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/FINAL-Bench/POCKET-35B-GGUF:Q4_K_M

Ollama
How to use FINAL-Bench/POCKET-35B-GGUF with Ollama:
```
ollama run hf.co/FINAL-Bench/POCKET-35B-GGUF:Q4_K_M
```
Unsloth Desktop

How to use FINAL-Bench/POCKET-35B-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf FINAL-Bench/POCKET-35B-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "FINAL-Bench/POCKET-35B-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use FINAL-Bench/POCKET-35B-GGUF with Docker Model Runner:
```
docker model run hf.co/FINAL-Bench/POCKET-35B-GGUF:Q4_K_M
```

Lemonade

How to use FINAL-Bench/POCKET-35B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull FINAL-Bench/POCKET-35B-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.POCKET-35B-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use FINAL-Bench/POCKET-35B-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf FINAL-Bench/POCKET-35B-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default FINAL-Bench/POCKET-35B-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use FINAL-Bench/POCKET-35B-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf FINAL-Bench/POCKET-35B-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "FINAL-Bench/POCKET-35B-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Meet POCKET-27B. Qwen3.8-27B in ARCHsloth Q6_K GGUF. Choose files & download · Quality & limits

🏆 SuperCLUE November 2026: #1 open-weight model in math and science reasoning

From the VIDRAFT Darwin family. Our flagship reasoning model, Darwin-180B-RSI-R3, was scored by SuperCLUE, China's most widely cited LLM benchmark, on its private November 2026 test set: 73.68 in mathematical and scientific reasoning, #1 among open-weight models and #2 overall among 19 models, behind only OpenAI's GPT-6.1-Sol and ahead of Gemini-4-Argon, DeepSeek-V4.1-Flash, Grok-4.7 and Qwen3.8-Max. The same recursive self-improvement line and the same team built the POCKET on-device builds.

Model	Developer	Open weights	Math & science reasoning
GPT-6.1-Sol	OpenAI	No	76.32
Darwin-180B-RSI-R3	VIDRAFT	Yes	73.68
Gemini-4-Argon	Google	No	71.05
DeepSeek-V4.1-Flash	DeepSeek	Yes	71.05
Grok-4.7	xAI	No	68.42
Qwen3.8-Max-0902	Alibaba	No	68.42
GLM-5.3	Zhipu AI	Yes	65.79
Kimi-K3.1-Pro	Moonshot	Yes	63.16
Qwen3.8-Flash (parent of Darwin-180B)	Alibaba	Yes	57.89

Darwin-180B-RSI inherited its expert layers from Qwen3.8-Flash-Next and rebuilt only the attention and shared-expert layers through recursive self-improvement. On the same board the parent, Qwen3.8-Flash, placed 15th overall (39.81) with 57.89 in math and science reasoning; Darwin placed 9th (43.08) with 73.68.

Overall intelligence index, November 2026 (screenshot)

Source: SuperCLUE general leaderboard, November 2026 (screenshots taken November 5, 2026). Scores are SuperCLUE's, not VIDRAFT's.

POCKET-35B-GGUF

Korean-tuned 35B local AI for PCs: choose a full-model GGUF build and run without a GPU.

Download files · Setup guide · Benchmarks

POCKET-35B · POCKET-26B · POCKET-Darwin-180B

At a glance

Item	Details
Artifact	Full 34.66B-parameter sparse MoE, approximately 3B active parameters per token; GGUF
Starting checkpoint	FINAL-Bench/Darwin-36B-Opus
Recommended first build	Q2_K, approximately 13 GB
Hardware	PC with sufficient RAM; a CPU-only execution path is available
Runtime	llama.cpp with qwen35moe architecture support; check the installed build
License	Apache-2.0; see License and attribution
Evidence status	Publisher measurements with device-specific conditions below

This repository contains the full-model PC builds, not the approximately 5 GB phone derivatives. For a phone, use the separate Korean GGUF, Korean MLX, or English GGUF repository and its own requirements.

Choose your build

Build	File size	Memory guidance	Best for
Q2_K — recommended	About 13 GB	Publisher guidance: 16–24 GB PC RAM; allow OS and context headroom	Balanced daily use
Q4_K_M	About 21 GB	Publisher guidance: 32 GB PC RAM	Highest measured Korean quality among these builds
Q3_K_M	About 16 GB	Publisher guidance: 24 GB PC RAM	Intermediate size and quality
IQ1_M	About 8.2 GB	Publisher guidance: 16 GB PC RAM	Smallest full-model file; greater quality loss

File sizes are rounded decimal GB, not process-memory measurements. Context, runtime buffers, the operating system and other applications need additional memory. The approximately 8.2 GB IQ1_M file is approximately 7.7 GiB; do not interpret those units as different builds.

Download and run

Install the Hugging Face CLI and a llama.cpp build that supports qwen35moe. The example downloads only the recommended file from the audited revision; it does not clone or download the whole repository.

1. Download the recommended build.

hf download FINAL-Bench/POCKET-35B-GGUF POCKET-35B-Q2_K.gguf --revision 3870039a87406fb90764b27e3984c8b1a36b269e --local-dir ./POCKET-35B

2. Run on the CPU.

llama-cli -m ./POCKET-35B/POCKET-35B-Q2_K.gguf --device none -ngl 0 -t 8 -c 4096 -n 256 -p "Explain why local AI can be useful."

The expected result is generated text in the terminal, not a fixed benchmark answer. These command examples were syntax-reviewed but not executed during this review.

Commands use the standalone llama.cpp executable names. Some distributions expose a unified llama launcher; consult that distribution's help and the official model-loading documentation. For -t, start with the physical-core count rather than all logical threads; more threads can reduce throughput.

For installation, memory errors and app-specific help, use the model-specific setup guide. Model files remain directly available from Hugging Face without a VIDRAFT login.

Runtime compatibility

Runtime	Requirement	Evidence and limits
llama.cpp	A build with qwen35moe support	The original card reports upstream llama.cpp use; an exact build/hash was not supplied for every measurement
Ollama	A version supporting this model architecture and GGUF	Generic Hugging Face GGUF integration exists; an exact file/version compatibility matrix is not supplied
LM Studio	A bundled runtime supporting this model architecture	Paste the repository URL into model search; check the app's compatibility information and available memory
PocketPal and phone runtimes	Use a separately supplied phone derivative and its requirements	No performance guarantee for these full-model PC files on phones
Apple MLX	A separately supplied MLX artifact, not this GGUF file	See POCKET-KR-MLX

The same file format does not guarantee support in every app or version. A dated, exact-version compatibility matrix remains a community contribution opportunity.

Hardware and performance

These are publisher-reported measurements, not promised speeds for arbitrary PCs. The CPU-only quickstart explicitly disables GPU acceleration with --device none; -ngl 0 alone is not the strongest no-GPU guarantee in every backend.

Device or mode	Configuration	Generation	Evidence and limits
Server CPU	Xeon, 16 threads; IQ1_M	27.0 tok/s	Same-machine comparison below; exact CPU SKU/build and run repetitions were not supplied
Datacenter GPU	H100; IQ1_M	197 tok/s	GPU result, not CPU-only performance
Mac CPU	MacBook M3 Pro, 18 GB, 8 threads; IQ1_M	13.8 tok/s	Measured in the original card
Mac Metal	MacBook M3 Pro, 18 GB; IQ1_M, tg64	25.4 tok/s	Metal uses the integrated GPU
Mac CPU	MacBook M3 Pro, 18 GB; Q2_K	19.5 tok/s	Different quant from the headline IQ1_M measurement
iPhone / Strix Halo	No matching throughput measurement supplied	Not measured	Do not infer speed from file size or a related platform

On the 18 GB M3 Pro, the original card recommends Q2_K on CPU because its approximately 13 GB file exceeds the recommended Metal memory budget. This is a device-specific recommendation, not a universal Apple-memory limit.

Same-machine Bonsai comparison and benchmark command

Measurement	POCKET-35B IQ1_M	Bonsai-27B Q1_0	Conditions
Server CPU generation	27.0 tok/s	10.1 tok/s	Xeon, 16 threads
H100 generation	197 tok/s	89 tok/s	Same stock llama.cpp comparison
H100 prompt processing	753 tok/s	1,816 tok/s	Bonsai is faster on this measurement
M3 Pro Metal generation	25.4 tok/s	12.8 tok/s	tg64, 18 GB MacBook
M3 Pro CPU generation	13.8 tok/s	4.4 tok/s	8 threads
M3 Pro Metal prompt processing	240.7 tok/s	73.4 tok/s	pp128
M3 Pro CPU prompt processing	45.5 tok/s	9.6 tok/s	pp128

The configurations do not establish equal quality, equal file size or a universal architecture advantage. The original card separately reports that Ternary-Bonsai-27B-Q2_0 required a PrismML fork; compatibility can change with releases.

To inspect the original CPU benchmark shape, first download IQ1_M instead of Q2_K, then run:

hf download FINAL-Bench/POCKET-35B-GGUF POCKET-35B-IQ1_M.gguf --revision 3870039a87406fb90764b27e3984c8b1a36b269e --local-dir ./POCKET-35B
llama-bench -m ./POCKET-35B/POCKET-35B-IQ1_M.gguf -p 128 -n 64 --device none -ngl 0 -t 16

This is a reproduction starting point, not proof of the published result on a different computer or runtime.

Quality and evaluation

Publisher-reported results. Each table has its own task, model and measurement conditions.

Build	Evaluation	Result	Conditions and limits
Q4_K_M	Korean Wikipedia perplexity	5.79	Lower is better; reference within this model
Q3_K_M	Korean Wikipedia perplexity	6.06	Same reported Korean comparison
Q2_K	Korean Wikipedia perplexity	6.49	Size/quality compromise
IQ1_M	Korean Wikipedia perplexity	9.69	Greater Korean degradation

Build	Evaluation	Result	Conditions and limits
Qwen3.6-35B-A3B reference	GPQA Diamond	73.2%	198 questions, greedy; a distinct reference model
POCKET-35B Q4_K_M	GPQA Diamond	68.7%	198 questions, greedy
POCKET-35B Q2_K	GPQA Diamond	60.1%	198 questions, greedy
POCKET-35B IQ1_M	HellaSwag	61.0%	400 questions
Bonsai-27B Q1_0 reference	HellaSwag	60.0%	Same reported 400-question comparison; confidence intervals overlap

The Qwen3.6 reference is not a same-checkpoint Darwin BF16 baseline. Do not interpret its difference as quantization-only loss. Small benchmark subsets and overlapping confidence intervals do not establish general quality equivalence.

Lineage and language-specific derivatives

POCKET is quantized from Darwin-36B-Opus, VIDRAFT's multi-generation Darwin model. The original card describes a Qwen3.5-family sparse MoE architecture with 256 experts and top-8 routing; the distinct Qwen3.6 model above is an evaluation reference, not a replacement starting-checkpoint declaration.

Component	Attribution
Starting checkpoint	Darwin-36B-Opus by VIDRAFT
GGUF quantization and runtime	Upstream llama.cpp; standard GGUF types
Full-model files in this repository	Q4_K_M, Q3_K_M, Q2_K and IQ1_M
Language-specific tuning and phone derivatives	Separate VIDRAFT KR/EN builds, with their own quality and compatibility trade-offs

The original card reports stronger Korean degradation at extreme precision and different Korean/English phone trade-offs. Do not transfer a language-specific derivative's quality or memory results to the full-model files in this repository.

Intended uses

Local text assistance, experimentation and Korean-language workflows on a PC with suitable memory.
Comparing quantization size, Korean perplexity and throughput on a clearly identified device.
CPU-only workflows where local execution is preferred; no cloud service is required by the supplied command.

Limitations

The smallest file in this repository is about 8.2 GB, not a 5 GB phone build.
IQ1_M trades quality for size; Q2_K or a larger build is preferable when quality matters.
MacBook M3 Pro measurements are supplied above. iPhone and Strix Halo throughput remain unmeasured in the supplied evidence.
File size is not a complete RAM budget; longer context increases memory requirements.
Published speed and benchmark results are device-, build-, quant- and protocol-specific, not universal guarantees.

License and attribution

This model card declares Apache-2.0, consistent with the current immediate starting checkpoint. Read the Apache-2.0 license and the starting checkpoint's license before redistribution.

Starting checkpoint: FINAL-Bench/Darwin-36B-Opus. POCKET quantization, language tuning and documentation: VIDRAFT / FINAL-Bench. Runtime tooling: the llama.cpp contributors. Keep required notices and attribution when redistributing.

Support and updates

For device selection and installation help, visit the VIDRAFT POCKET setup guide. Report reproducible issues in the model discussions, including the filename, runtime version, device, memory and command; do not post private prompts or credentials.

A browser-accessible POCKET-35B CPU demo is a separate hosted service; its availability and latency are not local-device measurements. No local-inference telemetry or website gate is required to obtain these public files.

POCKET family

Model	Role	Start here
POCKET-35B-GGUF	Full 35B PC builds; Q2_K recommended first	Model card
POCKET-26B-GGUF	Korean-tuned Gemma4 GGUF; 11 GB and 17 GB options	Model card
POCKET-Darwin-180B-GGUF	Text-only RSI reasoning; four-shard download	Model card

Explore the POCKET collection. For phone-oriented derivatives, see KR GGUF, KR MLX and EN GGUF.

More POCKET projects and research

Qwen3.8 Flash-Next GGUF is a separate parent-model package, distinct from the POCKET-Darwin-180B RSI-R3 weights. Image projects are also separate: POCKET-Image-Zimage, POCKET-Zimage-CPU and Image Studio.

Research collections: Darwin, Aether, VKAE.