Instructions to use Open4bits/Qwen3-14B-Base-mlx-fp16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Open4bits/Qwen3-14B-Base-mlx-fp16 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Open4bits/Qwen3-14B-Base-mlx-fp16")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Open4bits/Qwen3-14B-Base-mlx-fp16")
model = AutoModelForCausalLM.from_pretrained("Open4bits/Qwen3-14B-Base-mlx-fp16")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

MLX

How to use Open4bits/Qwen3-14B-Base-mlx-fp16 with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("Open4bits/Qwen3-14B-Base-mlx-fp16")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps
LM Studio

vLLM

How to use Open4bits/Qwen3-14B-Base-mlx-fp16 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Open4bits/Qwen3-14B-Base-mlx-fp16"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Open4bits/Qwen3-14B-Base-mlx-fp16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Open4bits/Qwen3-14B-Base-mlx-fp16

SGLang

How to use Open4bits/Qwen3-14B-Base-mlx-fp16 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Open4bits/Qwen3-14B-Base-mlx-fp16" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Open4bits/Qwen3-14B-Base-mlx-fp16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Open4bits/Qwen3-14B-Base-mlx-fp16" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Open4bits/Qwen3-14B-Base-mlx-fp16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Pi new

How to use Open4bits/Qwen3-14B-Base-mlx-fp16 with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Open4bits/Qwen3-14B-Base-mlx-fp16"

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Open4bits/Qwen3-14B-Base-mlx-fp16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use Open4bits/Qwen3-14B-Base-mlx-fp16 with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Open4bits/Qwen3-14B-Base-mlx-fp16"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Open4bits/Qwen3-14B-Base-mlx-fp16

Run Hermes

hermes

MLX LM

How to use Open4bits/Qwen3-14B-Base-mlx-fp16 with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "Open4bits/Qwen3-14B-Base-mlx-fp16"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "Open4bits/Qwen3-14B-Base-mlx-fp16"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "Open4bits/Qwen3-14B-Base-mlx-fp16",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Docker Model Runner
How to use Open4bits/Qwen3-14B-Base-mlx-fp16 with Docker Model Runner:
```
docker model run hf.co/Open4bits/Qwen3-14B-Base-mlx-fp16
```

Here’s a professional GitHub-ready README.md for Open4bits/Qwen3-14B-Base-MLX-FP16:

Open4bits / Qwen3-14B-Base-MLX-FP16

This repository provides the Qwen3-14B Base model converted to MLX format with FP16 precision, published by Open4bits to enable efficient high-performance inference with reduced memory usage and broad hardware compatibility.

The underlying Qwen3-14B model and architecture are developed and owned by the original creators. This repository contains an FP16 precision MLX conversion of the original model weights.

Open4bits has started supporting MLX models to broaden compatibility with emerging quantization formats and efficient runtimes, allowing improved performance on a range of platforms.

Model Overview

Qwen3-14B Base is a 14-billion parameter transformer-based language model designed for strong general understanding, reasoning, and instruction following. This release uses FP16 precision in MLX format, enabling efficient inference with balanced speed and quality.

Model Details

Base Model: Qwen3-14B
Precision: FP16 (float16)
Format: MLX
Task: Text generation, instruction following
Weight tying: Preserved
Compatibility: MLX-enabled inference engines and runtimes

The FP16 format provides improved performance and reduced memory consumption compared to full FP32 precision while retaining high generation quality.

Intended Use

This model is intended for:

High-performance text generation and conversational applications
CPU-based or accelerator-supported deployments
Research, experimentation, and prototyping
Offline or self-hosted AI systems

Limitations

Lower precision compared to non-quantized models
Output quality depends on prompt design and inference parameters
Not optimized for highly specialized domain-specific tasks without further fine-tuning

License

This model follows the Apache 2.0 of the base Qwen3-14B model. Users must comply with the licensing conditions defined by the original model creators.

Support

If you find this model useful, please consider supporting the project. Your support helps Open4bits continue releasing and maintaining high-quality efficient models for the community.

Downloads last month: 10

Safetensors

Model size

15B params

Tensor type

F16

MLX

Hardware compatibility

Quantized

Model tree for Open4bits/Qwen3-14B-Base-mlx-fp16

Base model

Qwen/Qwen3-14B-Base

Finetuned

(70)

this model

Collection including Open4bits/Qwen3-14B-Base-mlx-fp16

MLX

Collection

23 items • Updated 7 days ago • 1