QwenASRKit-Pro (Qwen3-ASR 1.7B Core ML)

Pre-compiled Core ML package of Qwen3-ASR 1.7B, optimized for Apple Silicon (Apple Neural Engine, GPU, and CPU) with 8-bit Palettization and Speculative Decoding for ultra-low latency on-device automatic speech recognition.

---

🌟 Key Highlights

• Base Architecture: Qwen/Qwen3-ASR-1.7B (Alibaba Cloud Qwen Team)
• 52+ Languages & Dialects: Industry-leading SOTA recognition across English (regional accents), Chinese (dialects), Japanese, Korean, European, and Asian languages.
• Speculative Decoding Acceleration: Includes a 7-token speculative drafter (Speculator.mlmodelc) delivering up to ~566.8 tokens/sec decoding throughput on Apple Silicon.
• Ultra-low RTF: Audio encoder executes in 7.47 ms per 1-second audio chunk (RTF: 0.007x).
• ANE Tile Chunking: Text decoder is split into TextDecoderC0 and TextDecoderC1 to comply with Apple Neural Engine (ANE) memory and compiler constraints.

---

📊 Benchmark Results (Apple M3 / macOS 26.5)

Tested with MLModelConfiguration.computeUnits = .all on Apple M3 (8GB RAM):

Latency Breakdown

Pipeline Stage	Module	Avg Latency (ms)	Min / Max (ms)	Description
Audio Preprocessing	Melspectrogram.mlmodelc	0.14 ms	0.13 / 0.16 ms	16 kHz 1-sec audio chunk
Convolution Subsampling	AudioEncoderConvSubsampler.mlmodelc	0.72 ms	0.66 / 0.79 ms	CNN time-downsampler
1.7B Audio Transformer	AudioEncoderTransformer.mlmodelc	6.62 ms	6.31 / 7.36 ms	8-bit Palettized Transformer
Total Audio Encoder	Full Encoder Pipeline	7.47 ms	—	RTF: 0.007x (1s audio in 7.5ms)
Text Decoder (Layers 0)	TextDecoderC0.mlmodelc	0.04 ms	0.03 / 0.05 ms	ANE Layer Chunk 0
Text Decoder (Layers 1)	TextDecoderC1.mlmodelc	0.05 ms	0.04 / 0.08 ms	ANE Layer Chunk 1
Embedding & LM Head	TextDecoderEmbedHead.mlmodelc	3.85 ms	3.81 / 3.95 ms	Token Logits & Projection
Autoregressive Step	Full Decoder (1 Token)	3.94 ms	—	254.0 tokens/sec
Speculative Drafter	Speculator.mlmodelc (7 tokens)	8.41 ms	8.22 / 8.72 ms	7-token draft prediction
Speculative Pipeline	Speculative Step (7 Tokens)	~12.35 ms / 7tok	—	~566.8 tokens/sec

---

📁 Repository Structure

qwen3-asr/
├── audio_encoder/
│   └── 1.7b/
│       ├── AudioEncoderConvSubsampler.mlmodelc   # CNN Subsampler
│       ├── AudioEncoderTransformer.mlmodelc     # 1.7B 8-bit Audio Encoder
│       └── Melspectrogram.mlmodelc              # Mel Spectrogram Extractor
└── text_decoder/
    └── 1.7b/
        ├── Speculator.mlmodelc                  # 7-token Speculative Drafter
        ├── TextDecoderC0.mlmodelc               # ANE Split Chunk 0
        ├── TextDecoderC1.mlmodelc               # ANE Split Chunk 1
        ├── TextDecoderEmbedHead.mlmodelc        # Embedding & LM Head
        ├── decoder_manifest.json                # Speculative Decoder Manifest
        ├── tokenizer.json                       # Tokenizer definition
        ├── tokenizer_config.json
        └── vocab.json

---

💻 Swift Integration Example

import CoreML

let config = MLModelConfiguration()
config.computeUnits = .all

// Load Encoder
let melModel = try MLModel(contentsOf: melURL, configuration: config)
let convModel = try MLModel(contentsOf: convURL, configuration: config)
let encoderModel = try MLModel(contentsOf: encoderURL, configuration: config)

// Load Decoder & Speculator
let c0Model = try MLModel(contentsOf: decC0URL, configuration: config)
let c1Model = try MLModel(contentsOf: decC1URL, configuration: config)
let headModel = try MLModel(contentsOf: headURL, configuration: config)
let speculatorModel = try MLModel(contentsOf: speculatorURL, configuration: config)

---

📜 License & Acknowledgements

• Base Model: Developed by the Qwen Team at Alibaba Cloud (Apache 2.0 / Qwen License).
• Core ML Conversion & Optimization: Optimized by Argmax, Inc. (Argmax Pro SDK).