SpeakerKit Pro (Core ML)

Pre-compiled Core ML package of state-of-the-art on-device Speaker Diarization models (including Sortformer V2.1 and PyAnnote.audio v3-Pro), optimized for Apple Silicon (Apple Neural Engine, GPU, and CPU) with W8A16 quantization for real-time multi-speaker recognition and segmentation.

🌟 Key Highlights

Dual State-of-the-Art Architectures:
- Sortformer V2.1 (sortformer/v2-1/384_94MB): End-to-end Conformer architecture supporting simultaneous multi-speaker segmentation (up to 4 speakers) in a single unified pass.
- PyAnnote.audio v3-Pro (speaker_segmenter & speaker_embedder & speaker_clusterer): Modular 3-stage diarization (VAD/Segmentation + 256-dim Speaker Embeddings + PLDA Clustering).
Extreme Low Latency (Apple M3 Measured):
- Sortformer V2.1: 42.67 ms for 30.7 seconds of audio (RTF: 0.0014x, ~720x real-time speed).
- PyAnnote v3-Pro: 94.72 ms for 30.0 seconds of audio (RTF: 0.0032x, ~315x real-time speed).
ANE & GPU Optimized: Quantized with W8A16 / Palettized weights for minimal memory footprint and zero thermal throttling on iOS / macOS.

📊 Benchmark Results (Apple M3 / macOS 26.5)

Tested with MLModelConfiguration.computeUnits = .all on Apple M3 (8GB RAM):

1. Sortformer V2.1 Pipeline (30.7s Audio / 491,520 Samples)

Module	Load Time (ms)	Avg Latency (ms)	Min / Max (ms)	Description
`MelSpectrogram.mlmodelc`	160.90 ms	4.54 ms	4.13 / 6.00 ms	Mel spectrogram feature extraction
`AudioConformerPreEncoder.mlmodelc`	112.15 ms	2.92 ms	2.83 / 3.15 ms	Conformer time-subsampling
`SortformerFullEncoder.mlmodelc`	9,387.90 ms	35.21 ms	34.71 / 37.52 ms	4-speaker simultaneous prediction
Total Sortformer Pipeline	—	42.67 ms	—	RTF: 0.0014x (~720x speed)

2. PyAnnote v3-Pro Pipeline (30.0s Audio / 480,000 Samples)

Module	Load Time (ms)	Avg Latency (ms)	Min / Max (ms)	Description
`SpeakerSegmenter.mlmodelc` (W8A16)	329.72 ms	71.73 ms	69.77 / 73.56 ms	VAD & Speaker boundary detection
`SpeakerEmbedderPreprocessor.mlmodelc`	56.71 ms	3.11 ms	2.97 / 3.19 ms	Acoustic preprocessor
`SpeakerEmbedder.mlmodelc` (W8A16)	4,316.98 ms	19.83 ms	18.02 / 28.43 ms	256-dimensional embedding vector
`PldaProjector.mlmodelc` (v4)	36.27 ms	0.05 ms	0.05 / 0.05 ms	PLDA clustering projection
Total PyAnnote Pipeline	—	94.72 ms	—	RTF: 0.0032x (~315x speed)

📁 Repository Structure

speakerkit-pro/
├── sortformer/
│   └── v2-1/
│       └── 384_94MB/
│           ├── AudioConformerPreEncoder.mlmodelc
│           ├── MelSpectrogram.mlmodelc
│           └── SortformerFullEncoder.mlmodelc
├── speaker_segmenter/
│   ├── pyannote-v3/ (W32A32, W8A16, W8A32)
│   └── pyannote-v3-pro/ (W32A32, W8A16)
│       └── SpeakerSegmenter.mlmodelc
├── speaker_embedder/
│   ├── pyannote-v3/ (W16A16, W6A16, W8A16)
│   └── pyannote-v3-pro/ (W16A16, W8A16)
│       ├── SpeakerEmbedder.mlmodelc
│       └── SpeakerEmbedderPreprocessor.mlmodelc
└── speaker_clusterer/
    └── pyannote-v4/
        └── W32A32/
            └── PldaProjector.mlmodelc

💻 Swift Integration Example

import CoreML

let config = MLModelConfiguration()
config.computeUnits = .all

// Load Sortformer End-to-End Diarization
let melModel = try MLModel(contentsOf: sortMelURL, configuration: config)
let preEncModel = try MLModel(contentsOf: sortPreEncURL, configuration: config)
let sortformer = try MLModel(contentsOf: sortFullEncURL, configuration: config)

// Load PyAnnote-v3-Pro Segmentation & Embedder
let segmenter = try MLModel(contentsOf: segmenterURL, configuration: config)
let embedderPre = try MLModel(contentsOf: embedderPreURL, configuration: config)
let embedder = try MLModel(contentsOf: embedderURL, configuration: config)
let plda = try MLModel(contentsOf: pldaURL, configuration: config)

📜 License & Acknowledgements

Original Architectures: Based on PyAnnote.audio (Hervé Bredin et al.) and Sortformer diarization.
Core ML Conversion & Optimization: Optimized by Argmax, Inc. (Argmax Pro SDK).

Downloads last month: -; Downloads are not tracked for this model. How to track