# MiniMax Music 3 `dav.pth` investigation

## Conclusion

The released `dav.pth` does **not** contain the native Music 3 waveform-to-RVQ tokenizer. It contains a continuous
Flow-VAE analysis/synthesis checkpoint: a waveform convolutional `encoder`, Gaussian posterior heads (`mean_proj` and
`logs_proj`), continuous `flow` transforms, and a waveform `decoder`. There is no RVQ/VQ quantizer and there are no
acoustic codebook embeddings. The `encoder.*` name is therefore real but is not the discrete encoder required by the
proposed continuation path.

The safe CPU-only inspection of release revision `fbdf52fbaaca799592917417eb05f1899f1255ec` found:

| Prefix | Tensor count | Representative evidence |
|---|---:|---|
| `encoder.*` | 119 | `encoder.block.0.weight_v [64,1,7]`; final trunk `encoder.block.6.weight_v [1024,1024,3]` |
| `mean_proj.*` | 2 | `mean_proj.weight [64,1024,1]` |
| `logs_proj.*` | 2 | `logs_proj.weight [64,1024,1]` |
| `dec_in_proj.*` | 2 | `dec_in_proj.weight [1024,64,1]` |
| `decoder.*` | 119 | first convolution `[1536,1024,7]`; final convolution `[1,96,7]` |
| `flow.*` | 304 | `flow.flows.0.pre.weight [256,32,1]` through `flow.flows.6.post.weight [32,256,1]` |

That is 548 tensors, all FP32, containing 122,904,034 values (491,616,136 tensor bytes). The file is 491,817,450
bytes and has SHA-256 `52adde6c6c52cca872f549449cd7b608677d1e69f4a98eee22da3403e63b56e8`. It is a flat
`OrderedDict`: there is no `generator` key, no `62000_generator` key or wrapper, and zero names matching
`quantizer`, `vq`, `rvq`, `codebook`, `codec`, or `tokenizer`. It also contains no serialized config or architecture
metadata. Tensor shapes plus released source are sufficient to instantiate the decoder, but cannot instantiate a
quantizer that has neither parameters nor configuration.

The reported community `62000_generator` wrapper is not present in this exact released file. `inspect_dav.py` still
recursively supports `generator`, `62000_generator`, and `state_dict`-style wrappers so a different checkpoint can be
checked without assuming it has the release layout. Names, shapes, and generic config can flag a candidate weight set,
but the inspector deliberately keeps `native_discrete_tokenizer_complete` and `can_encode_native_music3_tokens` false
until an exact compatible executable architecture/API has been implemented and verified.

## The actual token and rendering paths

Music 3 uses eight discrete streams at 25 frames/s. They are language-model symbols used to produce continuous
conditioning hidden states; they are **not** DAV latent codes and are never passed directly to the DAV decoder.

```text
caption + lyrics + <|audio_start|>
                |
                v
Global Qwen LM: c0 in [0, 16383] (vocabulary token c0 + 151675)
                |
                v
Local RVQ-depth LM: c1..c7, each in [0, 1023]
                |
                +--> frame token row [c0,c1,...,c7]
                |
                +--> global hidden + 7 depth hiddens = 8 * 4096
                     frame_hiddens [1, frames, 32768]
                                      |
                                      v
                         condition projection + flow-matching DiT
                                      |
                                      v
                         continuous Flow-VAE latent [B,128,T]
                                      |
                                      v
                         DAV dec_in_proj + decoder -> waveform
```

The token trajectory convention used by the provided comparator is `[frames, 8]`; `[8, frames]` is accepted only
when explicitly declared as `codebooks_first`. No heuristic transposition is performed.

The per-frame autoregressive details are:

1. The global LM samples only `<|audio_end|>` or the 16,384 c0 vocabulary IDs beginning at offset 151675.
2. The local depth model begins with the projected global hidden and the projected c0 embedding, then samples c1
   through c7 autoregressively. It has seven 1,024-way output heads.
3. For feedback to the global LM, c0 uses the global token embedding at `c0 + 151675`. Each residual code uses the
   Qwen checkpoint's `model.audio_extra_embedding.weight` in seven disjoint 1,024-row bands. Those seven embeddings
   are summed with c0 and scaled by `1/sqrt(8)`. This is an **LM embedding table**, not the absent waveform
   quantizer's acoustic centroids.
4. Codebooks are frame-aligned (one row contains all eight codes); there is no delayed-codebook layout in this path.
   Conditional and classifier-free-unconditional rows are paired during sampling. The first decode immediately after
   `<|audio_start|>` primes the feedback loop and is not emitted as an acoustic hidden frame. Each subsequent emitted
   frame concatenates one 4096-wide global hidden with seven 4096-wide local hiddens.
5. The 25 Hz rate is explicit in Diffusers and also follows `24000 / 960` in the condition encoder configuration.

Special IDs are fixed by the music tokenizer:

| Token | ID |
|---|---:|
| `<|im_start|>` | 151644 |
| `<|im_end|>` | 151645 |
| `<|audio_cfg|>` | 151654 |
| `<|audio_start|>` | 151669 |
| `<|audio_end|>` | 151670 |
| `<|caption_start|>` | 151671 |
| `<|caption_end|>` | 151672 |
| `<|lyrics_start|>` | 151673 |
| `<|lyrics_end|>` | 151674 |
| first c0/audio-code vocabulary ID | 151675 |

The Qwen shards contain the global audio-symbol embeddings plus the local autoregressive depth decoder
(`model.audio_extra_embedding.*` and `model.audio_decoder.*`). They contain no waveform encoder, nearest-neighbor
quantizer, RVQ codebooks, or equivalent audio-tokenizer module. A model that can *predict* token IDs from text is not
therefore able to infer those IDs from a waveform.

## Successful official H100 generation capture

A pinned end-to-end Diffusers run succeeded on the remote NVIDIA H100 using model revision
`fbdf52fbaaca799592917417eb05f1899f1255ec`, Diffusers revision
`90b4e34e79a86ec5e7f2437634fe95ecd2108796`, CUDA with bfloat16 weights, and seed 7. The exact request used prompt
`Instrumental French house, 126 BPM, E minor, filtered disco loop, punchy kick and warm bass.`, lyrics
`[instrumental]`, requested duration 1.0 seconds, and 30 inference steps. It made 26 calls to the official
`_generate_depth_codes`: one priming call followed by 25 emitted frames. After skipping only that priming row,
`music3_internal_tokens.npy` is a genuine internal `[25,8]` trajectory at 25 Hz.

The observed inclusive per-codebook ranges were:

| Codebook | Minimum | Maximum | Legal range |
|---|---:|---:|---:|
| c0 | 1012 | 16163 | 0..16383 |
| c1 | 95 | 984 | 0..1023 |
| c2 | 25 | 1005 | 0..1023 |
| c3 | 42 | 950 | 0..1023 |
| c4 | 83 | 941 | 0..1023 |
| c5 | 2 | 967 | 0..1023 |
| c6 | 3 | 984 | 0..1023 |
| c7 | 67 | 957 | 0..1023 |

The corresponding official renderer output, `music3_generated_1s.wav`, is 44.1 kHz stereo with 44,032 samples per
channel (0.9984580499 seconds). The reusable `capture_generated_tokens.py` utility reproduces this observation by
temporarily monkeypatching the official depth-code function, validating its paired `[2,8]` result, restoring the
function after generation, and saving only calls after the priming row. Before model loading it resolves the requested
Hub revision to a concrete snapshot commit, verifies the exact Diffusers commit, rejects dirty tracked files in a
source checkout, and hashes both the relevant Diffusers source file and the capture script. It writes temporary WAV,
NPY, and JSON files, atomically replaces the two data artifacts, and replaces metadata last. The metadata records the
runtime and request plus SHA-256 digests of the final WAV and NPY, so an interrupted mixed-generation bundle cannot
validate. It records `"wav_reencoding_performed": false` because this is an internal generation capture, not WAV
analysis.

For the final canonical rerun, the WAV SHA-256 is
`e52c8884acdfbe9a71badac1ef410a5e5b23512dfc3d56a6f76228adaae87e11`, the NPY SHA-256 is
`42de27f795e8a1f1b97ae85fd3a540d432a8b178fdc63f7037bf0eea08c61350`, and the metadata JSON SHA-256 is
`7232d979f21cdf769e96c047b11ee31bbe28a9ee30ec6cd6960455c44b71aa02`. Metadata embeds the first two digests
along with capture-script SHA-256 `e4b6ecb87652deaa987a722e7c281aec2196295a717066aec05ef195361c0f1f`
and relevant Diffusers encoder-source SHA-256
`247ca4492a531c6593485b9fb11503da8607d03315c2f8b313dc6ae44985336b`.

This run proves that the documented token shape, ranges, frame rate, priming alignment, and generation/rendering path
are executable. It does **not** supply the missing WAV-to-tokenizer path. Calling `encode_audio()` for the generated
WAV is still blocked by the released DAV capabilities, so there is no recovered token trajectory to compare.
Per-codebook WAV re-encoding agreement percentages therefore remain unavailable; comparing
`music3_internal_tokens.npy` with itself would only be a self-comparison and must not be reported as re-encoding
agreement.

## What conversion omits

The Diffusers converter maps only `dec_in_proj.*` and `decoder.*` from `dav.pth`. Its explicit conversion dictionary
and decoder loop leave 427 of the 548 tensors unmapped: all 119 `encoder.*`, both `mean_proj.*`, both `logs_proj.*`,
and all 304 `flow.*` tensors. SGLang-Omni makes the same decoder-only selection with prefixes
`("dec_in_proj.", "decoder.")`. That omission is real, but restoring those keys would expose only continuous DAV
analysis/posterior operations. It would not restore the absent RVQ quantizer/codebooks.

## Answers to the six requested questions

1. **Is the original audio encoder present?** A waveform analysis encoder is present as `encoder.*`, but the native
   discrete/RVQ tokenizer encoder is not. Its outputs feed continuous `mean_proj`/`logs_proj` and `flow` modules.
2. **Is the RVQ quantizer present?** No. There are zero matching quantizer/RVQ/VQ parameters and no acoustic
   codebooks in `dav.pth`. The Qwen local “RVQ depth decoder” predicts residual token IDs; it is not a waveform
   quantizer.
3. **Can arbitrary WAV audio be converted into valid Music 3 tokens?** No, not with the released weights. Calling
   `encode_audio()` raises `NativeTokenizerUnavailableError` before opening the WAV and never invents token IDs.
4. **Do re-encoded tokens match Music 3 internal tokens?** This experiment is blocked because no re-encoded tokens
   can be produced. Reporting per-codebook percentages would be fabricated. `native_token_compatibility.py` performs
   strict range/layout validation and exact per-codebook comparison once both real trajectories exist.
5. **Can those tokens seed continuation?** No token trajectory can be recovered, so continuation cannot be seeded.
   The current released inference entry points also start from text plus `<|audio_start|>` rather than accepting an
   existing token/LM-cache history. Even if a tokenizer is released later, continuation must replay each complete
   eight-code frame through the exact feedback embedding/depth-hidden path and preserve the priming alignment.
6. **What is missing?** The compatible waveform-to-code encoder/quantizer implementation, its RVQ acoustic codebook
   weights and config, plus an official existing-audio history/prefill interface. Nothing here justifies retraining or
   substituting a generic codec.

There is intentionally no `continue_audio.py`: without native tokens, such a CLI could only ignore the input audio,
mislabel continuous latents as tokens, or use an incompatible replacement codec. All three would violate the stated
conditioning contract. Likewise, a token round trip is impossible: Music 3 renders `frame_hiddens` through a
flow-matching model and DAV, not discrete codes directly through DAV.

## Reproducible checks

The inspection, comparison, blocked-encode, and test paths are CPU-only. Checkpoint loading uses
`torch.load(..., weights_only=True, map_location="cpu")`. The optional `capture_generated_tokens.py` command is the
explicit exception: it loads the pinned official generation runtime and uses the requested device (CUDA by default)
only to observe internally generated tokens. `pyproject.toml` requires `torch>=2.10.0`, after the vulnerable
versions listed in [GHSA-63cw-57p8-fm3p](https://github.com/advisories/GHSA-63cw-57p8-fm3p).

Install the optional generation-capture runtime with `pip install -e '.[capture]'`; that extra pins the audited
Diffusers Git revision and explicitly declares Accelerate, Hugging Face Hub, Transformers, and SoundFile in addition
to the base NumPy and Torch requirements.

```bash
python inspect_dav.py /path/to/dav.pth --sha256
python inspect_dav.py /path/to/dav.pth --json

# Both commands return nonzero and say BLOCKED before opening missing.wav.
python encode_audio.py missing.wav --dav /path/to/dav.pth --json
python round_trip_test.py missing.wav reconstructed.wav --dav /path/to/dav.pth --json

# Exact comparison, only when genuine internal and re-encoded trajectories exist.
python native_token_compatibility.py internal.npy recovered.npy \
  --reference-layout frames_first --recovered-layout frames_first --json

# Capture tokens from generation itself; this does not re-encode the saved WAV.
python capture_generated_tokens.py \
  --prompt "Instrumental French house, 126 BPM, E minor, filtered disco loop, punchy kick and warm bass." \
  --lyrics "[instrumental]" --audio-duration 1 --num-inference-steps 30 --seed 7 \
  --local-files-only \
  --output-wav /path/to/music3_generated_1s.wav \
  --output-tokens /path/to/music3_internal_tokens.npy \
  --output-metadata /path/to/music3_generation_capture.json

# Generated-sample integration is enabled only when all four are set.
MINIMAX_DAV_PATH=/path/to/dav.pth \
MINIMAX_GENERATED_WAV_PATH=/path/to/music3_generated_1s.wav \
MINIMAX_INTERNAL_TOKENS_PATH=/path/to/music3_internal_tokens.npy \
MINIMAX_GENERATION_CAPTURE_PATH=/path/to/music3_generation_capture.json \
pytest
```

Synthetic tests cover flat and nested/`62000_generator` checkpoints, capability verdicts, pre-WAV failure, explicit
token layouts, all codebook ranges, exact/mismatched agreement, unequal frame counts, capture ordering, metadata-last
replacement, source cleanliness, and partial artifact configuration. The environment-gated integration test checks
the real 548-tensor release, validates the full generated-capture schema, recomputes the WAV/NPY/source/script hashes,
and confirms the blocked encoder path. Impossible experiments--WAV token encoding, token round trip, re-encoding
agreement, and continuation--are explicitly reported as blocked rather than marked passed.

## Audited sources and revisions

- MiniMax release checkpoint: [MiniMaxAI/MiniMax-Music3 at revision
  `fbdf52fbaaca799592917417eb05f1899f1255ec`](https://huggingface.co/MiniMaxAI/MiniMax-Music3/tree/fbdf52fbaaca799592917417eb05f1899f1255ec).
- Hugging Face Diffusers revision `90b4e34e79a86ec5e7f2437634fe95ecd2108796`: the
  [Music 3 converter's DAV selection](https://github.com/huggingface/diffusers/blob/90b4e34e79a86ec5e7f2437634fe95ecd2108796/scripts/convert_minimax_music3_to_diffusers.py#L96-L127),
  [global/local token generation and feedback embedding](https://github.com/huggingface/diffusers/blob/90b4e34e79a86ec5e7f2437634fe95ecd2108796/src/diffusers/modular_pipelines/minimax_music3/encoders.py#L102-L148),
  [25 Hz autoregressive loop](https://github.com/huggingface/diffusers/blob/90b4e34e79a86ec5e7f2437634fe95ecd2108796/src/diffusers/modular_pipelines/minimax_music3/encoders.py#L287-L358), and
  [seven-head local depth decoder](https://github.com/huggingface/diffusers/blob/90b4e34e79a86ec5e7f2437634fe95ecd2108796/src/diffusers/models/transformers/minimax_music3_rvq_depth_decoder.py#L91-L139).
- SGLang-Omni revision `d0edde030334a1e54a6644ba3ab21eabc4a01f73`: exact
  [special IDs and audio-code offset](https://github.com/sgl-project/sglang-omni/blob/d0edde030334a1e54a6644ba3ab21eabc4a01f73/sglang_omni/models/minimax_music3/prompt.py#L9-L20),
  [eight-code feedback embedding](https://github.com/sgl-project/sglang-omni/blob/d0edde030334a1e54a6644ba3ab21eabc4a01f73/sglang_omni/models/minimax_music3/sglang_model.py#L82-L96),
  [per-frame code/hidden alignment](https://github.com/sgl-project/sglang-omni/blob/d0edde030334a1e54a6644ba3ab21eabc4a01f73/sglang_omni/models/minimax_music3/model_runner.py#L252-L289), and
  [decoder-only DAV selection](https://github.com/sgl-project/sglang-omni/blob/d0edde030334a1e54a6644ba3ab21eabc4a01f73/sglang_omni/models/minimax_music3/dav.py#L114-L154).