SID-GPT 25M

A GPT model trained to generate Commodore 64 SID music by learning from legendary composers.

Model Description

SID-GPT learns to predict SID register states frame-by-frame, essentially learning the "language" of C64 chiptune music. Trained on 2,410 songs from HVSC, it produces output with recognizable musical structures: kick drums, PWM sweeps, basslines, and arpeggios.

Parameter	Value
Parameters	25.7M
Architecture	8 layers, 8 heads, 512 embedding
Block Size	1020 tokens (20 frames)
Effective Context	12 frames (0.24 sec)
Vocabulary	22 tokens
Validation Loss	0.207
Training Time	31 hours on M4 MacBook

Training Data

Source: HVSC (High Voltage SID Collection)
Size: 1GB of register dump sequences (2,410 SID files)
Composers: DRAX (530 songs), Laxity (287), Rob Hubbard (96), Jeroen Tel (176), Martin Galway (40), and 10 others

Files

File	Size	Description
`sid-gpt-xxxx.bin`	98 MB	Exported weights for Zig inference
`sid-gpt-xxxx.pt`	295 MB	PyTorch checkpoint (includes optimizer state)
`config.json`	1 KB	Model configuration

Usage

Zig Inference Engine (Recommended)

The native Zig engine runs at ~350-120 tok/s with SIMD and KV caching, depending on context window:

# Clone repository
git clone https://github.com/M64GitHub/SidGPT
cd SidGPT
zig build -Doptimize=ReleaseFast

# Download model
wget https://huggingface.co/M64/sid-gpt-25m/resolve/main/sid-gpt-1700.bin -P models/

# Generate and play
./zig-out/bin/sidgpt --model models/sid-gpt-1700.bin --frames 700 --temp 0.90 --seed 7391738265 --context 12 | ./zig-out/bin/sidgpt-play

# Or export to WAV
./zig-out/bin/sidgpt --model models/sid-gpt-1700.bin --frames 700 --temp 0.90 --seed 7391738265 --context 12 --output music.txt
./zig-out/bin/sidgpt-play music.txt --output-wav music.wav

Python Inference

cd training
python sample_sid.py --checkpoint path/to/sid-gpt-1700.pt --num_frames 700 --temperature 0.95

Generation Tips

Good seeds to try: 1337, 7391738264, 7391738265, 4829173650

Audio Samples

Generated outputs from this model:

Sample	Seed	Temp	Description
test.wav	7391738265	0.95	Melodic arps with bassline and kicks

Proof of Concept Status

Despite only 12 frames (0.24 sec) of context, the model learned real SID techniques:

Kick drums - Pulse wave frequency sweeps transitioning to noise
PWM sweeps - Pulse width modulation fades (Rob Hubbard signature)
Basslines - Melodic bass patterns with movement
Arpeggios - Fast note sequences typical of SID music
Leads - Fading-in lead voices

Limitations

Short context: 12 frames = no long-range song structure
Seed dependent: Quality varies significantly with random seed
No conditioning: Cannot specify style/artist (planned for v2)
Pattern matching: Learns techniques, not "composing"

Training Details

Loss progression:
  Iter 0:     2.88 (random)
  Iter 200:   0.96 (structure learned)
  Iter 700:   0.37 (musical patterns)
  Iter 1000:  0.27 (kick drums, PWM)
  Iter 2000:  0.21 (best checkpoint)

Training was stopped at iter 2000 when validation loss plateaued and train/val gap exceeded 30% (indicating overfitting).

Technical Details

Data Format

Each frame is 25 SID registers encoded as 50 hex characters + newline:

B0080005410A306011C0064108200016800D41082000B4031F
B0084005410A30601100074108200016C00D41082000B4031F
...
<end>

50 frames = 1 second of audio
Vocabulary: 0-9, A-F, <, >, d, e, n, \n (22 tokens)

Inference Optimizations

The Zig engine includes:

KV Cache: 50-100x speedup for autoregressive generation
SIMD: @Vector(8, f32) operations, 24x speedup
Sliding Window: Infinite generation beyond context length

Citation

@misc{sidgpt2026,
  author = {Mario Schallner},
  title = {SID-GPT: Transformer-based Commodore 64 Music Generation},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/M64/sid-gpt-25m}
}

Acknowledgments

Thanks to the legendary C64 composers whose work made this possible: Matt Gray, Jeroen Tel, Rob Hubbard, Martin Galway, DRAX, Laxity, and all contributors to HVSC.

Downloads last month: 25

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

M64
/

sid-gpt-25m