DEPLOY.md

Deploying MLX Model Explorer

The Space and dataset are created private, verified end to end, and only then made public.

Prerequisites: hf auth login as a member of mlx-community with write access, plus the project venv (uv pip install -r requirements.txt).

1. Verify locally

.venv/bin/python -m pytest
EXPLORER_SINK=local EXPLORER_FLUSH_SECONDS=20 .venv/bin/uvicorn app.main:app --port 7860
# walk through the UI; events land in .runtime/local_dataset/data/incoming/
docker build --platform linux/amd64 -t mlx-model-explorer . \
  && docker run --rm -p 7860:7860 -e EXPLORER_SINK=local mlx-model-explorer

2. Create private repos

.venv/bin/python scripts/deploy.py status
.venv/bin/python scripts/deploy.py create      # refuses if the Space exists; add --allow-existing-dataset if the dataset was created earlier

This creates mlx-community/mlx-model-explorer-data (private dataset, card uploaded) and codelion/mlx-model-explorer (private Docker Space with EXPLORER_SINK=hub, DATASET_REPO, EXPLORER_FLUSH_SECONDS=3600).

Why the Space isn't in mlx-community: Hugging Face now requires a Team or Enterprise plan for an organization to run Docker or Gradio Spaces on free CPU (402 Payment Required). The dataset can live in the org. Once an org admin has a plan or a hardware grant, deploy there with SPACE_REPO=mlx-community/mlx-model-explorer scripts/deploy.py create --allow-existing-dataset followed by upload.

3. Dataset write token

Create a fine-grained token at https://huggingface.co/settings/tokens with write permission on mlx-community/mlx-model-explorer-data only. Don't use a personal all-access token: anything in a Space secret is available to the running app.

HF_DATASET_WRITE_TOKEN=hf_xxx .venv/bin/python scripts/deploy.py secret

4. Test the ingestion path against the private dataset from this machine

EXPLORER_SINK=hub EXPLORER_DATA_PREFIX=data/_test EXPLORER_FLUSH_SECONDS=30 \
  HF_TOKEN=$(cat ~/.cache/huggingface/token) .venv/bin/uvicorn app.main:app --port 7860

5. Upload and test the private Space

.venv/bin/python scripts/deploy.py variable EXPLORER_DATA_PREFIX data/_test   # keep test events separate
.venv/bin/python scripts/deploy.py upload
.venv/bin/python scripts/deploy.py status                                     # wait for RUNNING

Open https://huggingface.co/spaces/codelion/mlx-model-explorer while logged in. Run the full flow, then confirm a shard appears under data/_test/ in the dataset (after the flush interval, or restart the Space to force a flush).

To submit a benchmark to the private Space, use its direct URL and a token:

python bench/mlx_explorer_bench.py --model mlx-community/Qwen3-0.6B-4bit \
  --submit https://huggingface.co/proxy/codelion-mlx-model-explorer.hf.space --hf-token $(cat ~/.cache/huggingface/token)

6. Go public (only after review)

.venv/bin/python scripts/deploy.py variable EXPLORER_DATA_PREFIX data/events
.venv/bin/python scripts/deploy.py clean-test-data
.venv/bin/python scripts/deploy.py publish --yes-make-public

publish refuses while data/_test/ still exists.

Optional: show the app on an mlx-community Static Space

Static Spaces are free for organizations. static_embed/ is a one-page wrapper that frames the running app full-screen. The app's CSP already allows https://huggingface.co and https://*.hf.space as parents; override with EXPLORER_FRAME_ANCESTORS if needed.

1. Rename or move the app Space first. Its *.hf.space URL changes with the name.
2. Make the app Space public. A private Space can't be framed for anonymous visitors.
3. Replace APP_URL in static_embed/index.html with the app's direct URL (currently https://huggingface.co/proxy/codelion-mlx-model-explorer.hf.space/), then:

hf repos create mlx-community/mlx-model-explorer --type space --space-sdk static --private
hf upload mlx-community/mlx-model-explorer static_embed . --repo-type space

Deployed as private mlx-community/mlx-model-explorer on 2026-09-14. The app renders inside the Hub page. Make the Static Space public at launch with hf repos settings mlx-community/mlx-model-explorer --type space --no-private (or from its Settings page).

Monthly compaction (run from any machine with write access)

Early each month, merge the previous month's hourly shards into one file:

.venv/bin/python scripts/compact.py            # dry run: shows shards, rows before/after
.venv/bin/python scripts/compact.py --apply    # one commit: add data/events/YYYY-MM.parquet, delete the month's data/incoming shards

It keeps the newest session row per visit, never edits row values, refuses the current month, and is safe to re-run (late shards are merged into the existing monthly file).

Updating

Change code, run the tests, docker build locally, then scripts/deploy.py upload -m "what changed". Events are append-only, so redeploys never touch existing data. On shutdown the app flushes buffered events, and anything that fails to upload is retried from the on-disk spool.

Kill switch

scripts/deploy.py variable EXPLORER_SINK off stops all collection. The UI keeps working and the event endpoint accepts nothing.