Instructions to use intel-ai/DeepSeek-OCR-2-MXFP8-CT-AutoRound with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use intel-ai/DeepSeek-OCR-2-MXFP8-CT-AutoRound with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="intel-ai/DeepSeek-OCR-2-MXFP8-CT-AutoRound", trust_remote_code=True)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("intel-ai/DeepSeek-OCR-2-MXFP8-CT-AutoRound", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use intel-ai/DeepSeek-OCR-2-MXFP8-CT-AutoRound with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "intel-ai/DeepSeek-OCR-2-MXFP8-CT-AutoRound"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "intel-ai/DeepSeek-OCR-2-MXFP8-CT-AutoRound",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/intel-ai/DeepSeek-OCR-2-MXFP8-CT-AutoRound

SGLang

How to use intel-ai/DeepSeek-OCR-2-MXFP8-CT-AutoRound with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "intel-ai/DeepSeek-OCR-2-MXFP8-CT-AutoRound" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "intel-ai/DeepSeek-OCR-2-MXFP8-CT-AutoRound",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "intel-ai/DeepSeek-OCR-2-MXFP8-CT-AutoRound" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "intel-ai/DeepSeek-OCR-2-MXFP8-CT-AutoRound",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use intel-ai/DeepSeek-OCR-2-MXFP8-CT-AutoRound with Docker Model Runner:
```
docker model run hf.co/intel-ai/DeepSeek-OCR-2-MXFP8-CT-AutoRound
```

DeepSeek-OCR-2-MXFP8-CT-AutoRound / quantization_config.json

INC4AI

Add files using upload-large-folder tool

92400e4 verified 28 days ago

Raw History Blame Contribute Delete

14.2 kB

{
  "config_groups": {
    "group_0": {
      "targets": [
        "Linear"
      ],
      "weights": {
        "num_bits": 8,
        "type": "float",
        "symmetric": true,
        "group_size": 32,
        "strategy": "group",
        "block_structure": null,
        "dynamic": false,
        "actorder": null,
        "scale_dtype": "torch.uint8",
        "zp_dtype": null,
        "observer": "memoryless_minmax",
        "observer_kwargs": {}
      },
      "input_activations": {
        "num_bits": 8,
        "type": "float",
        "symmetric": true,
        "group_size": 32,
        "strategy": "group",
        "block_structure": null,
        "dynamic": true,
        "actorder": null,
        "scale_dtype": "torch.uint8",
        "zp_dtype": null,
        "observer": null,
        "observer_kwargs": {}
      },
      "output_activations": null,
      "format": null
    }
  },
  "quant_method": "compressed-tensors",
  "kv_cache_scheme": null,
  "format": "mxfp8-quantized",
  "quantization_status": "compressed",
  "global_compression_ratio": null,
  "ignore": [
    "lm_head",
    "model.embed_tokens",
    "model.layers.1.mlp.gate",
    "model.layers.10.mlp.gate",
    "model.layers.11.mlp.gate",
    "model.layers.2.mlp.gate",
    "model.layers.3.mlp.gate",
    "model.layers.4.mlp.gate",
    "model.layers.5.mlp.gate",
    "model.layers.6.mlp.gate",
    "model.layers.7.mlp.gate",
    "model.layers.8.mlp.gate",
    "model.layers.9.mlp.gate",
    "model.projector.layers",
    "model.qwen2_model.model.model.layers.0.mlp.down_proj",
    "model.qwen2_model.model.model.layers.0.mlp.gate_proj",
    "model.qwen2_model.model.model.layers.0.mlp.up_proj",
    "model.qwen2_model.model.model.layers.0.self_attn.k_proj",
    "model.qwen2_model.model.model.layers.0.self_attn.o_proj",
    "model.qwen2_model.model.model.layers.0.self_attn.q_proj",
    "model.qwen2_model.model.model.layers.0.self_attn.v_proj",
    "model.qwen2_model.model.model.layers.1.mlp.down_proj",
    "model.qwen2_model.model.model.layers.1.mlp.gate_proj",
    "model.qwen2_model.model.model.layers.1.mlp.up_proj",
    "model.qwen2_model.model.model.layers.1.self_attn.k_proj",
    "model.qwen2_model.model.model.layers.1.self_attn.o_proj",
    "model.qwen2_model.model.model.layers.1.self_attn.q_proj",
    "model.qwen2_model.model.model.layers.1.self_attn.v_proj",
    "model.qwen2_model.model.model.layers.10.mlp.down_proj",
    "model.qwen2_model.model.model.layers.10.mlp.gate_proj",
    "model.qwen2_model.model.model.layers.10.mlp.up_proj",
    "model.qwen2_model.model.model.layers.10.self_attn.k_proj",
    "model.qwen2_model.model.model.layers.10.self_attn.o_proj",
    "model.qwen2_model.model.model.layers.10.self_attn.q_proj",
    "model.qwen2_model.model.model.layers.10.self_attn.v_proj",
    "model.qwen2_model.model.model.layers.11.mlp.down_proj",
    "model.qwen2_model.model.model.layers.11.mlp.gate_proj",
    "model.qwen2_model.model.model.layers.11.mlp.up_proj",
    "model.qwen2_model.model.model.layers.11.self_attn.k_proj",
    "model.qwen2_model.model.model.layers.11.self_attn.o_proj",
    "model.qwen2_model.model.model.layers.11.self_attn.q_proj",
    "model.qwen2_model.model.model.layers.11.self_attn.v_proj",
    "model.qwen2_model.model.model.layers.12.mlp.down_proj",
    "model.qwen2_model.model.model.layers.12.mlp.gate_proj",
    "model.qwen2_model.model.model.layers.12.mlp.up_proj",
    "model.qwen2_model.model.model.layers.12.self_attn.k_proj",
    "model.qwen2_model.model.model.layers.12.self_attn.o_proj",
    "model.qwen2_model.model.model.layers.12.self_attn.q_proj",
    "model.qwen2_model.model.model.layers.12.self_attn.v_proj",
    "model.qwen2_model.model.model.layers.13.mlp.down_proj",
    "model.qwen2_model.model.model.layers.13.mlp.gate_proj",
    "model.qwen2_model.model.model.layers.13.mlp.up_proj",
    "model.qwen2_model.model.model.layers.13.self_attn.k_proj",
    "model.qwen2_model.model.model.layers.13.self_attn.o_proj",
    "model.qwen2_model.model.model.layers.13.self_attn.q_proj",
    "model.qwen2_model.model.model.layers.13.self_attn.v_proj",
    "model.qwen2_model.model.model.layers.14.mlp.down_proj",
    "model.qwen2_model.model.model.layers.14.mlp.gate_proj",
    "model.qwen2_model.model.model.layers.14.mlp.up_proj",
    "model.qwen2_model.model.model.layers.14.self_attn.k_proj",
    "model.qwen2_model.model.model.layers.14.self_attn.o_proj",
    "model.qwen2_model.model.model.layers.14.self_attn.q_proj",
    "model.qwen2_model.model.model.layers.14.self_attn.v_proj",
    "model.qwen2_model.model.model.layers.15.mlp.down_proj",
    "model.qwen2_model.model.model.layers.15.mlp.gate_proj",
    "model.qwen2_model.model.model.layers.15.mlp.up_proj",
    "model.qwen2_model.model.model.layers.15.self_attn.k_proj",
    "model.qwen2_model.model.model.layers.15.self_attn.o_proj",
    "model.qwen2_model.model.model.layers.15.self_attn.q_proj",
    "model.qwen2_model.model.model.layers.15.self_attn.v_proj",
    "model.qwen2_model.model.model.layers.16.mlp.down_proj",
    "model.qwen2_model.model.model.layers.16.mlp.gate_proj",
    "model.qwen2_model.model.model.layers.16.mlp.up_proj",
    "model.qwen2_model.model.model.layers.16.self_attn.k_proj",
    "model.qwen2_model.model.model.layers.16.self_attn.o_proj",
    "model.qwen2_model.model.model.layers.16.self_attn.q_proj",
    "model.qwen2_model.model.model.layers.16.self_attn.v_proj",
    "model.qwen2_model.model.model.layers.17.mlp.down_proj",
    "model.qwen2_model.model.model.layers.17.mlp.gate_proj",
    "model.qwen2_model.model.model.layers.17.mlp.up_proj",
    "model.qwen2_model.model.model.layers.17.self_attn.k_proj",
    "model.qwen2_model.model.model.layers.17.self_attn.o_proj",
    "model.qwen2_model.model.model.layers.17.self_attn.q_proj",
    "model.qwen2_model.model.model.layers.17.self_attn.v_proj",
    "model.qwen2_model.model.model.layers.18.mlp.down_proj",
    "model.qwen2_model.model.model.layers.18.mlp.gate_proj",
    "model.qwen2_model.model.model.layers.18.mlp.up_proj",
    "model.qwen2_model.model.model.layers.18.self_attn.k_proj",
    "model.qwen2_model.model.model.layers.18.self_attn.o_proj",
    "model.qwen2_model.model.model.layers.18.self_attn.q_proj",
    "model.qwen2_model.model.model.layers.18.self_attn.v_proj",
    "model.qwen2_model.model.model.layers.19.mlp.down_proj",
    "model.qwen2_model.model.model.layers.19.mlp.gate_proj",
    "model.qwen2_model.model.model.layers.19.mlp.up_proj",
    "model.qwen2_model.model.model.layers.19.self_attn.k_proj",
    "model.qwen2_model.model.model.layers.19.self_attn.o_proj",
    "model.qwen2_model.model.model.layers.19.self_attn.q_proj",
    "model.qwen2_model.model.model.layers.19.self_attn.v_proj",
    "model.qwen2_model.model.model.layers.2.mlp.down_proj",
    "model.qwen2_model.model.model.layers.2.mlp.gate_proj",
    "model.qwen2_model.model.model.layers.2.mlp.up_proj",
    "model.qwen2_model.model.model.layers.2.self_attn.k_proj",
    "model.qwen2_model.model.model.layers.2.self_attn.o_proj",
    "model.qwen2_model.model.model.layers.2.self_attn.q_proj",
    "model.qwen2_model.model.model.layers.2.self_attn.v_proj",
    "model.qwen2_model.model.model.layers.20.mlp.down_proj",
    "model.qwen2_model.model.model.layers.20.mlp.gate_proj",
    "model.qwen2_model.model.model.layers.20.mlp.up_proj",
    "model.qwen2_model.model.model.layers.20.self_attn.k_proj",
    "model.qwen2_model.model.model.layers.20.self_attn.o_proj",
    "model.qwen2_model.model.model.layers.20.self_attn.q_proj",
    "model.qwen2_model.model.model.layers.20.self_attn.v_proj",
    "model.qwen2_model.model.model.layers.21.mlp.down_proj",
    "model.qwen2_model.model.model.layers.21.mlp.gate_proj",
    "model.qwen2_model.model.model.layers.21.mlp.up_proj",
    "model.qwen2_model.model.model.layers.21.self_attn.k_proj",
    "model.qwen2_model.model.model.layers.21.self_attn.o_proj",
    "model.qwen2_model.model.model.layers.21.self_attn.q_proj",
    "model.qwen2_model.model.model.layers.21.self_attn.v_proj",
    "model.qwen2_model.model.model.layers.22.mlp.down_proj",
    "model.qwen2_model.model.model.layers.22.mlp.gate_proj",
    "model.qwen2_model.model.model.layers.22.mlp.up_proj",
    "model.qwen2_model.model.model.layers.22.self_attn.k_proj",
    "model.qwen2_model.model.model.layers.22.self_attn.o_proj",
    "model.qwen2_model.model.model.layers.22.self_attn.q_proj",
    "model.qwen2_model.model.model.layers.22.self_attn.v_proj",
    "model.qwen2_model.model.model.layers.23.mlp.down_proj",
    "model.qwen2_model.model.model.layers.23.mlp.gate_proj",
    "model.qwen2_model.model.model.layers.23.mlp.up_proj",
    "model.qwen2_model.model.model.layers.23.self_attn.k_proj",
    "model.qwen2_model.model.model.layers.23.self_attn.o_proj",
    "model.qwen2_model.model.model.layers.23.self_attn.q_proj",
    "model.qwen2_model.model.model.layers.23.self_attn.v_proj",
    "model.qwen2_model.model.model.layers.3.mlp.down_proj",
    "model.qwen2_model.model.model.layers.3.mlp.gate_proj",
    "model.qwen2_model.model.model.layers.3.mlp.up_proj",
    "model.qwen2_model.model.model.layers.3.self_attn.k_proj",
    "model.qwen2_model.model.model.layers.3.self_attn.o_proj",
    "model.qwen2_model.model.model.layers.3.self_attn.q_proj",
    "model.qwen2_model.model.model.layers.3.self_attn.v_proj",
    "model.qwen2_model.model.model.layers.4.mlp.down_proj",
    "model.qwen2_model.model.model.layers.4.mlp.gate_proj",
    "model.qwen2_model.model.model.layers.4.mlp.up_proj",
    "model.qwen2_model.model.model.layers.4.self_attn.k_proj",
    "model.qwen2_model.model.model.layers.4.self_attn.o_proj",
    "model.qwen2_model.model.model.layers.4.self_attn.q_proj",
    "model.qwen2_model.model.model.layers.4.self_attn.v_proj",
    "model.qwen2_model.model.model.layers.5.mlp.down_proj",
    "model.qwen2_model.model.model.layers.5.mlp.gate_proj",
    "model.qwen2_model.model.model.layers.5.mlp.up_proj",
    "model.qwen2_model.model.model.layers.5.self_attn.k_proj",
    "model.qwen2_model.model.model.layers.5.self_attn.o_proj",
    "model.qwen2_model.model.model.layers.5.self_attn.q_proj",
    "model.qwen2_model.model.model.layers.5.self_attn.v_proj",
    "model.qwen2_model.model.model.layers.6.mlp.down_proj",
    "model.qwen2_model.model.model.layers.6.mlp.gate_proj",
    "model.qwen2_model.model.model.layers.6.mlp.up_proj",
    "model.qwen2_model.model.model.layers.6.self_attn.k_proj",
    "model.qwen2_model.model.model.layers.6.self_attn.o_proj",
    "model.qwen2_model.model.model.layers.6.self_attn.q_proj",
    "model.qwen2_model.model.model.layers.6.self_attn.v_proj",
    "model.qwen2_model.model.model.layers.7.mlp.down_proj",
    "model.qwen2_model.model.model.layers.7.mlp.gate_proj",
    "model.qwen2_model.model.model.layers.7.mlp.up_proj",
    "model.qwen2_model.model.model.layers.7.self_attn.k_proj",
    "model.qwen2_model.model.model.layers.7.self_attn.o_proj",
    "model.qwen2_model.model.model.layers.7.self_attn.q_proj",
    "model.qwen2_model.model.model.layers.7.self_attn.v_proj",
    "model.qwen2_model.model.model.layers.8.mlp.down_proj",
    "model.qwen2_model.model.model.layers.8.mlp.gate_proj",
    "model.qwen2_model.model.model.layers.8.mlp.up_proj",
    "model.qwen2_model.model.model.layers.8.self_attn.k_proj",
    "model.qwen2_model.model.model.layers.8.self_attn.o_proj",
    "model.qwen2_model.model.model.layers.8.self_attn.q_proj",
    "model.qwen2_model.model.model.layers.8.self_attn.v_proj",
    "model.qwen2_model.model.model.layers.9.mlp.down_proj",
    "model.qwen2_model.model.model.layers.9.mlp.gate_proj",
    "model.qwen2_model.model.model.layers.9.mlp.up_proj",
    "model.qwen2_model.model.model.layers.9.self_attn.k_proj",
    "model.qwen2_model.model.model.layers.9.self_attn.o_proj",
    "model.qwen2_model.model.model.layers.9.self_attn.q_proj",
    "model.qwen2_model.model.model.layers.9.self_attn.v_proj",
    "model.qwen2_model.query_1024",
    "model.qwen2_model.query_768",
    "model.sam_model.blocks.0.attn.proj",
    "model.sam_model.blocks.0.attn.qkv",
    "model.sam_model.blocks.0.mlp.lin1",
    "model.sam_model.blocks.0.mlp.lin2",
    "model.sam_model.blocks.1.attn.proj",
    "model.sam_model.blocks.1.attn.qkv",
    "model.sam_model.blocks.1.mlp.lin1",
    "model.sam_model.blocks.1.mlp.lin2",
    "model.sam_model.blocks.10.attn.proj",
    "model.sam_model.blocks.10.attn.qkv",
    "model.sam_model.blocks.10.mlp.lin1",
    "model.sam_model.blocks.10.mlp.lin2",
    "model.sam_model.blocks.11.attn.proj",
    "model.sam_model.blocks.11.attn.qkv",
    "model.sam_model.blocks.11.mlp.lin1",
    "model.sam_model.blocks.11.mlp.lin2",
    "model.sam_model.blocks.2.attn.proj",
    "model.sam_model.blocks.2.attn.qkv",
    "model.sam_model.blocks.2.mlp.lin1",
    "model.sam_model.blocks.2.mlp.lin2",
    "model.sam_model.blocks.3.attn.proj",
    "model.sam_model.blocks.3.attn.qkv",
    "model.sam_model.blocks.3.mlp.lin1",
    "model.sam_model.blocks.3.mlp.lin2",
    "model.sam_model.blocks.4.attn.proj",
    "model.sam_model.blocks.4.attn.qkv",
    "model.sam_model.blocks.4.mlp.lin1",
    "model.sam_model.blocks.4.mlp.lin2",
    "model.sam_model.blocks.5.attn.proj",
    "model.sam_model.blocks.5.attn.qkv",
    "model.sam_model.blocks.5.mlp.lin1",
    "model.sam_model.blocks.5.mlp.lin2",
    "model.sam_model.blocks.6.attn.proj",
    "model.sam_model.blocks.6.attn.qkv",
    "model.sam_model.blocks.6.mlp.lin1",
    "model.sam_model.blocks.6.mlp.lin2",
    "model.sam_model.blocks.7.attn.proj",
    "model.sam_model.blocks.7.attn.qkv",
    "model.sam_model.blocks.7.mlp.lin1",
    "model.sam_model.blocks.7.mlp.lin2",
    "model.sam_model.blocks.8.attn.proj",
    "model.sam_model.blocks.8.attn.qkv",
    "model.sam_model.blocks.8.mlp.lin1",
    "model.sam_model.blocks.8.mlp.lin2",
    "model.sam_model.blocks.9.attn.proj",
    "model.sam_model.blocks.9.attn.qkv",
    "model.sam_model.blocks.9.mlp.lin1",
    "model.sam_model.blocks.9.mlp.lin2",
    "model.sam_model.neck.0",
    "model.sam_model.neck.2",
    "model.sam_model.net_2",
    "model.sam_model.net_3",
    "model.sam_model.patch_embed.proj"
  ],
  "provider": "auto-round"
}

	{
	"config_groups": {
	"group_0": {
	"targets": [
	"Linear"
	],
	"weights": {
	"num_bits": 8,
	"type": "float",
	"symmetric": true,
	"group_size": 32,
	"strategy": "group",
	"block_structure": null,
	"dynamic": false,
	"actorder": null,
	"scale_dtype": "torch.uint8",
	"zp_dtype": null,
	"observer": "memoryless_minmax",
	"observer_kwargs": {}
	},
	"input_activations": {
	"num_bits": 8,
	"type": "float",
	"symmetric": true,
	"group_size": 32,
	"strategy": "group",
	"block_structure": null,
	"dynamic": true,
	"actorder": null,
	"scale_dtype": "torch.uint8",
	"zp_dtype": null,
	"observer": null,
	"observer_kwargs": {}
	},
	"output_activations": null,
	"format": null
	}
	},
	"quant_method": "compressed-tensors",
	"kv_cache_scheme": null,
	"format": "mxfp8-quantized",
	"quantization_status": "compressed",
	"global_compression_ratio": null,
	"ignore": [
	"lm_head",
	"model.embed_tokens",
	"model.layers.1.mlp.gate",
	"model.layers.10.mlp.gate",
	"model.layers.11.mlp.gate",
	"model.layers.2.mlp.gate",
	"model.layers.3.mlp.gate",
	"model.layers.4.mlp.gate",
	"model.layers.5.mlp.gate",
	"model.layers.6.mlp.gate",
	"model.layers.7.mlp.gate",
	"model.layers.8.mlp.gate",
	"model.layers.9.mlp.gate",
	"model.projector.layers",
	"model.qwen2_model.model.model.layers.0.mlp.down_proj",
	"model.qwen2_model.model.model.layers.0.mlp.gate_proj",
	"model.qwen2_model.model.model.layers.0.mlp.up_proj",
	"model.qwen2_model.model.model.layers.0.self_attn.k_proj",
	"model.qwen2_model.model.model.layers.0.self_attn.o_proj",
	"model.qwen2_model.model.model.layers.0.self_attn.q_proj",
	"model.qwen2_model.model.model.layers.0.self_attn.v_proj",
	"model.qwen2_model.model.model.layers.1.mlp.down_proj",
	"model.qwen2_model.model.model.layers.1.mlp.gate_proj",
	"model.qwen2_model.model.model.layers.1.mlp.up_proj",
	"model.qwen2_model.model.model.layers.1.self_attn.k_proj",
	"model.qwen2_model.model.model.layers.1.self_attn.o_proj",
	"model.qwen2_model.model.model.layers.1.self_attn.q_proj",
	"model.qwen2_model.model.model.layers.1.self_attn.v_proj",
	"model.qwen2_model.model.model.layers.10.mlp.down_proj",
	"model.qwen2_model.model.model.layers.10.mlp.gate_proj",
	"model.qwen2_model.model.model.layers.10.mlp.up_proj",
	"model.qwen2_model.model.model.layers.10.self_attn.k_proj",
	"model.qwen2_model.model.model.layers.10.self_attn.o_proj",
	"model.qwen2_model.model.model.layers.10.self_attn.q_proj",
	"model.qwen2_model.model.model.layers.10.self_attn.v_proj",
	"model.qwen2_model.model.model.layers.11.mlp.down_proj",
	"model.qwen2_model.model.model.layers.11.mlp.gate_proj",
	"model.qwen2_model.model.model.layers.11.mlp.up_proj",
	"model.qwen2_model.model.model.layers.11.self_attn.k_proj",
	"model.qwen2_model.model.model.layers.11.self_attn.o_proj",
	"model.qwen2_model.model.model.layers.11.self_attn.q_proj",
	"model.qwen2_model.model.model.layers.11.self_attn.v_proj",
	"model.qwen2_model.model.model.layers.12.mlp.down_proj",
	"model.qwen2_model.model.model.layers.12.mlp.gate_proj",
	"model.qwen2_model.model.model.layers.12.mlp.up_proj",
	"model.qwen2_model.model.model.layers.12.self_attn.k_proj",
	"model.qwen2_model.model.model.layers.12.self_attn.o_proj",
	"model.qwen2_model.model.model.layers.12.self_attn.q_proj",
	"model.qwen2_model.model.model.layers.12.self_attn.v_proj",
	"model.qwen2_model.model.model.layers.13.mlp.down_proj",
	"model.qwen2_model.model.model.layers.13.mlp.gate_proj",
	"model.qwen2_model.model.model.layers.13.mlp.up_proj",
	"model.qwen2_model.model.model.layers.13.self_attn.k_proj",
	"model.qwen2_model.model.model.layers.13.self_attn.o_proj",
	"model.qwen2_model.model.model.layers.13.self_attn.q_proj",
	"model.qwen2_model.model.model.layers.13.self_attn.v_proj",
	"model.qwen2_model.model.model.layers.14.mlp.down_proj",
	"model.qwen2_model.model.model.layers.14.mlp.gate_proj",
	"model.qwen2_model.model.model.layers.14.mlp.up_proj",
	"model.qwen2_model.model.model.layers.14.self_attn.k_proj",
	"model.qwen2_model.model.model.layers.14.self_attn.o_proj",
	"model.qwen2_model.model.model.layers.14.self_attn.q_proj",
	"model.qwen2_model.model.model.layers.14.self_attn.v_proj",
	"model.qwen2_model.model.model.layers.15.mlp.down_proj",
	"model.qwen2_model.model.model.layers.15.mlp.gate_proj",
	"model.qwen2_model.model.model.layers.15.mlp.up_proj",
	"model.qwen2_model.model.model.layers.15.self_attn.k_proj",
	"model.qwen2_model.model.model.layers.15.self_attn.o_proj",
	"model.qwen2_model.model.model.layers.15.self_attn.q_proj",
	"model.qwen2_model.model.model.layers.15.self_attn.v_proj",
	"model.qwen2_model.model.model.layers.16.mlp.down_proj",
	"model.qwen2_model.model.model.layers.16.mlp.gate_proj",
	"model.qwen2_model.model.model.layers.16.mlp.up_proj",
	"model.qwen2_model.model.model.layers.16.self_attn.k_proj",
	"model.qwen2_model.model.model.layers.16.self_attn.o_proj",
	"model.qwen2_model.model.model.layers.16.self_attn.q_proj",
	"model.qwen2_model.model.model.layers.16.self_attn.v_proj",
	"model.qwen2_model.model.model.layers.17.mlp.down_proj",
	"model.qwen2_model.model.model.layers.17.mlp.gate_proj",
	"model.qwen2_model.model.model.layers.17.mlp.up_proj",
	"model.qwen2_model.model.model.layers.17.self_attn.k_proj",
	"model.qwen2_model.model.model.layers.17.self_attn.o_proj",
	"model.qwen2_model.model.model.layers.17.self_attn.q_proj",
	"model.qwen2_model.model.model.layers.17.self_attn.v_proj",
	"model.qwen2_model.model.model.layers.18.mlp.down_proj",
	"model.qwen2_model.model.model.layers.18.mlp.gate_proj",
	"model.qwen2_model.model.model.layers.18.mlp.up_proj",
	"model.qwen2_model.model.model.layers.18.self_attn.k_proj",
	"model.qwen2_model.model.model.layers.18.self_attn.o_proj",
	"model.qwen2_model.model.model.layers.18.self_attn.q_proj",
	"model.qwen2_model.model.model.layers.18.self_attn.v_proj",
	"model.qwen2_model.model.model.layers.19.mlp.down_proj",
	"model.qwen2_model.model.model.layers.19.mlp.gate_proj",
	"model.qwen2_model.model.model.layers.19.mlp.up_proj",
	"model.qwen2_model.model.model.layers.19.self_attn.k_proj",
	"model.qwen2_model.model.model.layers.19.self_attn.o_proj",
	"model.qwen2_model.model.model.layers.19.self_attn.q_proj",
	"model.qwen2_model.model.model.layers.19.self_attn.v_proj",
	"model.qwen2_model.model.model.layers.2.mlp.down_proj",
	"model.qwen2_model.model.model.layers.2.mlp.gate_proj",
	"model.qwen2_model.model.model.layers.2.mlp.up_proj",
	"model.qwen2_model.model.model.layers.2.self_attn.k_proj",
	"model.qwen2_model.model.model.layers.2.self_attn.o_proj",
	"model.qwen2_model.model.model.layers.2.self_attn.q_proj",
	"model.qwen2_model.model.model.layers.2.self_attn.v_proj",
	"model.qwen2_model.model.model.layers.20.mlp.down_proj",
	"model.qwen2_model.model.model.layers.20.mlp.gate_proj",
	"model.qwen2_model.model.model.layers.20.mlp.up_proj",
	"model.qwen2_model.model.model.layers.20.self_attn.k_proj",
	"model.qwen2_model.model.model.layers.20.self_attn.o_proj",
	"model.qwen2_model.model.model.layers.20.self_attn.q_proj",
	"model.qwen2_model.model.model.layers.20.self_attn.v_proj",
	"model.qwen2_model.model.model.layers.21.mlp.down_proj",
	"model.qwen2_model.model.model.layers.21.mlp.gate_proj",
	"model.qwen2_model.model.model.layers.21.mlp.up_proj",
	"model.qwen2_model.model.model.layers.21.self_attn.k_proj",
	"model.qwen2_model.model.model.layers.21.self_attn.o_proj",
	"model.qwen2_model.model.model.layers.21.self_attn.q_proj",
	"model.qwen2_model.model.model.layers.21.self_attn.v_proj",
	"model.qwen2_model.model.model.layers.22.mlp.down_proj",
	"model.qwen2_model.model.model.layers.22.mlp.gate_proj",
	"model.qwen2_model.model.model.layers.22.mlp.up_proj",
	"model.qwen2_model.model.model.layers.22.self_attn.k_proj",
	"model.qwen2_model.model.model.layers.22.self_attn.o_proj",
	"model.qwen2_model.model.model.layers.22.self_attn.q_proj",
	"model.qwen2_model.model.model.layers.22.self_attn.v_proj",
	"model.qwen2_model.model.model.layers.23.mlp.down_proj",
	"model.qwen2_model.model.model.layers.23.mlp.gate_proj",
	"model.qwen2_model.model.model.layers.23.mlp.up_proj",
	"model.qwen2_model.model.model.layers.23.self_attn.k_proj",
	"model.qwen2_model.model.model.layers.23.self_attn.o_proj",
	"model.qwen2_model.model.model.layers.23.self_attn.q_proj",
	"model.qwen2_model.model.model.layers.23.self_attn.v_proj",
	"model.qwen2_model.model.model.layers.3.mlp.down_proj",
	"model.qwen2_model.model.model.layers.3.mlp.gate_proj",
	"model.qwen2_model.model.model.layers.3.mlp.up_proj",
	"model.qwen2_model.model.model.layers.3.self_attn.k_proj",
	"model.qwen2_model.model.model.layers.3.self_attn.o_proj",
	"model.qwen2_model.model.model.layers.3.self_attn.q_proj",
	"model.qwen2_model.model.model.layers.3.self_attn.v_proj",
	"model.qwen2_model.model.model.layers.4.mlp.down_proj",
	"model.qwen2_model.model.model.layers.4.mlp.gate_proj",
	"model.qwen2_model.model.model.layers.4.mlp.up_proj",
	"model.qwen2_model.model.model.layers.4.self_attn.k_proj",
	"model.qwen2_model.model.model.layers.4.self_attn.o_proj",
	"model.qwen2_model.model.model.layers.4.self_attn.q_proj",
	"model.qwen2_model.model.model.layers.4.self_attn.v_proj",
	"model.qwen2_model.model.model.layers.5.mlp.down_proj",
	"model.qwen2_model.model.model.layers.5.mlp.gate_proj",
	"model.qwen2_model.model.model.layers.5.mlp.up_proj",
	"model.qwen2_model.model.model.layers.5.self_attn.k_proj",
	"model.qwen2_model.model.model.layers.5.self_attn.o_proj",
	"model.qwen2_model.model.model.layers.5.self_attn.q_proj",
	"model.qwen2_model.model.model.layers.5.self_attn.v_proj",
	"model.qwen2_model.model.model.layers.6.mlp.down_proj",
	"model.qwen2_model.model.model.layers.6.mlp.gate_proj",
	"model.qwen2_model.model.model.layers.6.mlp.up_proj",
	"model.qwen2_model.model.model.layers.6.self_attn.k_proj",
	"model.qwen2_model.model.model.layers.6.self_attn.o_proj",
	"model.qwen2_model.model.model.layers.6.self_attn.q_proj",
	"model.qwen2_model.model.model.layers.6.self_attn.v_proj",
	"model.qwen2_model.model.model.layers.7.mlp.down_proj",
	"model.qwen2_model.model.model.layers.7.mlp.gate_proj",
	"model.qwen2_model.model.model.layers.7.mlp.up_proj",
	"model.qwen2_model.model.model.layers.7.self_attn.k_proj",
	"model.qwen2_model.model.model.layers.7.self_attn.o_proj",
	"model.qwen2_model.model.model.layers.7.self_attn.q_proj",
	"model.qwen2_model.model.model.layers.7.self_attn.v_proj",
	"model.qwen2_model.model.model.layers.8.mlp.down_proj",
	"model.qwen2_model.model.model.layers.8.mlp.gate_proj",
	"model.qwen2_model.model.model.layers.8.mlp.up_proj",
	"model.qwen2_model.model.model.layers.8.self_attn.k_proj",
	"model.qwen2_model.model.model.layers.8.self_attn.o_proj",
	"model.qwen2_model.model.model.layers.8.self_attn.q_proj",
	"model.qwen2_model.model.model.layers.8.self_attn.v_proj",
	"model.qwen2_model.model.model.layers.9.mlp.down_proj",
	"model.qwen2_model.model.model.layers.9.mlp.gate_proj",
	"model.qwen2_model.model.model.layers.9.mlp.up_proj",
	"model.qwen2_model.model.model.layers.9.self_attn.k_proj",
	"model.qwen2_model.model.model.layers.9.self_attn.o_proj",
	"model.qwen2_model.model.model.layers.9.self_attn.q_proj",
	"model.qwen2_model.model.model.layers.9.self_attn.v_proj",
	"model.qwen2_model.query_1024",
	"model.qwen2_model.query_768",
	"model.sam_model.blocks.0.attn.proj",
	"model.sam_model.blocks.0.attn.qkv",
	"model.sam_model.blocks.0.mlp.lin1",
	"model.sam_model.blocks.0.mlp.lin2",
	"model.sam_model.blocks.1.attn.proj",
	"model.sam_model.blocks.1.attn.qkv",
	"model.sam_model.blocks.1.mlp.lin1",
	"model.sam_model.blocks.1.mlp.lin2",
	"model.sam_model.blocks.10.attn.proj",
	"model.sam_model.blocks.10.attn.qkv",
	"model.sam_model.blocks.10.mlp.lin1",
	"model.sam_model.blocks.10.mlp.lin2",
	"model.sam_model.blocks.11.attn.proj",
	"model.sam_model.blocks.11.attn.qkv",
	"model.sam_model.blocks.11.mlp.lin1",
	"model.sam_model.blocks.11.mlp.lin2",
	"model.sam_model.blocks.2.attn.proj",
	"model.sam_model.blocks.2.attn.qkv",
	"model.sam_model.blocks.2.mlp.lin1",
	"model.sam_model.blocks.2.mlp.lin2",
	"model.sam_model.blocks.3.attn.proj",
	"model.sam_model.blocks.3.attn.qkv",
	"model.sam_model.blocks.3.mlp.lin1",
	"model.sam_model.blocks.3.mlp.lin2",
	"model.sam_model.blocks.4.attn.proj",
	"model.sam_model.blocks.4.attn.qkv",
	"model.sam_model.blocks.4.mlp.lin1",
	"model.sam_model.blocks.4.mlp.lin2",
	"model.sam_model.blocks.5.attn.proj",
	"model.sam_model.blocks.5.attn.qkv",
	"model.sam_model.blocks.5.mlp.lin1",
	"model.sam_model.blocks.5.mlp.lin2",
	"model.sam_model.blocks.6.attn.proj",
	"model.sam_model.blocks.6.attn.qkv",
	"model.sam_model.blocks.6.mlp.lin1",
	"model.sam_model.blocks.6.mlp.lin2",
	"model.sam_model.blocks.7.attn.proj",
	"model.sam_model.blocks.7.attn.qkv",
	"model.sam_model.blocks.7.mlp.lin1",
	"model.sam_model.blocks.7.mlp.lin2",
	"model.sam_model.blocks.8.attn.proj",
	"model.sam_model.blocks.8.attn.qkv",
	"model.sam_model.blocks.8.mlp.lin1",
	"model.sam_model.blocks.8.mlp.lin2",
	"model.sam_model.blocks.9.attn.proj",
	"model.sam_model.blocks.9.attn.qkv",
	"model.sam_model.blocks.9.mlp.lin1",
	"model.sam_model.blocks.9.mlp.lin2",
	"model.sam_model.neck.0",
	"model.sam_model.neck.2",
	"model.sam_model.net_2",
	"model.sam_model.net_3",
	"model.sam_model.patch_embed.proj"
	],
	"provider": "auto-round"
	}