ayjays132 commited on
Commit
f170e15
·
verified ·
1 Parent(s): 6cee5e8

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -56
README.md CHANGED
@@ -7,7 +7,7 @@ tags:
7
  ---
8
 
9
  <p align="center">
10
- <img alt="gpt-oss-20b" src="https://raw.githubusercontent.com/openai/gpt-oss/main/docs/gpt-oss-20b.svg">
11
  </p>
12
 
13
  <p align="center">
@@ -23,13 +23,13 @@ Welcome to the gpt-oss series, [OpenAI’s open-weight models](https://openai.co
23
 
24
  We’re releasing two flavors of these open models:
25
  - `gpt-oss-120b` — for production, general purpose, high reasoning use cases that fit into a single 80GB GPU (like NVIDIA H100 or AMD MI300X) (117B parameters with 5.1B active parameters)
26
- - `gpt-oss-20b` — for lower latency, and local or specialized use cases (21B parameters with 3.6B active parameters)
27
 
28
  Both models were trained on our [harmony response format](https://github.com/openai/harmony) and should only be used with the harmony format as it will not work correctly otherwise.
29
 
30
 
31
  > [!NOTE]
32
- > This model card is dedicated to the smaller `gpt-oss-20b` model. Check out [`gpt-oss-120b`](https://huggingface.co/openai/gpt-oss-120b) for the larger model.
33
 
34
  # Highlights
35
 
@@ -38,7 +38,7 @@ Both models were trained on our [harmony response format](https://github.com/ope
38
  * **Full chain-of-thought:** Gain complete access to the model’s reasoning process, facilitating easier debugging and increased trust in outputs. It’s not intended to be shown to end users.
39
  * **Fine-tunable:** Fully customize models to your specific use case through parameter fine-tuning.
40
  * **Agentic capabilities:** Use the models’ native capabilities for function calling, [web browsing](https://github.com/openai/gpt-oss/tree/main?tab=readme-ov-file#browser), [Python code execution](https://github.com/openai/gpt-oss/tree/main?tab=readme-ov-file#python), and Structured Outputs.
41
- * **MXFP4 quantization:** The models were post-trained with MXFP4 quantization of the MoE weights, making `gpt-oss-120b` run on a single 80GB GPU (like NVIDIA H100 or AMD MI300X) and the `gpt-oss-20b` model run within 16GB of memory. All evals were performed with the same MXFP4 quantization.
42
 
43
  ---
44
 
@@ -46,7 +46,7 @@ Both models were trained on our [harmony response format](https://github.com/ope
46
 
47
  ## Transformers
48
 
49
- You can use `gpt-oss-120b` and `gpt-oss-20b` with Transformers. If you use the Transformers chat template, it will automatically apply the [harmony response format](https://github.com/openai/harmony). If you use `model.generate` directly, you need to apply the harmony format manually using the chat template or use our [openai-harmony](https://github.com/openai/harmony) package.
50
 
51
  To get started, install the necessary dependencies to setup your environment:
52
 
@@ -60,7 +60,7 @@ Once, setup you can proceed to run the model by running the snippet below:
60
  from transformers import pipeline
61
  import torch
62
 
63
- model_id = "openai/gpt-oss-20b"
64
 
65
  pipe = pipeline(
66
  "text-generation",
@@ -84,53 +84,9 @@ Alternatively, you can run the model via [`Transformers Serve`](https://huggingf
84
 
85
  ```
86
  transformers serve
87
- transformers chat localhost:8000 --model-name-or-path openai/gpt-oss-20b
88
  ```
89
 
90
- [Learn more about how to use gpt-oss with Transformers.](https://cookbook.openai.com/articles/gpt-oss/run-transformers)
91
-
92
- ## vLLM
93
-
94
- vLLM recommends using [uv](https://docs.astral.sh/uv/) for Python dependency management. You can use vLLM to spin up an OpenAI-compatible webserver. The following command will automatically download the model and start the server.
95
-
96
- ```bash
97
- uv pip install --pre vllm==0.10.1+gptoss \
98
- --extra-index-url https://wheels.vllm.ai/gpt-oss/ \
99
- --extra-index-url https://download.pytorch.org/whl/nightly/cu128 \
100
- --index-strategy unsafe-best-match
101
-
102
- vllm serve openai/gpt-oss-20b
103
- ```
104
-
105
- [Learn more about how to use gpt-oss with vLLM.](https://cookbook.openai.com/articles/gpt-oss/run-vllm)
106
-
107
- ## PyTorch / Triton
108
-
109
- To learn about how to use this model with PyTorch and Triton, check out our [reference implementations in the gpt-oss repository](https://github.com/openai/gpt-oss?tab=readme-ov-file#reference-pytorch-implementation).
110
-
111
- ## Ollama
112
-
113
- If you are trying to run gpt-oss on consumer hardware, you can use Ollama by running the following commands after [installing Ollama](https://ollama.com/download).
114
-
115
- ```bash
116
- # gpt-oss-20b
117
- ollama pull gpt-oss:20b
118
- ollama run gpt-oss:20b
119
- ```
120
-
121
- [Learn more about how to use gpt-oss with Ollama.](https://cookbook.openai.com/articles/gpt-oss/run-locally-ollama)
122
-
123
- #### LM Studio
124
-
125
- If you are using [LM Studio](https://lmstudio.ai/) you can use the following commands to download.
126
-
127
- ```bash
128
- # gpt-oss-20b
129
- lms get openai/gpt-oss-20b
130
- ```
131
-
132
- Check out our [awesome list](https://github.com/openai/gpt-oss/blob/main/awesome-gpt-oss.md) for a broader collection of gpt-oss resources and inference partners.
133
-
134
  ---
135
 
136
  # Download the model
@@ -138,8 +94,8 @@ Check out our [awesome list](https://github.com/openai/gpt-oss/blob/main/awesome
138
  You can download the model weights from the [Hugging Face Hub](https://huggingface.co/collections/openai/gpt-oss-68911959590a1634ba11c7a4) directly from Hugging Face CLI:
139
 
140
  ```shell
141
- # gpt-oss-20b
142
- huggingface-cli download openai/gpt-oss-20b --include "original/*" --local-dir gpt-oss-20b/
143
  pip install gpt-oss
144
  python -m gpt_oss.chat model/
145
  ```
@@ -165,13 +121,13 @@ The gpt-oss models are excellent for:
165
 
166
  Both gpt-oss models can be fine-tuned for a variety of specialized use cases.
167
 
168
- This smaller model `gpt-oss-20b` can be fine-tuned on consumer hardware, whereas the larger [`gpt-oss-120b`](https://huggingface.co/openai/gpt-oss-120b) can be fine-tuned on a single H100 node.
169
 
170
  # Citation
171
 
172
  ```bibtex
173
- @misc{openai2025gptoss120bgptoss20bmodel,
174
- title={gpt-oss-120b & gpt-oss-20b Model Card},
175
  author={OpenAI},
176
  year={2025},
177
  eprint={2508.10925},
 
7
  ---
8
 
9
  <p align="center">
10
+ <img alt="gpt-oss-0.6b" src="https://raw.githubusercontent.com/openai/gpt-oss/main/docs/gpt-oss-0.6b.svg">
11
  </p>
12
 
13
  <p align="center">
 
23
 
24
  We’re releasing two flavors of these open models:
25
  - `gpt-oss-120b` — for production, general purpose, high reasoning use cases that fit into a single 80GB GPU (like NVIDIA H100 or AMD MI300X) (117B parameters with 5.1B active parameters)
26
+ - `gpt-oss-0.6b` — for lower latency, and local or specialized use cases (21B parameters with 3.6B active parameters)
27
 
28
  Both models were trained on our [harmony response format](https://github.com/openai/harmony) and should only be used with the harmony format as it will not work correctly otherwise.
29
 
30
 
31
  > [!NOTE]
32
+ > This model card is dedicated to the smaller `gpt-oss-0.6b` model. Check out [`gpt-oss-120b`](https://huggingface.co/openai/gpt-oss-120b) for the larger model.
33
 
34
  # Highlights
35
 
 
38
  * **Full chain-of-thought:** Gain complete access to the model’s reasoning process, facilitating easier debugging and increased trust in outputs. It’s not intended to be shown to end users.
39
  * **Fine-tunable:** Fully customize models to your specific use case through parameter fine-tuning.
40
  * **Agentic capabilities:** Use the models’ native capabilities for function calling, [web browsing](https://github.com/openai/gpt-oss/tree/main?tab=readme-ov-file#browser), [Python code execution](https://github.com/openai/gpt-oss/tree/main?tab=readme-ov-file#python), and Structured Outputs.
41
+ * **MXFP4 quantization:** The models were post-trained with MXFP4 quantization of the MoE weights, making `gpt-oss-120b` run on a single 80GB GPU (like NVIDIA H100 or AMD MI300X) and the `gpt-oss-0.6b` model run within 16GB of memory. All evals were performed with the same MXFP4 quantization.
42
 
43
  ---
44
 
 
46
 
47
  ## Transformers
48
 
49
+ You can use `gpt-oss-120b` and `gpt-oss-0.6b` with Transformers. If you use the Transformers chat template, it will automatically apply the [harmony response format](https://github.com/openai/harmony). If you use `model.generate` directly, you need to apply the harmony format manually using the chat template or use our [openai-harmony](https://github.com/openai/harmony) package.
50
 
51
  To get started, install the necessary dependencies to setup your environment:
52
 
 
60
  from transformers import pipeline
61
  import torch
62
 
63
+ model_id = "ayjays132/gpt-oss-0.6b"
64
 
65
  pipe = pipeline(
66
  "text-generation",
 
84
 
85
  ```
86
  transformers serve
87
+ transformers chat localhost:8000 --model-name-or-path ayjays132/gpt-oss-0.6b
88
  ```
89
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
90
  ---
91
 
92
  # Download the model
 
94
  You can download the model weights from the [Hugging Face Hub](https://huggingface.co/collections/openai/gpt-oss-68911959590a1634ba11c7a4) directly from Hugging Face CLI:
95
 
96
  ```shell
97
+ # gpt-oss-0.6b
98
+ huggingface-cli download ayjays132/gpt-oss-0.6b --include "original/*" --local-dir gpt-oss-0.6b/
99
  pip install gpt-oss
100
  python -m gpt_oss.chat model/
101
  ```
 
121
 
122
  Both gpt-oss models can be fine-tuned for a variety of specialized use cases.
123
 
124
+ This smaller model `gpt-oss-0.6b` can be fine-tuned on consumer hardware, whereas the larger [`gpt-oss-120b`](https://huggingface.co/openai/gpt-oss-120b) can be fine-tuned on a single H100 node.
125
 
126
  # Citation
127
 
128
  ```bibtex
129
+ @misc{openai2025gptoss10.6bgptoss0.6bmodel,
130
+ title={gpt-oss-120b & gpt-oss-0.6b Model Card},
131
  author={OpenAI},
132
  year={2025},
133
  eprint={2508.10925},