# Optimum for Intelยฎ Gaudiยฎ AI Accelerator Optimum for Intel Gaudi AI accelerator is the interface between Hugging Face libraries (Transformers, Diffusers, Accelerate,...) and [Intel Gaudi AI Accelerators (HPUs)](https://docs.habana.ai/en/latest/index.html). It provides a set of tools that enable easy model loading, training and inference on single- and multi-HPU settings for various downstream tasks as shown in the table below. Tutorials Learn the basics and become familiar with training transformers on HPUs with ๐Ÿค— Optimum. Start here if you are using ๐Ÿค— Optimum for Intel Gaudi for the first time! How-to guides Practical guides to help you achieve a specific goal. Take a look at these guides to learn how to use ๐Ÿค— Optimum for Intel Gaudi to solve real-world problems. The Intel Gaudi AI accelerator family currently includes three product generations: [Intel Gaudi 1](https://habana.ai/products/gaudi/), [Intel Gaudi 2](https://habana.ai/products/gaudi2/), and [Intel Gaudi 3](https://habana.ai/products/gaudi3/). Each server is equipped with 8 devices, known as Habana Processing Units (HPUs), providing 128GB of memory on Gaudi 3, 96GB on Gaudi 2, and 32GB on the first-gen Gaudi. For more details on the underlying hardware architecture, check out the [Gaudi Architecture Overview](https://docs.habana.ai/en/latest/Gaudi_Overview/Gaudi_Architecture.html). Optimum for Intel Gaudi library is fully compatible with all three generations of Gaudi accelerators. For in-depth examples of running workloads on Gaudi, explore the following blog posts: - [Benchmarking Intel Gaudi 2 with NVIDIA A100 GPUs](https://huggingface.co/blog/habana-gaudi-2-benchmark) - [Accelerating Vision-Language Models: BridgeTower on Habana Gaudi2](https://huggingface.co/blog/bridgetower) The following model architectures, tasks and device distributions have been validated for Optimum for Intel Gaudi: In the tables below, โœ… means single-card, multi-card and DeepSpeed have all been validated. - Transformers: | Architecture | Training | Inference | Tasks | |:------------------------------------------------------|:-----------------------------:|:-----------------------------------------------:|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | BERT | โœ… | โœ… | [text classification](/examples/text-classification)[question answering](/examples/question-answering)[language modeling](/examples/language-modeling)[text feature extraction](/examples/text-feature-extraction) | | RoBERTa | โœ… | โœ… | [question answering](/examples/question-answering)[language modeling](/examples/language-modeling) | | ALBERT | โœ… | โœ… | [question answering](/examples/question-answering)[language modeling](/examples/language-modeling) | | DistilBERT | โœ… | โœ… | [question answering](/examples/question-answering)[language modeling](/examples/language-modeling) | | GPT2 | โœ… | โœ… | [language modeling](/examples/language-modeling)[text generation](/examples/text-generation) | | BLOOM(Z) | | DeepSpeed | [text generation](/examples/text-generation) | | StarCoder / StarCoder2 | โœ… | Single-card | [language modeling](/examples/language-modeling)[text generation](/examples/text-generation) | | GPT-J | DeepSpeed | Single cardDeepSpeed | [language modeling](/examples/language-modeling)[text generation](/examples/text-generation) | | GPT-Neo | | Single card | [text generation](/examples/text-generation) | | GPT-NeoX | DeepSpeed | DeepSpeed | [language modeling](/examples/language-modeling)[text generation](/examples/text-generation) | | OPT | | DeepSpeed | [text generation](/examples/text-generation) | | Llama 2 / CodeLlama / Llama 3 / Llama Guard / Granite | โœ… | โœ… | [language modeling](/examples/language-modeling)[text generation](/examples/text-generation)[question answering](/examples/question-answering)[text classification](/examples/text-classification) (Llama Guard) | | StableLM | | Single card | [text generation](/examples/text-generation) | | Falcon | LoRA | โœ… | [language modeling](/examples/language-modeling)[text generation](/examples/text-generation) | | CodeGen | | Single card | [text generation](/examples/text-generation) | | MPT | | Single card | [text generation](/examples/text-generation) | | Mistral | | Single card | [text generation](/examples/text-generation) | | Phi | โœ… | Single card | [language modeling](/examples/language-modeling)[text generation](/examples/text-generation) | | Mixtral | | Single card | [text generation](/examples/text-generation) | | Persimmon | | Single card | [text generation](/examples/text-generation) | | Qwen2 / Qwen3 | Single card | Single card | [language modeling](/examples/language-modeling)[text generation](/examples/text-generation) | | Qwen2-MoE | | Single card | [text generation](/examples/text-generation) | | Gemma | โœ… | Single card | [language modeling](/examples/language-modeling)[text generation](/examples/text-generation) | | Gemma2 | | โœ… | [text generation](/examples/text-generation) | | Gemma3 | | โœ… | [text generation](/examples/text-generation) | | XGLM | | Single card | [text generation](/examples/text-generation) | | Cohere | | Single card | [text generation](/examples/text-generation) | | T5 / Flan T5 | โœ… | โœ… | [summarization](/examples/summarization)[translation](/examples/translation)[question answering](/examples/question-answering#fine-tuning-t5-on-squad20) | | BART | | Single card | [summarization](/examples/summarization)[translation](/examples/translation)[question answering](/examples/question-answering#fine-tuning-t5-on-squad20) | | ViT | โœ… | โœ… | [image classification](/examples/image-classification) | | Swin | โœ… | โœ… | [image classification](/examples/image-classification) | | Wav2Vec2 | โœ… | โœ… | [audio classification](/examples/audio-classification)[speech recognition](/examples/speech-recognition) | | Whisper | โœ… | โœ… | [speech recognition](/examples/speech-recognition) | | SpeechT5 | | Single card | [text to speech](/examples/text-to-speech) | | CLIP | โœ… | โœ… | [contrastive image-text training](/examples/contrastive-image-text) | | BridgeTower | โœ… | โœ… | [contrastive image-text training](/examples/contrastive-image-text) | | ESMFold | | Single card | [protein folding](/examples/protein-folding) | | Blip | | Single card | [visual question answering](/examples/visual-question-answering)[image to text](/examples/image-to-text) | | OWLViT | | Single card | [zero shot object detection](/examples/zero-shot-object-detection) | | ClipSeg | | Single card | [object segmentation](/examples/object-segementation) | | Llava / Llava-next / Llava-onevision | | Single card | [image to text](/examples/image-to-text) | | idefics2 | LoRA | Single card | [image to text](/examples/image-to-text) | | Paligemma | | Single card | [image to text](/examples/image-to-text) | | Segment Anything Model | | Single card | [object segmentation](/examples/object-segementation) | | VideoMAE | | Single card | [Video classification](/examples/video-classification) | | TableTransformer | | Single card | [table object detection](/examples/table-detection) | | DETR | | Single card | [object detection](/examples/object-detection) | | Mllama | LoRA | โœ… | [image to text](/examples/image-to-text) | | MiniCPM3 | | Single card | [text generation](/examples/text-generation) | | Baichuan2 | DeepSpeed | Single card | [language modeling](/examples/language-modeling)[text generation](/examples/text-generation) | | DeepSeek-V2 | โœ… | โœ… | [text generation](/examples/text-generation) | | DeepSeek-V3 / Moonlight | | โœ… | [text generation](/examples/text-generation) | | ChatGLM | DeepSpeed | Single card | [language modeling](/examples/language-modeling)[text generation](/examples/text-generation) | | Qwen2-VL | | Single card | [image to text](/examples/image-to-text) | | Qwen2.5-VL | | Single card | [image to text](/examples/image-to-text) | | VideoLLaVA | | Single card | [Video comprehension](/examples/video-comprehension) | | GLM-4V | | Single card | [image to text](/examples/image-to-text) | | Arctic | | DeepSpeed | [text generation](/examples/text-generation) | | GPT-OSS | | DeepSpeed | [text generation](/examples/text-generation) | - Diffusers | Architecture | Training. | Inference | Tasks | |----------------------------|:----------------------:|:-----------------------------:|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Stable Diffusion | โœ… | โœ… | [text-to-image generation](/examples/stable-diffusion) | | Stable Diffusion XL | โœ… | โœ… | [text-to-image generation](/examples/stable-diffusion) | | Stable Diffusion Depth2img | | Single card | [depth-to-image generation](/examples/stable-diffusion) | | Stable Diffusion 3 | โœ… | โœ… | [text-to-image generation](/examples/stable-diffusion#stable-diffusion-3-and-35-sd3) | | LDM3D | | Single card | [text-to-image generation](/examples/stable-diffusion) | | FLUX.1 | LoRA | Single card | [text-to-image generation](/examples/stable-diffusion) | | Qwen Image | | Single card | [text-to-image generation](https://github.com/huggingface/optimum-habana/tree/main/examples/stable-diffusion) | | Text to Video | | Single card | [text-to-video generation](/examples/stable-diffusion#text-to-video-generation) | | Image to Video | | Single card | [image-to-video generation](/examples/stable-diffusion#image-to-video-generation) | | i2vgen-xl | | Single card | [image-to-video generation](/examples/stable-diffusion#I2vgen-xl) | | Wan | | โœ… | [text-to-video generation](/examples/stable-diffusion#text-to-video-with-wan-22)[image-to-video generation](/examples/stable-diffusion#image-to-video-with-wan-22) | - PyTorch Image Models/TIMM: | Architecture | Training | Inference | Tasks | |---------------------|:--------:|:-----------------------------:|:--------------------------------------------------------------------------| | FastViT | | Single card | [image classification](/examples/image-classification) | - TRL: | Architecture | Training | Inference | Tasks | |------------------|:--------:|:---------:|:-------------------------------------------------| | Llama 2 | โœ… | | [DPO Pipeline](/examples/trl) | | Llama 2 | โœ… | | [PPO Pipeline](/examples/trl) | | Stable Diffusion | โœ… | | [DDPO Pipeline](/examples/trl) | Other models and tasks supported by the ๐Ÿค— Transformers and ๐Ÿค— Diffusers library may also work. You can refer to this [section](https://github.com/huggingface/optimum-habana#how-to-use-it) for using them with ๐Ÿค— Optimum for Intel Gaudi. In addition, [this page](/examples) explains how to modify any [example](https://github.com/huggingface/transformers/tree/main/examples/pytorch) from the ๐Ÿค— Transformers library to make it work with ๐Ÿค— Optimum for Intel Gaudi.