Hugging Face's logo

empyrealworks
/
EmbeddingGemma-2-GGUF

GGUF
embedding
feature-extraction
Model card Files
xet
 Community
How to use from
llama.cpp
Install (macOS, Linux) 
curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible embeddings server:
llama serve -hf empyrealworks/EmbeddingGemma-2-GGUF:Q8_0 --embeddings
Install from WinGet (Windows) 
winget install llama.cpp
# Start a local OpenAI-compatible embeddings server:
llama serve -hf empyrealworks/EmbeddingGemma-2-GGUF:Q8_0 --embeddings
Build from source code 
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible embeddings server:
./build/bin/llama serve -hf empyrealworks/EmbeddingGemma-2-GGUF:Q8_0 --embeddings
Quick Links

EmbeddingGemma 2 GGUF (UD-Q4_K_XL)

embeddinggemma-2-UD-Q4_K_XL.gguf is an unmodified copy of the file from unsloth/embeddinggemma-2-GGUF at revision ba3888272494be64ed88c9eb536ddc61a1be73d5.

  • Size: 175673856 bytes
  • SHA-256: ea905fd08e8061db77a0031cbb5096d459d7cfd2bdd0ad2ba66e483f16719493

The base model is Google's EmbeddingGemma 2, distributed under the Apache 2.0 license.

embeddinggemma-2-512d-UD-Q4_K_XL.gguf

The file above with one tensor changed, giving 512-dimensional vectors. Its output projection output.weight (512 to 768, Q6_K) was dequantized and factored in FP64 as W = U diag(S) Vh, and replaced by the 512 x 512 F32 matrix diag(S) Vh (the method of tomaarsen/embeddinggemma-2-512d). Because U has orthonormal columns, the vectors keep the norms and cosine similarities of the 768-dimensional originals, up to floating-point rounding. Every other tensor is byte-identical to the file above, and gemma-embedding2.embedding_length_out is 512. These vectors cannot be truncated below 512 dimensions: the factorization does not keep the Matryoshka structure.

  • Size: 176400064 bytes
  • SHA-256: 6e535e12366cf8ebf6635b1c1c0079610c09a36c1b0ec4220009aa40f2b54f58

embeddinggemma-2-512d-Q8_0.gguf

Made the same way from the Q8_0 file of unsloth/embeddinggemma-2-GGUF at revision 031f0d4b35536f69ab3509d4893c923264fcf253 (embeddinggemma-2-Q8_0.gguf, 309855520 bytes, SHA-256 6f1bd4ac6c5df7444f9cca7ca36cafe6cfa34cd6f49fefb1e0b4be8143aed8bc): its output projection output.weight (512 to 768, Q8_0) was dequantized and factored in FP64 as W = U diag(S) Vh, and replaced by the 512 x 512 F32 matrix diag(S) Vh. Every other tensor is byte-identical to that file, and gemma-embedding2.embedding_length_out is 512. Each factorization fixes its own basis, so these vectors cannot be mixed with those of the UD-Q4_K_XL 512-dimensional file above.

  • Size: 310486304 bytes
  • SHA-256: 20bd7c54d03634f599e8a9a35a399d2d2c3573097c3404efd2f76150c554bc95
Downloads last month
296
GGUF
Model size
0.3B params
Architecture
gemma-embedding2
Hardware compatibility
Log In to add your hardware

4-bit

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for empyrealworks/EmbeddingGemma-2-GGUF

Base model

google/embeddinggemma-2
Quantized
(122)
this model