nvidia

/

llama-embed-nemotron-8b

like 163

Follow

NVIDIA 58.1k

Hey! thanks for this release.

At this moment llama.cpp is not able to convert the models into gguf, it looks like your architecture is still not supported. Is there any plan to release the models in this quantization?

python3 convert_hf_to_gguf.py models/nvidia/llama-embed-nemotron-8b/ --outfile llama-embed-f16.gguf
INFO:hf-to-gguf:Loading model: llama-embed-nemotron-8b
INFO:hf-to-gguf:Model architecture: LlamaBidirectionalModel
ERROR:hf-to-gguf:Model LlamaBidirectionalModel is not supported

Thanks in advance