Exllama
😽
6
Chat: exllama v2
Pablo Carrera (パブロ)
pabloce
AI & ML interests
Hello, hello!
Recent Activity
updated
a Space about 12 hours ago
pabloce/exllama updated
a dataset 2 days ago
pabloce/esbieta-recipes-qa liked
a dataset 4 days ago
togethercomputer/CoderForge-Preview Organizations
updated a Space about 12 hours ago
updated a dataset 2 days ago
liked a dataset 4 days ago
upvoted an article 11 days ago
Article
GGML and llama.cpp join HF to ensure the long-term progress of Local AI
- +4
•
479
upvoted 2 articles about 2 months ago
Article
We Got Claude to Fine-Tune an Open Source LLM
•
607
Article
New in llama.cpp: Anthropic Messages API
•
39
upvoted an article 3 months ago
Article
New in llama.cpp: Model Management
•
126
upvoted a collection 3 months ago
liked a Space 4 months ago
liked 2 models 7 months ago
upvoted a changelog 7 months ago
Hugging Face Changelog
Introducing a better Hugging Face CLI
• 96
liked a model 8 months ago
upvoted an article 8 months ago
Article
Unlocking Healthcare AI: I'm Releasing State-of-the-Art Medical Models for Free. Forever.
•
152
liked 2 models 8 months ago
reacted to cgeorgiaw's post with ❤️ 8 months ago
Post
2810
Huge new bio datasets just dropped!!!
Check out them out @
ginkgo-datapoints
Read the blog for more info: https://huggingface.co/blog/cgeorgiaw/gdp
Check out them out @
Read the blog for more info: https://huggingface.co/blog/cgeorgiaw/gdp
- 1 reply
reacted to bartowski's post with 🤗 9 months ago
Post
93402
Was going to post this on /r/LocalLLaMa, but apparently it's without moderation at this time :')
bartowski/mistralai_Mistral-Small-3.2-24B-Instruct-2506-GGUF
Was able to use previous mistral chat templates, some hints from Qwen templates, and Claude to piece together a seemingly working chat template, tested it with llama.cpp server and got perfect results, though lmstudio still seems to be struggling for some reason (don't know how to specify a jinja file there)
Outlined the details of the script and results in my llama.cpp PR to add the jinja template:
https://github.com/ggml-org/llama.cpp/pull/14349
Start server with a command like this:
and it should be perfect! Hoping it'll work for ALL tools if lmstudio gets an update or something, not just llama.cpp, but very happy to see it works flawlessly in llama.cpp
In the meantime, will try to open a PR to minja to make the strftime work, but no promises :)
bartowski/mistralai_Mistral-Small-3.2-24B-Instruct-2506-GGUF
Was able to use previous mistral chat templates, some hints from Qwen templates, and Claude to piece together a seemingly working chat template, tested it with llama.cpp server and got perfect results, though lmstudio still seems to be struggling for some reason (don't know how to specify a jinja file there)
Outlined the details of the script and results in my llama.cpp PR to add the jinja template:
https://github.com/ggml-org/llama.cpp/pull/14349
Start server with a command like this:
./llama-server -m /models/mistralai_Mistral-Small-3.2-24B-Instruct-2506-Q4_K_M.gguf --jinja --chat-template-file /models/Mistral-Small-3.2-24B-Instruct-2506.jinjaand it should be perfect! Hoping it'll work for ALL tools if lmstudio gets an update or something, not just llama.cpp, but very happy to see it works flawlessly in llama.cpp
In the meantime, will try to open a PR to minja to make the strftime work, but no promises :)
liked 2 models 9 months ago
