DEV Community

#llamacpp

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
How to Use a Remote Ollama or llama.cpp Server From a Desktop AI App on Windows in 2026
alichherawalla profile

How to Use a Remote Ollama or llama.cpp Server From a Desktop AI App on Windows in 2026

#ollama #llamacpp #windows #ai
Comments
6 min read
Hands-On: Qwen3.8-Flash-Next and a Glimpse into Qwen4's Design Philosophy — llama.cpp Support Status and A100 Benchmarks
oooocean66 profile

Hands-On: Qwen3.8-Flash-Next and a Glimpse into Qwen4's Design Philosophy — llama.cpp Support Status and A100 Benchmarks

#ai #llm #llamacpp #qwen
Comments
20 min read
Extending llama.cpp to Support Reflection Beam-1.0-501B-A23B: Handling Complex MoE and Attention Architectures
pdrojack profile

Extending llama.cpp to Support Reflection Beam-1.0-501B-A23B: Handling Complex MoE and Attention Architectures

#llamacpp #mixtureofexperts #attention #modelconversion
Comments
3 min read
Accuracy vs Speed: Gemma 4 E2B Quantization Tradeoffs, BF16 to Q2_K

Hacktoberfest Open-Source AI Challenge: Ghost in the Machine Submission 👻🌼
xbill profile

Accuracy vs Speed: Gemma 4 E2B Quantization Tradeoffs, BF16 to Q2_K

#devchallenge #hf26challenge #gemma #llamacpp
11
Comments 6
15 min read
Running MoE LLMs Larger Than Your RAM: A Practical Guide to mmap, Expert Offloading and SSD Streaming with llama.cpp
eme_gug_0821b41b948be6516 profile

Running MoE LLMs Larger Than Your RAM: A Practical Guide to mmap, Expert Offloading and SSD Streaming with llama.cpp

#llm #llamacpp #moe #performance
Comments
8 min read
Four GPUs, One Quantization Ladder: Gemma 4 E2B From a 4 GB Laptop to an NVIDIA L4
xbill profile

Four GPUs, One Quantization Ladder: Gemma 4 E2B From a 4 GB Laptop to an NVIDIA L4

#gemma #llamacpp #quantization #nvidia
13
Comments 2
11 min read
llama-server's prompt cache reuses KV computed under a different LoRA scale
homelabpm profile

llama-server's prompt cache reuses KV computed under a different LoRA scale

#llamacpp #llm #selfhosted #lora
Comments
6 min read
Performance vs Quality: Gemma 4 E2B Quantization on an NVIDIA L4 and a Tesla T4
xbill profile

Performance vs Quality: Gemma 4 E2B Quantization on an NVIDIA L4 and a Tesla T4

#gemma #llamacpp #aws #quantization
7
Comments 1
11 min read
Fine-tuning Qwen2.5-1.5B on a Mac to fix a parroting chatbot (and halve the prompt)
kharlampiiev profile

Fine-tuning Qwen2.5-1.5B on a Mac to fix a parroting chatbot (and halve the prompt)

#ai #llm #machinelearning #llamacpp
Comments
6 min read
Performance vs Quality: Gemma 4 E2B Quantization on an NVIDIA L4 and a Tesla T4
xbill profile

Performance vs Quality: Gemma 4 E2B Quantization on an NVIDIA L4 and a Tesla T4

#gemma #llamacpp #aws #quantization
Comments
11 min read
Running 100B+ MoE Models on a Single RTX 4090: A Practical Guide to Expert Offloading with llama.cpp
eme_gug_0821b41b948be6516 profile

Running 100B+ MoE Models on a Single RTX 4090: A Practical Guide to Expert Offloading with llama.cpp

#llm #llamacpp #gpu #machinelearning
Comments
8 min read
Running 100B+ MoE LLMs on a Single RTX 4090: A Practical Guide to Expert Offloading with llama.cpp
eme_gug_0821b41b948be6516 profile

Running 100B+ MoE LLMs on a Single RTX 4090: A Practical Guide to Expert Offloading with llama.cpp

#llm #llamacpp #gpu #selfhosted
Comments
7 min read
When Chat Templates Go Wrong
pielounw profile

When Chat Templates Go Wrong

#llm #opensource #ai #llamacpp
Comments
7 min read
Size Your Model to Where You Run It: Gemma 4 E2B and E4B on an AWS L4 and Two Small GPUs
xbill profile

Size Your Model to Where You Run It: Gemma 4 E2B and E4B on an AWS L4 and Two Small GPUs

#gemma #llamacpp #aws #cuda
8
Comments
10 min read
llama.cpp splits Hindi and Thai differently for Nemotron 3 because its GGUF says pixtral
homelabpm profile

llama.cpp splits Hindi and Thai differently for Nemotron 3 because its GGUF says pixtral

#llm #llamacpp #selfhosted #ai
Comments
6 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.