Quantization - DEV Community

Skip to content

DEV Community

👋 Sign in for the ability to sort posts by relevant, latest, or top.

Unsloth Releases Qwen3.6-27B-NVFP4: Enhanced Throughput and Agentic Coding for Developers

#aiml #largelanguagemodels #quantization #qwen

3 min read

Bonsai-27B: A 1-Bit LLM for On-Device Inference with Llama.cpp and MLX

#llm #quantization #1bit #gguf

3 min read

JoTeq the First

Quantizing MedGemma to INT4 (GPTQ/W4A16): Everything That Broke Along the Way

#machinelearning #llm #quantization #opensource

6 min read

Self-Hosting a Model Means Self-Hosting Its Evaluation Too

#ai #llm #quantization #mlops

5 min read

Gemma 4 QAT on a 1080 Ti: What 'Quantization-Aware' Actually Buys — and Fitting the 12B on 8 GB at 16k

#llm #machinelearning #gemma #quantization

5 min read

Quantization formats compared: GGUF vs GPTQ vs AWQ vs NF4

#llm #quantization #mlops #tutorial

7 min read

INT8 Q/DQ Calibration on Blackwell: 1.8 the TRT 10 + FP16 Baseline

#tensorrt #quantization #gpu #machinelearning

7 min read

LLM Quantization Levels Compared: Q4_K_M vs Q8_0 vs FP16 [2026]

#localllm #quantization #gguf #ollama

15 min read

How to Pick a GGUF Quant Level for Your VRAM Budget

#localllm #gguf #quantization #gpu

4 min read

Why your quantized LLM loses its MTP heads and how to keep them

#machinelearning #llm #python #quantization

5 min read

The Best Result This Week Was a Failed Prediction — Phase-3a Doesn't Transfer

#quantization #hsaq #methodology #granite

1 min read

Two Localizers, Both Wrong: Bounding a Quantization Cost That Wouldn't Close

#quantization #hsaq #methodology #granite

1 min read

When the Sensitivity Metric Lies: A Drift-Inversion Smoking Gun in Mixed-Precision LLM Quantization

#quantization #hsaq #awq #granite

8 min read

GGUF Quantization Explained: Q4_K_M vs Q5_K_M vs Q8 — Which to Pick (2026)

#llamacpp #gguf #quantization #localai

5 min read

1-bit, 545 megabytes, zero API keys — local AI that beats GPT-5.4

#ai #llm #local #quantization

2 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.