https://minamism.com/tags/vllm/Recent content in Vllm on minamismHugoen-USFri, 11 Sep 2026 01:00:00 +0900https://minamism.com/posts/vllm-int5-7-quantization/Fri, 11 Sep 2026 01:00:00 +0900https://minamism.com/posts/vllm-int5-7-quantization/vLLM can natively execute INT5/6/7 weights. Here is an overview of how well they perform, along with results showing that quantizing embed_tokens and lm_head causes minimal loss.