1 post tagged with vLLM
Two GX10 boxes, a 230B MoE coding model, and one flag I should never have set. How 25 tok/s became 40.4, and why it stops there.
