> entire point...smaller download could not justify...
Q4_K_M has layers and layers of consensus and polling and surveying and A/B testing and benchmarking to show there's ~0 quality degradation. Built over a couple years.
Llama 3.3 already shows a degradation from Q5 to Q4.
As compression improves over the years, the effects of even Q5 quantization will begin to appear
> entire point...smaller download could not justify...
Q4_K_M has layers and layers of consensus and polling and surveying and A/B testing and benchmarking to show there's ~0 quality degradation. Built over a couple years.