https://2317ecb3.hugo-blog-e6x.pages.dev/tags/llm/ Recent content in Llm on Aditya Naik | Personal Website Hugo en Tue, 27 Oct 2026 21:13:26 +0100 https://2317ecb3.hugo-blog-e6x.pages.dev/note/inference-harness/ Tue, 27 Oct 2026 21:13:26 +0100 https://2317ecb3.hugo-blog-e6x.pages.dev/note/inference-harness/ <p>Amazing things are happening in local llm space. One of them is <a href="https://github.com/Niko1221/Strata" target="_blank" rel="noopener" >Strata</a > , allowing people to run massive sized models across SSD + RAM + Graphics Card + VRAM etc. with a reasonable tokens per second outputs. The development continues with dizzying pace. More are coming, <a href="https://github.com/mw00/project-maya" target="_blank" rel="noopener" >Maya</a > is one of them. Suddenly RAM is back in business.</p> <p>Now we are waiting for Qwen4 27B any day now. I have been using Qwen3.8 27B and Qwen3.6 variant (Nail Q4_S primarily) and pretty happy with them. So very excited about small local llms and the experimentation potential on customer hardware</p>