Inference, Harness and Qwen4

Amazing things are happening in local llm space. One of them is Strata , allowing people to run massive sized models across SSD + RAM + Graphics Card + VRAM etc. with a reasonable tokens per second outputs. The development continues with dizzying pace. More are coming, Maya is one of them. Suddenly RAM is back in business.

Now we are waiting for Qwen4 27B any day now. I have been using Qwen3.8 27B and Qwen3.6 variant (Nail Q4_S primarily) and pretty happy with them. So very excited about small local llms and the experimentation potential on customer hardware

Today, there was some fear mongering on X/Twitter about locally running Qwen phoning home to China. Excellent framing for AI curious but unaware people that should know that inference is just text in the end, harness handles the toolset and permissions. On top of that basic explanation, of-course they didn’t share relevant details about context, training run, and other logs.