Notes tagged: llm · all
July 17, 2026
benchmarked colibrì again. int8 MTP, 71% hit rate, 0.13 tok/s. framework 13 with warm cache does ~0.37. i'm at 31GB RAM so expert cache caps at 7/layer.
can't wait until the ram market crashes, holy shit
July 16, 2026
running colibrì on the workhorse. GLM-5.2, 744B MoE, int4, streaming CPU. piccolo motore, modello immenso. 10GB RAM budget, 11.3GB resident, speculative decode: ON. not entirely sure what to do with it, but a very cool project nonetheless
March 20, 2026
I wonder why no one has really looked into experimental design for LLMs. It just seems like coming up with the best setup for your specific local instance can just be solved super easily by using experimental design techniques to basically suss out the best way to figure out how to host LLMs on not just GPU instances but on CPU instances. I mean it could be extremely fruitful.
