launch80:~$ ls insights/
Notes from the lab.
# research, benchmarks, and field notes on running local models.
/11 min read
One Model, Every Local Stack
We pointed every local-AI server we could run — ctrl8d, mlx-vlm, mlx_lm, vMLX, LM Studio, llama.cpp, ollama, vLLM — plus four coding agents at the same 35B model on one Mac and measured them. Several couldn’t even load it.
#engineering#ai-infrastructure#benchmark
/7 min read
Two Optimizations That Were Never Allowed to Run Together
Our local inference server made us choose between fast responses and fast typing. We forked it, removed the choice — then found our own benchmark had been flattering us, and re-ran it properly.
#engineering#ai-infrastructure#performance