Notes from building LLM systems that have to work.
What actually holds up in production — retrieval, cost, deployment, and the tradeoffs nobody puts in the sales deck. Written from client work and our own builds.
Ollama, vLLM, or llama.cpp: picking a server you won't have to replace
The three are not competing to be the best inference server. They are three different bets on how many people will use the thing at once — and that's the only question that decides which one you want.
Read the post →Long context didn't kill RAG — it changed what RAG is for
Every time a model ships a bigger context window, someone declares retrieval obsolete. Then they try to put a real corpus in the prompt and rediscover, at $1.50 a question, why retrieval exists.
Read the post →Two 128GB boxes, one bandwidth problem
NVIDIA's DGX Spark and AMD's Ryzen AI Max+ 395 both put 128GB of unified memory on a desk. Their memory bandwidth is nearly identical — so the real gaps are native FP4, prefill, how many people you can serve at once, and what happens when you stack more than one.
Read the post →The local-vs-cloud cost argument is usually wrong
Most teams justify on-prem AI with a token-cost spreadsheet. Run the numbers honestly and cloud usually wins on cost alone — which means the real reasons to go local are the ones nobody puts in the deck.
Read the post →How to work out what one GPU can actually run
Vendor benchmarks won't tell you whether a model fits on your card. The arithmetic will — and the constraint is almost never the weights.
Read the post →Vector search alone is not a retrieval system
The demo works and then someone searches for an invoice number. What separates a RAG prototype from something people trust is almost entirely in the retrieval layer.
Read the post →Have a workflow you're trying to figure out?
Book a 30-minute discovery call. We'll map it and tell you straight whether it belongs on your hardware or on a cloud API.