From the sky to the bedrock.
Big projects or small ones: AI agents, apps, cloud platforms and the hardware underneath. Pick a layer to explore.
What brings you here?
Over fifteen years of building software and the systems it runs on. Start with the problem in front of you.
Nonprofits, media and IoT companies, enterprise SaaS, small business.
- IoT & enterprise SaaSTSI
- Media appliancesNorth Shore Automation
- Lake healthLittle Sand Lake Area Association
- Heritage nonprofitSteiger Heritage Club
What I’m building right now.
Short posts from the build log as the work happens, and long-form case studies once it’s done.
Build log Live from Mastodon
@jacob on feed.jacobnollette.com
- View on Mastodon ↗
None of that was supposed to be about drives. But sizing from worst case meant measuring the drives properly, and measuring them properly is how I found that TRIM had never run on any of the Ceph flash: 15 of 16 devices had issued exactly zero discard commands, across roughly 13,700 power-on hours each. Ceph ships that setting off by default.
- View on Mastodon ↗
The Ceph PG count had been too low for months and I'd deferred raising it twice, because the nodes sat at 80% memory requested and a split needs somewhere to go. They were using 25%. Once the requests were honest the split just started. A fair number of things on a deferred list are not blocked by the thing the list says is blocking them.
- View on Mastodon ↗
Removing swap changed how I size everything. Before, a request ten times too large was just waste. Now the fat allocation is the whole budget, because there's no valve left. Found 103 GiB of Kubernetes memory requested against 10 GiB actually used, and four VMs holding 160 GiB to run 11. Re-sized on measured 7-day peaks. Limits untouched, so anything that genuinely needs more still gets it.
Case studies
Long-form write-ups: what broke, and what fixed it.
Nine Copies of Every Write
My Kubernetes control plane had been running five times slower than etcd considers healthy for over a year. I spent two hours blaming Ceph recovery throttles, caused an outage proving it, and then found…
Agents & AIThe Question That Vanished, and the Fix That Was Almost Right
A self-hosted AI agent crashed with "no user query found in messages" — on a request that demonstrably contained a user query. The cause was an upstream renderer that validates messages after the server…
Agents & AI · Cloud & InfrastructureTwo Cards, One Model, and the Ethernet Between Them
Pooling two Tesla T40s under llama.cpp's RPC backend bought a quarter-million-token context window on hardware I own — and cost half the generation speed, because every single token had to round-trip over plain TCP…
Software Donkey builds the agents and apps, and teaches small teams to do it too.
A standing team of six AI agents runs real work every day. The studio’s first apps are in private testing.
Tell me what you’re building.
Most projects begin with a conversation. I’ll tell you honestly whether I’m the right fit and what a first step looks like.