https://www.sethpratt.com/writing Notes on evals, agents, inference and the Lab, with the numbers left in. en-us https://www.sethpratt.com/writing/testing-a-vr-gamehttps://www.sethpratt.com/writing/testing-a-vr-gameSun, 11 Oct 2026 16:00:00 GMTA VR game with half a million bot episodes and four billion invariant checks a night, the headset sessions where nearly every problem escaped them, and how each escape became a new eval. https://www.sethpratt.com/writing/watching-a-language-model-thinkhttps://www.sethpratt.com/writing/watching-a-language-model-thinkSat, 10 Oct 2026 16:00:00 GMTWe pulled real numbers out of Qwen language models and turned them into films, VR scenes and the live figures on this page. Some pictures showed real mechanisms; some were artefacts of the drawing; one well-known result didn’t reproduce at all. https://www.sethpratt.com/writing/running-a-team-of-ai-agentshttps://www.sethpratt.com/writing/running-a-team-of-ai-agentsSat, 10 Oct 2026 16:00:00 GMTThree weeks of two lead AI agents and their helpers, told as six stories about clear asks, interruptions, handoffs, ownership, quality and where the human stays. https://www.sethpratt.com/writing/when-the-metric-says-yeshttps://www.sethpratt.com/writing/when-the-metric-says-yesSat, 10 Oct 2026 16:00:00 GMTThe automatic gates every render passes, what they caught, the bugs inside the gates themselves, and a nebula that beat its shape metric and still looked like a sock puppet. https://www.sethpratt.com/writing/one-sim-seven-platformshttps://www.sethpratt.com/writing/one-sim-seven-platformsSat, 10 Oct 2026 16:00:00 GMTHow AI agents turn a web prototype into native code for Vision Pro, iPhone, iPad, Mac, Apple TV and Quest without porting it twice, and the tests that make that safe. https://www.sethpratt.com/writing/tuning-a-game-with-ai-playtestershttps://www.sethpratt.com/writing/tuning-a-game-with-ai-playtestersSat, 10 Oct 2026 16:00:00 GMTTwenty-one revisions of a VR physics game, scored by cold AI tester panels and noisy bots: what moved the scores, where the judges disagreed, and what only a human caught. https://www.sethpratt.com/writing/autoresearch-at-homehttps://www.sethpratt.com/writing/autoresearch-at-homeSat, 10 Oct 2026 16:00:00 GMTEight loops that generate candidates, score them against evals on local GPUs and keep the winners, from video and games to strange attractors and artificial life, and every place the scorer turned out to be wrong. https://www.sethpratt.com/writing/chaos-with-nothing-to-orbithttps://www.sethpratt.com/writing/chaos-with-nothing-to-orbitFri, 09 Oct 2026 16:00:00 GMTA seven-term chaotic system with no resting point anywhere in space, found by a GPU search and checked at 64-bit precision, and what we can and can’t claim about it. https://www.sethpratt.com/writing/create-constrained-generationhttps://www.sethpratt.com/writing/create-constrained-generationTue, 06 Oct 2026 16:00:00 GMTHow Everdust’s Create turns a sentence into a playable game: the model writes data from registered parts, and a validator proves every game can be won and lost before anyone sees it. https://www.sethpratt.com/writing/tuning-autotalenthttps://www.sethpratt.com/writing/tuning-autotalentTue, 29 Sep 2026 16:00:00 GMTHow AutoTalent’s sourcing agent was tuned with measurements: a live production test, where each round’s time and money went, a fast judgment model that became a gate instead of a scorer, and a coding agent that overbuilt. https://www.sethpratt.com/writing/private-local-ai-assistanthttps://www.sethpratt.com/writing/private-local-ai-assistantThu, 24 Sep 2026 16:00:00 GMTWhat a clean-install test with network captures found in a local AI app built for privacy, what it took to fix, and how a benchmark picked its model. https://www.sethpratt.com/writing/local-agent-evaluationshttps://www.sethpratt.com/writing/local-agent-evaluationsThu, 17 Sep 2026 16:00:00 GMTLessons from repeated coding tasks, model adaptation, and a grading bug that changed the result without changing the answer. https://www.sethpratt.com/writing/ambient-clinical-documentation-open-weightshttps://www.sethpratt.com/writing/ambient-clinical-documentation-open-weightsWed, 16 Sep 2026 16:00:00 GMTWhat seven speech recognizers, a LoRA fine-tune and a claim-level verifier did on open mock consultations, and where fluency metrics hide the damage. https://www.sethpratt.com/writing/finding-song-starts-in-dj-mixeshttps://www.sethpratt.com/writing/finding-song-starts-in-dj-mixesTue, 15 Sep 2026 16:00:00 GMTWhat eight public mixes and 190 DJ-published timestamps taught me about fingerprints, transition detectors, and the limits of “correct”. https://www.sethpratt.com/writing/fable-creative-experimentshttps://www.sethpratt.com/writing/fable-creative-experimentsThu, 03 Sep 2026 16:00:00 GMTThe films, songs, and interactive stories that emerged when I gave an AI collaborator creative freedom on my local hardware. https://www.sethpratt.com/writing/creative-ai-tools-autoresearchhttps://www.sethpratt.com/writing/creative-ai-tools-autoresearchThu, 03 Sep 2026 16:00:00 GMTWhere each tool proved useful, where it fell short, and how evaluators guide my next creative experiment. https://www.sethpratt.com/writing/local-model-performance-engineeringhttps://www.sethpratt.com/writing/local-model-performance-engineeringThu, 27 Aug 2026 16:00:00 GMTMeasured tradeoffs in local inference, from Blackwell compatibility and speculative decoding to more trustworthy evaluations. https://www.sethpratt.com/writing/long-running-ai-agentshttps://www.sethpratt.com/writing/long-running-ai-agentsTue, 25 Aug 2026 16:00:00 GMTHow I design task-specific agents that can resume work, recover from failures, and know when to ask for help. https://www.sethpratt.com/writing/evaluating-agentic-videohttps://www.sethpratt.com/writing/evaluating-agentic-videoSat, 22 Aug 2026 16:00:00 GMTWhat my rendering tests revealed about dialogue, lip sync, visual continuity, and selecting better takes. https://www.sethpratt.com/writing/refusal-ablation-measuredhttps://www.sethpratt.com/writing/refusal-ablation-measuredFri, 21 Aug 2026 16:00:00 GMTWhat removing the refusal direction from a mixture-of-experts model cost in capability, which implementation choice decided it, and what a community “abliterated” upload actually did.