LTX, H3, Music3 and ACE-Step: what I learned

Choose the control you actually need

I start with the part of a piece that must remain mine: a spoken line, a recognizable character, a melody, a particular rhythm of cuts. Different tools expose different controls. A model that produces a convincing first sample can still be the wrong tool when the fifth shot must match the first.

LTX became a laboratory for short, speech-led video: identity references, voice references, guidance settings, and the awkward transition from one generated clip to the next. H3 became especially useful for narrative scenes with native sound and recurring characters. Neither removes the need to inspect an actual performance.

What I mean by Karpathy-style autoresearch

Karpathy’s autoresearch repository gives an agent a narrow editable training program, a fixed experiment budget, and a validation metric. It tries a change, evaluates it, and keeps or discards the result. The important idea for my work is that a proposed improvement has to survive a repeatable experiment.

My rendering work adapts that pattern; it does not run the unmodified repository or retrain H3 and LTX after every clip. The LTX autoresearch code defines configurations, generates candidates, records quality observations, and writes ranked results. Successive runners explored different parameter grids and assembly strategies. Humans and agents used those findings to revise the next experiment.

The search space included identity, video and audio guidance; seed; frame count; resolution; reference images and audio; LoRA strengths; and how multiple clips were assembled. That is much more specific than asking an agent to “make the video better.” A useful trial names its hypothesis before it spends GPU time.

LTX: the evaluator became part of the experiment

The early LTX records contain dozens of experiments, with separate attention to faces, speech, motion and assembly. One April 5 set found no transcription errors on an 18-word line and a word error rate of 0.083 on a 22-word line. That was evidence about those clips and settings—not a universal 18-word limit. It suggested testing the speech budget rather than endlessly changing a visual prompt.

The same record found that an early audio-quality heuristic did not distinguish an improvement that listening made apparent. That motivated bringing in a learned speech-quality estimator. The evaluator itself needed evaluation: a number that stays flat while the experience changes is not a useful optimization target.

Later experiments shifted attention from individual frames to assembled sequences. Better face scores did not prevent worse transitions. I began treating the cut as its own artifact: compare identity across the boundary, inspect motion discontinuity, and watch the result in sequence. A collection of individually good shots is not automatically a coherent video.

These findings primarily come from the LTX 2.3 workflows. I have also brought LTX 2.5 into the local stack, but installing a newer model does not retroactively make the older measurements results for that model.

H3: close the loop around the rendered artifact

H3’s evaluation path starts with cheap file and signal checks, then asks whether the speech and sound match the intended scene. The post-render hook records results beside the output. More expensive visual or specialist checks are selective, not something I assume ran just because a video exists.

The useful next action depends on the failure. An isolated bad take may justify a fresh seed. Repeated unwanted narration calls for a different prompt or recipe. A broken script extractor calls for a harness repair, not another render. My operating notes generally allow a small two- or three-candidate retry budget before changing the explanation.

This is an evaluator-guided production loop, not a claim of fully autonomous creative self-improvement. Some retry paths remain driver-specific. Missing checks stay visible as missing evidence, and an acceptable technical result can still be a scene I do not want in the film.

Music taught me to separate generation from composition

ACE-Step made it possible to turn Fable’s written lyrics into a performed song and to explore different vocal arrangements. Long Exposure taught a different lesson: one long generated score became too uniform. Shorter chapter cues, selected and arranged across the film, gave the edit more shape. The improvement came from composition and selection, not from generating a longer file.

Music3 has been valuable for song and remix exploration. But a reference that influences musical material is not a guarantee of vocal identity. In my own-voice experiments, the more useful path separated song generation, vocal-stem cleanup, and voice conversion instead of expecting every part of the task to be controlled by one prompt or adapter.

A DiT-only Music3 LoRA experiment did not transfer the vocal identity we were testing. That was a valuable negative result: successfully loading trained weights was not evidence that I had gained control over the desired characteristic. Voice-similarity measurements needed listening and matched references, not interpretation as an identity percentage.

Second Winter moved toward explicit composition: lyrics and melody were specified together, then realized through SoulX-Singer. ACE-Step’s continuation modes were not a drop-in way to preserve that already-composed vocal; the experiments regenerated it. Keeping an independent instrumental bed was the more controllable arrangement.

The supporting tools often determine whether a piece works

In my tested LTX workflows, Qwen-Image-2512 produced AI-generated reference faces that survived lip-sync rendering where several other image sources did not. I treat that as a tested compatibility result, not a general ranking of image models. A beautiful still and a useful animation anchor are different things.

Higgs supplied a voice for Tell Me Again, while ink supplied branching choices. That changed the audience’s role from watching to responding. SeedVR2 belongs later in the workflow, when finishing and resolution matter; it cannot rescue a performance whose timing or expression is wrong. ComfyUI is useful for connecting stages, not as a substitute for evaluating their outputs.

Optimize for an accepted piece, not a winning metric

A June take-selection experiment made the warning concrete: the better lip-sync score came with worse mouth noise. Optimizing one dimension had selected a less pleasant performance. My response is to preserve candidates and their component scores so I can change the selection rule without paying to regenerate everything.

The objective I care about is time and effort per accepted scene, song, or finished piece. I want repeatable checks, comparison against the existing recipe, and a final watch-and-listen pass. When a result changes the way I work, the wiki records the tested scope, the failure mode, and what the next agent should do differently.

That is the practical value of owning the local stack and working with inspectable tools: I can investigate the failure instead of accepting it as a mysterious property of a product. Automation makes more exploration possible. Editorial judgment gives that exploration a direction.