Under Review

Sound Sparks Motion

Audio and Text Tuning for Video Motion Editing

Temporarily anonymized Author names and affiliations are hidden during review.
Temporarily anonymized Code
Given a source video and an edit prompt, our method tunes the audio latent as a conditioning parameter to the LTX video model, such that the desired motion edit is realized in the edited video.
Given a source video and an edit prompt, our method tunes the audio latent as a conditioning parameter to the LTX video model, such that the desired motion edit is realized in the edited video.

Abstract

Motion-centric video editing remains difficult for large generative video models, which often respond well to appearance changes but struggle to produce specific, localized actions or state transitions in an existing clip. We introduce Sound Sparks Motion, a training-free framework that enables motion editing in an audio-visual video generation model by tuning its internal multimodal conditioning signals at test time.

Rather than modifying model weights, our method tunes only two lightweight variables: an audio latent derived from the source video and a residual perturbation in the text-conditioning. We find that this combination can encourage motion edits that the underlying model often struggles to realize under prompt-only control.

Since there is no direct way to evaluate temporal alignment between text and motion, we guide the tuning process using a vision–language model that provides feedback indicating whether the intended motion appears in the generated video. This simple supervision yields an effective semantic objective for motion editing, while regularization and perceptual-temporal constraints help preserve content and visual quality. Beyond per-video tuning, we show that the learned latent controls are transferable across videos, suggesting that they capture reusable motion-edit directions rather than overfitting to a single example.

Method

Our method turns a frozen retake-style video editor into a motion-aware editor through test-time latent tuning — no fine-tuning required.

01

Freeze the Model

The LTX-2 video generation model weights remain entirely frozen. We never backpropagate into the model parameters.

02

Optimize Conditioning

We tune two lightweight variables — an audio latent and a text residual — via gradient descent. Qwen2.5-VL provides a differentiable motion-alignment signal as supervision.

03

Apply via Retake

The optimized conditioning embeddings are injected into the Retake pipeline to regenerate only the target time window, preserving the rest of the video.

Architecture diagram showing the Sound Sparks Motion pipeline
Overview of the Sound Sparks Motion pipeline. The audio latent α and text residual Δv are optimized jointly while all model weights remain frozen.

Videos

The full supplementary video and the baseline comparison reel.

Supplementary video Method, qualitative results, transfer, and generalization to H3 · 6:05
Baseline comparison 12 scenarios against 7 baselines, plus the H3 study · 6:40

Results on LTX-2

Hit Spark the motion! on any card to apply its edit.

Generalization to H3

Percentages are from a human study of 25 scenarios with 15 participants (375 judgements): on each card, how often ours was preferred, judged more natural, and judged the stronger edit, against the same for H3 alone. Ties make up the rest. These clips carry sound.

Audio, text, or both

The audio latent alone, the text residual alone, and the two together, beside H3 with no editing. The reel opens with the human study over all 12 scenarios (15 participants), then shows the 5 where the combination won clearly.

Audio, text, or both The two tuned signals ablated on H3, with the human study · 1:57

Baseline comparison

Twelve scenarios against seven baselines. Pick a scenario to compare. In a study of 21 participants, our result was ranked first in 56.3% of trials, against 21.4% for the next best method.

How it compares

Measured against seven recent video-editing systems on the same 12 motion-edit scenarios — judged both by people and by a vision–language model.

Human evaluation

Win rate

21 participants · 12 scenarios · 252 judgements

  • Sound Sparks Motion 0.0%
  • LoRA-Edit 0.0%
  • Runway-Aleph 0.0%
  • LTX2.3 Retake 0.0%
  • UniEdit 0.0%
  • Kiwi-Edit 0.0%
  • VACE-Wan 0.0%
  • VACE-LTX 0.0%

Fraction of first-place selections.

2.6×more first-place votes than the strongest baseline.
VLM evaluation

Task success

GPT-5.5 as judge · 12 scenarios

  • Sound Sparks Motion 0.0%
  • LTX2.3 Retake 0.0%
  • Runway-Aleph 0.0%
  • LoRA-Edit 0.0%
  • UniEdit 0.0%
  • Kiwi-Edit 0.0%
  • VACE-Wan 0.0%
  • VACE-LTX 0.0%

Fraction of scenarios where the edit was achieved.

+33.3points of task success over the strongest baseline.

Motion transfer

The audio latent and text residual tuned on one clip, applied to another. Hit Transfer the motion! to carry them across.

The boy is laughing out loud.
Optimized onA man laughing out loud
audio latenttext residual
New input
A cat yawning.
Optimized onA dog yawning
audio latenttext residual
New input
A man shouting.
Optimized onAn angry man shouting
audio latenttext residual
New input
A red bird opening its wings.
Optimized onA bird opening its wings
audio latenttext residual
New input

Where each signal attends

Average cross-attention for the two tuned signals, over the first frame of the result. A2V is audio-to-video, T2V is text-to-video; switch between them on any card.

"A bird opening its wing."
"Turtle extending its neck."
"A goldfish jumps out of the fish tank into the air."
"The man climbs up the rock."
"A man is laughing out loud."
"The man pets the dog."
"The robot waving its hand."
"A dog yawning."

Ablations

Every variant of a scenario plays at once. Each reel replays its first scenario at half speed.

Effect of the supervision signal CLIP vs. XCLIP vs. a vision-language model, on 4 scenarios · 1:31
Effect of the regularizers None, LPIPS, temporal, latent, and all three, on 4 scenarios · 1:26

BibTeX

Temporarily anonymized for review.