EgoPlay introduces event-triggered video editing for egocentric streams: given a prompt like "when X happens, do Y," the model must detect the trigger, preserve pre-event frames, and edit only the post-event continuation. We build a 106K-pair event-editing dataset, train an end-to-end video diffusion editor, and improve over EgoEdit by 25.2% in editing quality.
We compare EgoPlay with the four baseline methods used in the Ego4D event-triggered benchmark. Each card shows the source clip and the GT target alongside the four method outputs, ranked within its mode by EgoPlay's VLM editing-quality score.
EgoPlay also preserves standard egocentric video editing ability when no event trigger is needed.
Edit prompt: add sponge in hands
Edit prompt: Replace the paperback held open by two hands with a frostbound mirror exhaling cloud of vapor, a slightly concave, almost-rectangular pane whose center is a milky, pale-blue reflective surface mottled by shifting, cloud-like vapor, its frame encrusted in jagged, snowflake-like ice crystals and a glittering white‑silver frost that emits a soft bluish glow and drifting particle sparks, with thin wisps of vapor pouring and curling out from the frosty edges.
Edit prompt: Seamlessly transform the paperback book held open in hands into an oak music box with an ornate carved lid and warm brown polished wood, compact and hand-sized, its rim and top overgrown in thick, velvety moss that glows a vivid teal‑cyan, throwing an eerie green light from within the box and producing scattered pinpoints of bioluminescent glow and soft, misty tufts that accentuate the contrast between the smooth wood, metal clasp, and the plush, organic texture.
Edit prompt: Transform the paperback opened in hands into a manila/tan folder, slightly overstuffed with a stepped top edge formed by multiple tabbed dividers; give it a matte paper texture, a small white rectangular label bearing fine printed text, a metal prong fastener visible along the spine, and soft diffuse lighting that creates gentle shadows and a subtle curvature across the surface.
Edit prompt: Change the kitchen to a small ramen shop counter with a wooden counter, hanging ladles, and warm paper lantern light.
Edit prompt: Transform the video into psychedelic poster art style
We conducted a pairwise preference study with 10 participants. Each participant completed 30 trials comparing EgoPlay against other baselines across five event-triggered editing scenarios. In each trial, participants were shown the ground-truth target alongside two anonymized method outputs and asked to indicate a preference or tie on instruction following, edit timing, edit quality, and overall preference.
EgoPlay is preferred over all baselines across every criterion. All reported win rates significantly exceed the 50% chance level under Wilson 95% confidence intervals.
Click a baseline to view win/tie/loss rates for instruction following, edit timing, edit quality, and overall preference.
EgoPlay takes a source video and an event-triggered instruction, then jointly infers when the trigger event occurs and what edit should appear afterward. The bidirectional model learns this behavior end-to-end from event-triggered V2V supervision, while the causal variant restricts temporal attention for streamable inference.
EgoPlay conditions a video diffusion editor on the source clip and event-triggered prompt, enabling trigger-aware preservation and post-event editing.
We build event-triggered training data from raw videos and annotated event spans. The pipeline filters clips, generates positive, negative, and event-free instructions, then synthesizes edited targets by applying the requested edit to the post-event segment while preserving the pre-event prefix.
The resulting dataset contains 106K event-triggered clip-prompt pairs covering positive triggers, fabricated-trigger negatives, and multi-event prompts.