How to replace the sky in a video so the new sky moves with the camera
Ask Vizard Agent to replace only the sky and say it must move with the camera. It masks the sky in every frame, tracks how the camera turns from features in the sky area, renders the new sky for each frame from that rotation, and cleans the edges around trees, wires and anything else that cuts into the sky.
What do you type into Vizard Agent?
Say what the new sky is and that everything else stays. The detail that changes the method is movement: a sky that should stay fixed in the world while the camera pans needs the camera's motion worked out, while a sky that only needs to look different can be a simpler swap.
Make the sky deep blue like space, with stars, planets and a giant black hole, without changing anything else. When the view moves, the stars, planets and black hole should move with it.
What does Vizard Agent actually do?
It treats the sky as being infinitely far away, so only the camera's rotation matters, not its position. That turns the job into a camera solve plus a panorama, which keeps the new sky locked to the world. In the session behind this article it:
- Built a sky mask for every frame and a wide panorama for the new sky, with thousands of extra stars.
- Tracked features only inside the sky, with the on-screen interface excluded, and fitted the camera's left-right and up-down turn for every frame.
- Rendered the sky for each frame from that rotation, and caught a bug where rotations were compounding by sending identical angles to every frame and checking the frames matched.
- Refined the mattes to remove a pale fringe of the old sky around trees and armour, and kept the interface overlays exactly as they were.
In a second session, on a handheld phone clip with overhead wires, cutting the sky out left a halo, so the agent used a generative edit told to change only the sky instead; it accepts clips up to ten seconds, so the clip was sped up to fit and slowed back.
When does this not work well?
When the sky is full of thin detail. Wires, bare branches and hair leave halos that are hard to clean, which is why the second session switched methods. That route has its own cost: a clip squeezed under the ten-second limit and stretched back has fewer distinct frames, so motion looks less smooth.
Common questions
Why does my new sky slide with the frame? It is a still. Vizard Agent renders it from the camera's rotation.
What about trees and wires? Vizard Agent refines the edges, but fine detail is the hard part.
Will the rest of the video change? No. Vizard Agent replaces only the masked sky.