After Effects Plugin for Sky Replacement
StyleFrame Team8 min read
After Effects has no single built-in sky replacement plugin, so most artists combine a keyer, a tracker, and a roto tool. A common stack is Boris FX Primatte Studio (or the built-in Keylight) to pull the sky matte, Mocha Pro (or the bundled Mocha AE) to track the camera move, and Roto Brush for hard edges. The replacement sky is then tracked, color-matched, and edge-blended into the plate. The plugins do the matte and the track, and the realism comes from how you build and integrate the new sky.
What a Sky Replacement Plugin Actually Needs to Do
Sky replacement is three problems at once, and it helps to judge any plugin by which one it solves:
- Separation: isolating the sky from everything in front of it, including fine edges like hair, leaves, and wires.
- Motion matching: making the new sky follow the camera so it doesn't slide or float.
- Integration: matching color, contrast, haze, and light behavior so the sky belongs to the shot.
No single tool handles all three well on every shot. That's why working compositors treat this as a small toolkit rather than a product category.
The Plugin and Tool Stack for After Effects
Keying: Primatte Studio, Keylight, and Ultra Key
For sky separation, a keyer is the usual first pass. Keylight and Ultra Key ship with After Effects. You sample the sky color with the eyedropper, then refine Screen Gain, Screen Balance, and Clip Black/White until the matte holds.
Boris FX's Primatte Studio is a third-party option with a visual workflow. You select the sky with a point or a rectangle, can sample clean background areas to pull more color into the key, and can use clean foreground controls to protect edge detail and light wrap. Its Final Matte view helps you judge the composite matte before committing.
Skies are rarely uniform. Gradients, haze near the horizon, and blown-out sun areas make a single key unreliable. Use the key for the bulk of the sky and handle the exceptions separately.
Tracking: Mocha Pro and Mocha AE
A static sky image pasted behind a moving shot is the fastest way to expose a fake. Planar tracking solves this. Mocha Pro can be applied to the sky layer directly in After Effects. A lighter Mocha AE is included with Creative Cloud (select a layer, then Animation > Track in Mocha AE).
Practical tracking tips:
- Track a stable, high-contrast area of the plate, such as distant trees or a ridge line, even if it's slightly out of focus.
- Start with translation, scale, and rotation only. Add perspective only if the shot needs it.
- Apply the track data to the replacement sky, not the other way around, unless you're stabilizing.
A "3D offset" technique, pushing the sky back in Z space, helps parallax feel believable when the camera moves.
Roto: Roto Brush and Manual Masks
When a subject overlaps the sky and the key struggles, Roto Brush can separate the subject. Feather the resulting matte to soften the edges. Manual masks also work well for localized fixes, such as keeping a sun flare or a bright patch of original sky you want to preserve.
Supporting Effects
These effects often make the difference between a sky that looks pasted in and one that looks real:
- Fractal Noise: a flexible way to build or augment cloud layers procedurally.
- Displacement maps and edge blurs: soften the transition between foreground and sky.
- Lens flare: adding a flare, for example from Video Copilot's tools, can unify the shot and add drama.
- Curves and other color correction: essential for matching black levels, contrast, and color temperature.
A Reliable Sky Replacement Workflow in After Effects
Here is a workflow that holds up across most shots.
- Prep the plate. Import the footage, create a comp that matches its frame rate and resolution, and name the layer "original plate." Duplicate it so you can key the copy while keeping a clean reference.
- Pull the key. Apply your keyer to the duplicate. Check the matte view, not just the composite, and watch for edge chatter over time.
- Clean up with roto. Use Roto Brush or masks for areas the key misses, like sun glare, thin branches, or tonal shifts near the horizon.
- Place the new sky below the plate. Make sure the replacement sky layer sits under the keyed footage.
- Track. Use Mocha to track the plate and apply that data to the sky layer. Offset the sky in Z if parallax needs it.
- Integrate. Match brightness, contrast, and color. Add edge blur or light wrap so foreground edges pick up sky color. Add subtle atmospheric haze near the horizon.
- Review in motion. Scrub through the shot at full resolution. Edge issues and sliding often only show up in playback.
- Render. Send the comp to the render queue once the matte, track, and grade hold across the whole shot.
Building a Sky That Holds Up
Matching the shot is only half the job. The sky itself has to look convincing, especially if clouds need to move.
A single still image can work for a locked-off shot with a minor camera move. For anything more dynamic, a static plate will feel flat. Experienced compositors suggest building skies from multiple sources instead of relying on one:
- Layer several cloud elements and transform each one differently so repetition isn't obvious.
- Combine timelapse footage with procedural elements like Fractal Noise.
- For dramatic effects, like storm clouds rolling in from the horizon, patch together an animated comp from several sources. Physics-based particle or fluid simulations are an option, and some artists shoot their own cloud footage when a shot demands it.
Lighting should also match. A sky with a low-angle golden sun won't work behind a subject lit by flat overcast light. Choose a replacement with a similar direction, softness, and color temperature, or plan to adjust the foreground grade to suit.
Where Styleframe Fits in a Sky Replacement Pipeline
Styleframe isn't a keyer or a tracker, so it doesn't replace the plugins above. It's an AI video and motion tool built for professional artistic control, and it's most useful for creating the sky itself before you bring it into After Effects or Nuke.
Here are practical ways it can contribute:
- Keyframe-driven sky plates: With Frames-to-Video, you define the start and end of a sky, such as a clear sky and a fully overcast one. Styleframe generates the transition between your keyframes, so you direct the progression rather than hoping a prompt lands.
- Reference-driven looks: Restyle uses reference images and clips for style transfer. If you have a reference grade or a painterly sky you want to hit, you can steer toward it.
- Production-ready output: 4K Upscale helps bring generated sky plates up to delivery resolution.
- Non-destructive iteration: Every generation is driven by your references, clips, and keyframes, and you can iterate without overwriting earlier versions.
Styleframe exports image sequences, which drop into After Effects or Nuke like any other plate. You keep your compositing workflow: generate the sky, bring it in, then key, track, and integrate with the tools you already trust.
Generated skies still need the integration work described above. Grade, edge treatment, and tracking remain your job, and they're what make the result look photographic.
Common Problems and Fixes
The edges glow or fringe. Edge contamination from the original sky is the usual cause. Choke the matte slightly, use edge correction in your keyer, and add light wrap from the new sky.
The sky slides against the horizon. The track is likely locked to the wrong plane. Pick a more distant, stable feature and consider a perspective track.
The sky looks too sharp or too clean. Real skies pass through lens softness, atmospheric haze, and sensor noise. Add slight blur, match grain, and lift the blacks near the horizon.
Flicker in the matte. Skies change subtly frame to frame. Stabilize the key with temporal smoothing, matte cleanup, or a roto-assisted approach in problem areas.
Lost detail in the sun or flare area. Keep it. Mask out the original highlight and let it sit over the new sky, then color-match the surroundings to it.
Choosing the Right Approach for Your Shot
Final Thoughts
The best after effects plugin for sky replacement is usually a combination: a keyer for the bulk of the matte, a planar tracker for motion, and roto for everything in between. Keep the sky believable by layering sources, matching light, and treating edges carefully. If you need a custom sky with real direction over how it evolves, a tool like Styleframe can help you generate it, and the final composite still happens in the tools you already use.