How to Make a Mask in After Effects
StyleFrame Team7 min read
To make a mask in After Effects, select a layer, choose a shape tool (Q cycles through them) or the Pen tool (G), and draw a closed path directly in the Composition panel. The closed path becomes a mask that controls which pixels of that layer are visible. If no layer is selected when you draw, After Effects creates a shape layer instead of a mask. Once the mask exists, press M to reveal its path, then adjust mode, feather, opacity, and expansion, or keyframe the path to animate it.
What a Mask Actually Does
A mask is a vector path attached to a layer that modifies that layer's alpha channel. Inside the path the layer stays visible, and outside it the layer becomes transparent. You can invert that behavior or combine several masks with different modes.
Because the mask belongs to the layer, it moves, scales, and rotates with the layer's Transform properties. That is useful for cutouts and reveals. It is a limitation when you want a shape that stays fixed while the footage moves underneath, and the track matte section below covers that case.
An open path does nothing to the layer's visibility. Only a closed path cuts out pixels, so always close the shape by clicking the first point or by finishing a shape-tool drag.
Create a Mask with the Shape Tools
The shape tools are the fastest route for geometric masks such as vignettes, rounded panels, and window reveals.
- Select the layer in the timeline. This step matters, because it decides whether you get a mask or a shape layer.
- Choose the Rectangle, Rounded Rectangle, Ellipse, Polygon, or Star tool. Press Q repeatedly to cycle through them.
- Click and drag in the Composition panel.
A few modifiers speed this up:
- Shift constrains the shape to equal sides, which gives you a square or a circle.
- Ctrl/Cmd draws from the center instead of from a corner.
Shape-tool masks are fully editable afterward. Select the layer, press M, and you can change the path points with the Selection tool (V) like any other mask.
Create a Mask with the Pen Tool
For anything organic, such as a person, a product, a window frame, or a rock, use the Pen tool.
- Select the layer.
- Press G to activate the Pen tool.
- Click to place corner points. Click and drag to pull out Bézier handles and create curves.
- Close the path by clicking the first point.
Work at a high zoom level and place fewer, better-positioned points rather than many rough ones. A mask with clean curves and minimal points is easier to feather, animate, and revise later.
Some editing tricks worth knowing:
- Hold Alt/Option and drag a handle to break it, so one side of the point moves independently. This is useful at sharp transitions.
- Hold Shift while placing points to constrain segments to an axis.
- Select points with the Selection tool and nudge them with the arrow keys for precise cleanup.
Understand the Mask Properties
Select the layer and press M to show Mask Path. Press M twice quickly (MM) to reveal all mask properties. Each does something different.
Mask Mode
The mode defines how a mask interacts with other masks on the same layer. The common ones are:
- Add: reveals the area inside the path. This is the default.
- Subtract: cuts the area out of the masks above it.
- Intersect: shows only where masks overlap.
- None: the mask is ignored for visibility, which is useful when you want the path for something else, like an effect's reference or a stroke.
Lighten, Darken, and Difference also exist and matter mostly when combining masks that have different opacities. Mask order in the stack affects the result, so reorder masks if the outcome looks wrong.
Mask Feather
Feather softens the edge. A small value often makes a composite more believable, because hard vector edges rarely match the softness of real footage.
Mask Opacity
This controls how strongly the mask cuts the layer. At 100% the mask has full effect. Lower values blend the masked region back toward the unmasked layer, which is useful for subtle shaping.
Mask Expansion
Expansion grows or shrinks the mask edge by a pixel amount without redrawing the path. It is the quickest way to fix a matte that eats into an edge or leaves a halo. Combined with a small feather, it handles most edge cleanup.
Inverted
The Inverted checkbox flips the mask so the outside of the path is visible and the inside is transparent. It is the standard way to punch a hole, such as cutting out a window to show background footage.
Animate a Mask
Masks become powerful when they move. Open the layer's Mask Path property and click the stopwatch to start keyframing. Move the playhead, then reshape the path with the Selection tool to set a new keyframe.
Some practical habits help here:
- Keep the same number of vertices across keyframes. After Effects interpolates between corresponding points, so inconsistent point counts lead to odd morphing.
- Set key poses at significant moves and let interpolation handle the in-betweens. Add keyframes only where the shape drifts off the subject.
- Use easing on Mask Path keyframes so shape changes don't feel mechanical.
- Animate Mask Expansion and Feather for wipes, reveals, and focus-pull-style effects.
A classic example is a text reveal. Draw a rectangle mask over a text layer, keyframe the Mask Path so the rectangle grows, and the text appears as the edge sweeps across it. The same result is possible with a shape layer used as a track matte, which is better if you want to animate the shape independently of the text.
Tracking a Mask to Moving Footage
If your subject moves, a static mask won't hold. Use mask tracking to follow the movement, then check the result frame by frame. Automated tracking rarely survives occlusion, changing shapes, or motion blur without manual correction, so budget time for tweaking keyframes. For complex subjects, automated selection tools can often generate a starting matte faster than hand-drawing every frame. Expect to refine edges either way.
Mask vs. Track Matte vs. Shape Layer
Choosing the right structure saves rework.
- Use a mask when the cutout belongs to a single layer and should move with it.
- Use a track matte when you want a separate layer, such as a shape or text, to define visibility on another layer. The matte can move independently, and one matte can be reused conceptually across setups.
- Use a shape layer when you need parametric shapes, repeaters, strokes, or path operations that go beyond what a mask offers.
Recent After Effects versions also let you limit individual effects to a mask, so you can restrict a blur or a color effect to part of a layer without an adjustment layer. This is worth testing when adjustment layers fight your parenting setup.
Common Problems and Fixes
The mask doesn't appear; a shape layer was created instead. You drew with no layer selected. Undo, select the target layer, and draw again.
Nothing is being cut out. Check that the path is closed and that the mask mode isn't set to None.
The edge looks jagged or has a fringe. Add a slight feather, reduce Mask Expansion by a pixel or two, and check for color spill at the edge. A fringe often needs a choke or a color correction on the edge rather than a different mask.
The mask is in the wrong place after moving the layer. The mask is tied to the layer's pixels. Either move the mask path with the layer intentionally, or switch to a track matte if you need independence.
Interpolation looks warped. You probably added or removed vertices between keyframes. Rebuild with consistent point counts.
Quick Reference
- Select the layer first, then draw.
- Q cycles shape tools, G selects the Pen tool.
- Shift constrains, Ctrl/Cmd draws from the center.
- M shows Mask Path, MM shows all mask properties.
- Close the path or nothing is cut out.
- Use feather and expansion to blend edges.
- Keyframe Mask Path to animate, keeping vertex counts consistent.
- Choose a track matte or shape layer when the cutout needs to move independently.
Once these basics are in muscle memory, masking becomes one of the fastest ways to isolate, reveal, and combine elements, and the foundation for most compositing work in After Effects.