Footage and sound

Sound, music and sound effects

Bring in music and voice, set levels in decibels, fade, pan and process sound, place library sound effects on a cut, and check the mix.

This page covers everything audible in a film: footage's own sound, uploaded music and voice, library sound effects, the levels, fades, pan and effects that shape them, and how to check a mix you can't listen to.

How sound works here

A layer's sound belongs to the layer, as in After Effects, so trimming, moving or speeding up a video carries its sound with it. The engine describes the mix as a plan (which file, which slice, when, at what gain, shaped how), and the export and the editor's player both read that plan.

A layer is in the plan when it holds an audio file, a video with sound or a composition with sound, its switches permit it and its source advances. A flat level of exactly zero leaves it out unless volume keys can raise it; a track at exactly zero stays silent even with layer keys. Positive gains and source advances remain in the plan, however small. The source samples and final delivery's precision still determine what is heard. While any sounding layer is soloed (soloLayer), only soloed sounds are mixed, in the export too. ymotion_render {"what":"waveform"} reports that plan and measures the delivered sound: silence when there is no sound, a copied source stream when the export copies one, or the mix graph the export builds. It is the quickest answer to "why can't I hear this?"; ymotion_render {"what":"audio"} prints the plan too, beside a mixed file.

Bringing sound in

Music and voice-overs come from the person's uploads, like footage. List what's there and place it by id; if what you need is missing, bring it in from the chat (Getting a file in) or ask the person to upload it at /uploads.

ymotion_media {"action":"list"}
ymotion_media {"action":"place","id":"<upload id>","start":0}

A sound lands on an audio track, made if there isn't one. Sound effects come from a shared library instead, described below.

Levels, fades and pan

Levels are in decibels, the unit a fader uses. 0 dB is the file as it is, -6 is half the amplitude and +6 double:

ymotion_edit {"ops":[{"verb":"setLevel","args":{"layerId":"<layer id>","db":-18}}]}

Underneath, the level is stored as a multiplier, audio.volume, from 0 to 4, so the ceiling is about +12 dB. setVolume sets that multiplier directly, and keyframes on audio.volume use it too: -6 dB is about 0.5, -12 dB about 0.25 and -18 dB about 0.125. A layer at volume 0 is left out of the mix, which is how to silence a video's own sound. setTrackVolume multiplies a whole track ({"id":"<track id>","volume":0.5}), and muteTrack silences one ({"id":"<track id>","muted":true}).

setFades sets both ends at once, in seconds, so pass 0 for an end with no fade, or its current length to keep it. Pan runs from -1 (left) to 1 (right):

ymotion_edit {"ops":[{"verb":"setFades","args":{"layerId":"<layer id>","fadeIn":1.5,"fadeOut":2}}]}
ymotion_edit {"ops":[{"verb":"setProp","args":{"layerId":"<layer id>","path":"audio.pan","value":-0.4}}]}

Two things about the pan law. At exactly 0 both sides play at full level, while anywhere else the two sides follow a constant-power curve, so moving a pan even slightly off centre makes the sound about 3 dB quieter, and a keyed pan that leaves or arrives at 0 steps by that much there. And a pan scales each side rather than moving the sound between them: a mono sound is placed, but on a stereo sound the far side is turned down, so whatever is only on that side fades with it. To move a whole stereo sound to one side, fold it to mono first with the stereo mixer effect (0.5 of each side into both), then pan.

The chain runs in After Effects' order: level, then audio effects, then fades, then pan, so a compressor never pulls a fade back up.

Levels that change over time

Volume, pan and the knobs of audio effects animate like any other property, with key times in seconds from the layer's own start. Fades are lengths rather than levels: keys on audio.fadeIn and audio.fadeOut change that duration at each sample, through their own curve and the layer clock. The multiplier is the progress through that end divided by its current duration, never longer than what is heard; a duration of 0 turns that fade off. A pan across four seconds:

ymotion_edit {"ops":[{"verb":"keyRamp","args":{"layerId":"<layer id>","path":"audio.pan","from":-1,"to":1,"opts":{"start":0,"end":4}}}]}

Example: ducking music under speech

Lower the music while someone talks and bring it back in the gaps, so the voice is easy to follow. Find the speech in the transcript or the waveform (a transcript's times are the file's own, see What an upload says); here a line runs from 2 to 5 seconds and the music starts at 0, so layer and timeline seconds agree:

ymotion_edit {"label":"duck the music","ops":[{"verb":"putTracks","args":{"layerId":"<music layer id>","tracks":[{"path":"audio.volume","keys":[{"t":1.7,"value":0.5},{"t":2,"value":0.125},{"t":5,"value":0.125},{"t":5.4,"value":0.5}]}]}}]}

The music sits at about -6 dB between lines and -18 dB under the voice, with short ramps so the change isn't heard as a jump. For a music layer that starts later, subtract its start from each time.

Volume retains its original eases, holds, loops, wiggles and posterized time, and larger key lists keep their motion rather than being averaged. Short plain linear lists of at most 32 volume keys retain their existing audio-frame expression; shaped and longer level curves are read per sample. The export, waveform and editor share that curve and the layer clock. The track fader multiplies the layer's level. Volume runs before the effects, and fades after them; a Backwards effect reverses the level ahead of it, while the fades stay on the output clock.

Audio effects

There are 14 audio effects, After Effects' audio set: levels, a compressor, bass and treble, a parametric EQ, high and low pass, delay, reverb, chorus, flanger, tremolo, vibrato, a stereo mixer and backwards. ymotion_find {"catalogue":"audio"} lists them, and ymotion_find {"catalogue":"audio","id":"compressor"} shows one with every knob, range and unit.

ymotion_edit {"ops":[{"verb":"addAudioEffect","args":{"layerId":"<layer id>","id":"highPass","params":{"freq":80}}}]}
ymotion_edit {"ops":[{"verb":"setAudioParam","args":{"layerId":"<layer id>","index":0,"id":"freq","value":100}}]}

The compressor starts working at its listed defaults, -18 dB and 3:1, including after Reset. Ratio 1 with zero makeup is a true bypass; nonzero makeup still plays its gain. Every other knobbed effect is off while all its controls sit at their declared defaults; Backwards always plays. For the others the default that keeps them off is a delay time of 0, a depth of 0 (chorus, flanger, tremolo, vibrato), a reverb wet of 0 or an EQ gain of 0. Give values in params or with setAudioParam, naming the effect by its chain index. Order matters: moveAudioEffect reorders, removeAudioEffectAt removes, and setEffectEnabled with "what":"audio" bypasses a row while keeping its settings and keys.

Starting points to adjust from:

A voice: a high pass around 80 Hz removes rumble; a compressor with a threshold near -24 dB and a ratio of 3 to 4 evens out loud and quiet words, and 3 to 6 dB of makeup brings the level back up afterwards.
A telephone: a high pass around 300 Hz with a low pass around 3000 Hz.
A room: reverb, with wet raised from its default of 0 (around 0.3 to start). The reverb is four early reflections spread by size: around 0.1 to 0.2 sounds like a room, while the default of 0.5 spreads them over 0.1 to 0.4 s and is heard as separate echoes.

Delay's one echo and reverb's four reflections sum without intermediate clipping in the server mix and browser preview. A later levels stage can reduce that floating headroom before delivery. Nothing limits the final sum: integer or compressed delivery can still distort a signal above full scale, so measure its peak.

Knobs keyframe at audio.effects.<index>.params.<knob>:

ymotion_edit {"ops":[{"verb":"keyRamp","args":{"layerId":"<layer id>","path":"audio.effects.0.params.freq","from":100,"to":1500,"opts":{"start":0,"end":3}}}]}

Some knobs are read once when the mix starts, so keys on them play the set value: every knob of delay, reverb, chorus, flanger, tremolo and vibrato, and the poles of the high and low pass. The catalogue marks them "keyframes": false. ymotion_read {"what":"properties","layerId":"<layer id>"} lists every path on a layer that takes keys, with its current value.

Sound effects from the library

A shared library of whooshes, impacts, risers, clicks, notifications and more is ready to use, with nothing to upload:

ymotion_find {"catalogue":"sfx"}
ymotion_find {"catalogue":"sfx","text":"short soft whoosh around 0.6 s"}
ymotion_find {"catalogue":"sfx","id":"<sfx id>"}

The first lists the groups. A search wants every word and falls back to the closest. A length such as "around 0.6 s" or "under 1 s" narrows the results and puts the nearest first, and "short" on its own means under half a second.

Each sound's moment was measured when it was added: a whoosh's peak, an impact's hit, a riser's end, the first keystroke of typing. placeSfx lands that moment exactly on at, so for a shutter on a cut, at is the time of the cut, not where the file starts:

ymotion_edit {"ops":[{"verb":"placeSfx","args":{"sfx":"<sfx id>","at":2.4}}]}

It goes on a free audio track holding only sound effects, or a new one called "SFX", so it never moves or cuts another layer. A marker on the layer annotates the intended moment, and the reply gives the layer id and the level it got. Moving or renaming that marker does not change the waveform's acoustic moment: that report uses the catalogue's source sync or measured PCM and says when neither is observable.

With no level asked for, a quiet sound is raised toward -14 LUFS, about speech at its loud moments, so it's heard over a voice: by 12 dB at most, with peaks kept under -1 dBTP, and a loud sound isn't turned down. A place object beside sfx and at chooses otherwise:

"place":{"level":-6} sets the gain in decibels, against the file as recorded; "place":{"loudness":-18} brings its loudest moment to that loudness in LUFS instead, up or down (still within +12 dB and the -1 dBTP ceiling), which helps even out sounds from different packs.
"place":{"align":"start"} puts the file's first sample on at.
"place":{"duration":0.5} keeps only the first half second of the layer, which with the default alignment begins before the moment by however far into the file the moment is, with a 5 ms fade where it cuts into sound.
"place":{"trackId":"<track id>"} chooses an audio track that's free at that time.

If the moment would need the file to start before 0, its head is trimmed so the moment still lands; when that would lose more than half the sound, as with a riser ending at 0, the call is refused, so place it later or with "align":"start". A time past the end of the composition is refused too, so place footage or music first. A film can use 64 different library sounds; repeats are free. Inside a nested composition, pass its id: ymotion_edit {"compId":"<comp id>","ops":[{"verb":"placeSfx","args":{"sfx":"<sfx id>","at":1}}]}.

Example: a shutter on a transition

ymotion_read {"what":"tracks"} lists each layer's start, which is where the cuts are. Each transition takes a camera shutter with its click on the cut; under continuous speech, keep it low and short:

ymotion_find {"catalogue":"sfx","group":"camera"}
ymotion_edit {"ops":[{"verb":"placeSfx","args":{"sfx":"<sfx id>","at":3}}]}
ymotion_render {"what":"waveform"}

Levels that work over speech

Ordinary talking-head footage, measured, sits around -16 LUFS overall and -14.5 at its loud moments, with peaks near -1 dBFS. Against that:

Sound effects need to reach about the voice's level. A whoosh turned down to -12 dB under speech disappears, so leave library sounds at the level they're given and lower one only when the waveform shows it standing well over the voice.
Music under speech usually sits 12 to 18 dB below the voice. A mastered song is generally louder than speech, often -9 to -14 LUFS, so about -18 to -24 dB while someone talks is a reasonable first guess, the lower end for a loud master; music with no voice over it can sit much nearer 0 dB.

Checking the mix

ymotion_render {"what":"waveform"} measures integrated loudness (LUFS), loudness range (LU), maximum momentary and short-term loudness, true peak (dBTP) when the server can measure it, and sample peak (dBFS). It measures each audible sound and shows 100 ms loudness lanes for the mix and stems on one −60 to 0 LUFS scale. The reply labels the delivery as none, copied, or mixed and says whether it measured an original source or a labelled conform. For a mixed export these numbers come from the export's mix before its AAC encode; the encode can raise true peak. A copied stream is measured on its own clock and channel layout. No meter has listened to the sound as a person would.

The reply can report a peak above −1 dBTP or a sample peak at 0 dBFS, a silence of at least 1 second below −60 LUFS, a word buried by other sound, or an effect that lands late. Speech less than 6 LU above the rest deserves attention; below 0 LU, the rest measures louder. It identifies words by index and time, never by inventing their text; the offered transcript call uses file seconds, not timeline seconds. A nested composition's audible sound is measured too, although a word mixed with other sounds inside that bus cannot be separated. These are measured structures, not a promise of intelligibility or a pleasant mix. Measured findings appear only when the owner enables them; the numbers and delivery facts remain available. What the measurements mean gives every threshold and the evidence labels.

If the meter is still working, the reply includes the plan and a time to ask again with the same call. An interim image of stored peaks is labelled stored peaks, not loudness. If this server's FFmpeg fails its meter self-test, the reply labels the fallback and explains that no loudness was measured. To inspect a stretch, use ymotion_render {"what":"waveform","from":2,"to":6}; from and to have been declared since 2026-10-03. ymotion_render {"what":"audio"} makes a file the person can play; ask them to check voice, effects, fades and the encode by ear. See Looking at your work.

Good to know

Speed changes: Server audio files, exports and meters use the picture's source clock for explicit ramps, native numeric rate keys and reverse. Pitch is kept by default; "patch":{"rate":0.5,"pitch":false} lets it fall with speed, like tape. A zero-rate hold is silent. The Node audio processor prepares moving clocks under the job's cancellation and resource limits; it never substitutes the first rate. A missing processor or failed preparation is reported.
The live browser preview does not run that Node processor: reversed and ramped sound is silent, and a speed change requesting kept pitch plays with changed pitch. Its warnings describe the preview. Check the rendered audio or export for that sound; do not mute it merely to work around the preview.
A nested composition's sound is summed, and the layer placing it applies its own level, fades and pan to the sum. It isn't drawn as a waveform lane. See Compositions and nesting.
Overlap transitions silence each extended layer past the cut, so the sound still cuts where it did. See Cuts and transitions.
Waveforms are measured once: a file placed through ymotion_media is measured on arrival and a library sound effect carries its own, while an asset added by hand with addAsset isn't drawn.