FFmpeg aide
Some tips with ffmpeg
Original article : https://lukaprincic.si/development-log/ffmpeg-audio-visualization-tricks
ffmpeg -i INPUT_AUDIO.wav -filter_complex "[0:a]avectorscope=s=480x480:zoom=1.5:rc=0:gc=200:bc=0:rf=0:gf=40:bf=0,format=yuv420p[v]; [v]pad=854:480:187:0[out]" -map "[out]" -map 0:a -b:v 700k -b:a 360k OUTPUT_VIDEO.mp4
The code above creates a mp4 video file with a vectorscope nicely centered inside a 854×480 (480p) video. If you need a 1:1 video, just exclude the pad part:
ffmpeg -i INPUT_AUDIO.wav -filter_complex "[0:a]avectorscope=s=480x480:zoom=1.5:rc=0:gc=200:bc=0:rf=0:gf=40:bf=0,format=yuv420p[v]" -map "[v]" -map 0:a -b:v 700k -b:a 360k OUTPUT_VIDEO.mp4
Documentation on ‘avectorscope’ filter is here: https://ffmpeg.org/ffmpeg-filters.html#avectorscope. One can play with zoom and other options to produce desired form.
ffmpeg -i INPUT.wav -filter_complex "[0:a]showwaves=mode=line:s=hd480:colors=White[v]" -map "[v]" -map 0:a -pix_fmt yuv420p -b:a 360k -r:a 44100 OUTPUT.mp4
more options: https://www.ffmpeg.org/ffmpeg-filters.html#showwaves
ffmpeg -i INPUT.wav -filter_complex "[0:a]showspectrum=s=854x480:mode=combined:slide=scroll:saturation=0.2:scale=log,format=yuv420p[v]" -map "[v]" -map 0:a -b:v 700k -b:a 360k OUTPUT.mp4
Above code will create almost completely desaturated spectrum of the audio sliding from right to left. Again, there are various options to tweak, see here: https://ffmpeg.org/ffmpeg-filters.html#showspectrum-1
ffmpeg -i INPUT.wav -filter_complex "[0:a]ahistogram=s=hd480:slide=scroll:scale=log,format=yuv420p[v]" -map "[v]" -map 0:a -b:a 360k OUTPUT.mp4
more options: https://www.ffmpeg.org/ffmpeg-filters.html#showwaves
Sometimes you want to just create a static image.
ffmpeg -i INPUT.wav -lavfi
showspectrumpic=s=hd480:legend=0,format=yuv420p
SPECTROGRAM.png
ffmpeg -loop 1 -i SPECTROGRAM.png -i INPUT.wav
-s hd480 -t 00:01:00 -pix_fmt yuv420p
-b:a 360k -r:a 44100 OUTPUT.mp4
ffmpeg -i INPUT.wav -lavfi
showspectrumpic=s=hd480:legend=0,format=yuv420p
SPECTROGRAM.png
ffmpeg -loop 1 -i SPECTROGRAM.png -i INPUT.wav
-s hd480 -t 00:01:00 -pix_fmt yuv420p
-b:a 360k -r:a 44100 OUTPUT.mp4
Above one is in two steps. More info here: https://www.ffmpeg.org/ffmpeg-filters.html#showspectrumpic
ffmpeg \
-i video1.mp4 -i video2.mp4
-filter_complex "[0:v:0] [0:a:0] [0:v:1] [0:a:1] concat=n=2:v=1:a=1 [v][a];
[v]drawtext=text='SOME TEXT':x=(w-text_w):y=(h-text_h):fontfile=OpenSans.ttf:fontsize=30:fontcolor=white[v]" \
-map "[v]" -map "[a]" -deinterlace \
-vcodec libx264 -pix_fmt yuv420p -preset $QUAL -r $FPS -g $(($FPS * 2)) -b:v $VBR \
-acodec libmp3lame -ar 44100 -threads 6 -qscale 3 -b:a 712000 -bufsize 512k \
-f flv "$YOUTUBE_URL/$KEY"
ffmpeg -i path/to/file.ext
for f in *.m4a; do ffmpeg -i "$f" -acodec libmp3lame -vn -b:a 320k "${f%.m4a}.mp3"; done
-g : GOP, for searchability
ffmpeg -i input.foo -vcodec bar -acodec baz -b:v 21000k -b:a 320k -g 150 -threads 4 output.bar
ffmpeg -r 18 -pattern_type glob -i '*.png' -b:v 21000k -s hd1080 -vcodec vp9 -an -pix_fmt yuv420p -deinterlace output.ext
-ss : start time / -t : seconds to cut / -autoexit : closes ffplay as soon as the audio finishes
ffmpeg -ss 00:34:24.85 -t 10 -i path/to/file.mp4 -f mp3 pipe:play | ffplay -i pipe:play -autoexit
-codecs # list codecs
-c:v # video codec (-vcodec) - 'copy' to copy stream
-c:a # audio codec (-acodec)
-fs SIZE # limit file size (bytes)
-b:v 1M # video bitrate (1M = 1Mbit/s)
-b:a 1M # audio bitrate
-aspect RATIO # aspect ratio (4:3, 16:9, or 1.25)
-r RATE # frame rate per sec
-s WIDTHxHEIGHT # frame size
-vn # no video
-aq QUALITY # audio quality (codec-specific)
-ar 44100 # audio sample rate (hz)
-ac 1 # audio channels (1=mono, 2=stereo)
-an # no audio
-vol N # volume (256=normal)
ffmpeg -y -i input.mp3 -loop 1 -i background.png \
-filter_complex "[0:a]showwaves=s=1280x720:mode=line,colorkey=0x000000:0.01:0.1,format=yuva420p[v];[1:v][v]overlay[outv]" \
-map "[outv]" -pix_fmt yuv420p -map 0:a -c:v libx264 -c:a copy -shortest output.mp
https://www.youtube.com/watch?v=zKYzJ_bEJVo
ffmpeg -y -i audio.mp3 -loop 1 -i image.jpg \
-filter_complex "[0:a]showwaves=s=1280x175:colors=Yellow:mode=line,format=yuv420p[v];[1:v][v]overlay=0:200[outv]" \
-map "[outv]" -pix_fmt yuv420p -map 0:a -c:v libx264 -c:a copy -shortest output9.mp4
https://www.youtube.com/watch?v=1htjI7YSNZo
ffmpeg -y -i audio.mp3 -loop 1 -i image.jpg \
-filter_complex "[0:a]showwaves=s=1280x720:mode=line,colorkey=0x000000:0.01:0.1,format=yuva420p[v];[1:v][v]overlay[outv]" \
-map "[outv]" -pix_fmt yuv420p -map 0:a -c:v libx264 -c:a copy -shortest output1.mp4
https://www.youtube.com/watch?v=H5dCKuN9Ius
ffmpeg -y -i audio.mp3 -loop 1 -i image.jpg \
-filter_complex "[0:a]showwaves=s=1280x175:colors=White:mode=p2p,format=yuv420p[v];[1:v][v]overlay=0:200[outv]" \
-map "[outv]" -pix_fmt yuv420p \
-map 0:a -c:v libx264 -c:a copy -shortest output12.mp4
https://trac.ffmpeg.org/wiki/FancyFilteringExamples
ffplay -f lavfi -i cellauto=rule=110
Other interesting cellauto rule values: 9, 18, 22, 26, 30, 41, 45, 50, 54, 60, 62, 73, 75, 77, 82, 86, 89, 90, 97, 99, 101, 102, 105, 107, 109, 110 (default), 124, 126, 129, 131, 133, 135, 137, 145, 146, 149, 150, 151, 153, 154, 161, 167, 169, 181, 182, 183, 193, 195, 210, 218, 225.
ffplay -f lavfi -i life=s=300x200:mold=10:r=60:ratio=0.1:death_color=#C83232:life_color=#00ff00,scale=1200:800:flags=16
ffplay -f lavfi -i life=s=640x480:mold=10:r=100:ratio=0.1:death_color=blue:life_color=#00ff00,boxblur=2:2
ffplay -f lavfi -i mandelbrot
ffplay -i INPUT -vf "crop=iw/2:ih:0:0,split[tmp];[tmp]hflip; hstack"
ffplay -f lavfi -i testsrc -vf "split=4[a][b][c][d];[b]lutrgb=g=0:b=0[x];[c]lutrgb=r=0:b=0[y];[d]lutrgb=r=0:g=0[z];[a][x][y][z]hstack=4"
ffplay -f lavfi -i testsrc -vf "split=4[a][b][c][d];[b]lutyuv=u=128:v=128[x];[c]lutyuv=y=0:v=128[y];[d]lutyuv=y=0:u=128[z];[a][x][y][z]hstack=4"
ffplay -f lavfi -i mandelbrot -vf "format=gbrp,split=4[a][b][c][d],[d]histogram=display_mode=0:level_height=244[dd],[a]waveform=m=1:d=0:r=0:c=7[aa],[b]waveform=m=0:d=0:r=0:c=7[bb],[c][aa]vstack[V],[bb][dd]vstack[V2],[V][V2]hstack"
ffplay -f lavfi -i mandelbrot -vf "format=yuv444p,split=4[a][b][c][d],[a]waveform[aa],[b][aa]vstack[V],[c]waveform=m=0[cc],[d]vectorscope=color4[dd],[cc][dd]vstack[V2],[V][V2]hstack"
Split the waveform filter to show broadcast range of the waveform (y values between 16 and 235) in green and out of broadcast range in red.
ffplay -i http://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_480p_surround-fix.avi -vf "split[a][b];[a]format=gray,waveform,split[c][d];[b]pad=iw:ih+256[padded];[c]geq=g=1:b=1[red];[d]geq=r=1:b=1,crop=in_w:220:0:16[mid];[red][mid]overlay=0:16[wave];[padded][wave]overlay=0:H-h"
Split the waveform filter to show broadcast range of the waveform (y values between 16 and 235) in green and out of broadcast range in red and also use envelope.
ffplay ~/matrixbench_mpeg2.mpg -vf "split[a][b];[a]waveform=e=3,split=3[c][d][e];[e]crop=in_w:20:0:235,lutyuv=v=180[low];[c]crop=in_w:16:0:0,lutyuv=y=val:v=180[high];[d]crop=in_w:220:0:16,lutyuv=v=110[mid] ; [b][high][mid][low]vstack=4"
ffmpeg -i video.ext -i audio.ext -c:v copy -c:a copy output.ext
Making some random "musical" keys:
% cat expr
# floor(t): 0 0 0 0 0 ... 1 1 1 1 1 ... 2 2 2 2 2
# => set a random key when floor(t) changes
if(eq(floor(t),ld(2)),
st(0,random(4)*3000+1000));
# the next value to compare floor(t) with
st(2,floor(t)+1);
# mod(t,1) makes t always in the range [0;1) for each key
st(1,mod(t,1));
# 0.6*... + 0.4*... for "echo" effect
# exp() to mitigate the sound according to the time
(0.6*sin(1*ld(0)*ld(1))+
0.4*sin(2*ld(0)*ld(1)))*exp(-4*ld(1))
And to test it:
ffplay -f lavfi -i "aevalsrc=$(grep -v '^#' expr|tr -d '\n'|sed 's/\([,;]\)/\\\1/g')"
Given the audio file april.flac:
ffplay -f lavfi 'amovie=april.flac,asplit=3[out1][a][b]; [a]showwaves=s=640x240[waves]; [b]showspectrum=s=640x240[spectrum]; [waves][spectrum] vstack[out0]'
Given the multichannel audio file tearsofsteel-surround.flac:
ffplay -f lavfi 'amovie=tearsofsteel-surround.flac,asplit=2[out1][a]; [a]showspectrum=color=channel:scale=cbrt:orientation=vertical:overlap=1:s=2048x1024[out0]'
Now with different colors and scaling:
ffplay -f lavfi 'amovie=tearsofsteel-surround.flac,asplit=2[out1][a]; [a]showspectrum=color=fire:scale=log:orientation=vertical:overlap=1:s=1024x1024[out0]'
Given the audio file input.flac:
ffplay -f lavfi 'amovie=input.flac,asplit=2[out1][a],[a]avectorscope=m=polar:s=800x400[out0]'
Given the audio file input.flac:
ffplay -f lavfi 'amovie=input.flac,asplit=2[out1][a],[a]showcqt[out0]'
Given the audio file input.flac:
ffmpeg -i input.flac -lavfi showspectrumpic=s=hd720 out.jpg