FFmpeg — Your powerful video/audio helper in your application

This article was published as a part of the Data Science Blogathon

The name is inspired by MPEG, which means Moving Picture Expert Group. The FF means Fast Forward. The MPEG group is the alliance of working groups behind setting many of the essential video standards today, including the MP4 format, which we are all familiar with.

Outcomes

1. What is FFmpeg?
2. Why do people use FFmpeg?
3. How to set up FFmpeg in Windows, Ubuntu, and macOS.
4. Libraries are available in FFmpeg for media manipulations.
5. Tools that FFmpeg comes with for command-line applications.
6. Usage and examples for FFprobe and FFplay.

Why do people use FFmpeg?

It is often referred to as the Swiss Army knife of media transcoding or streaming. We can use FFmpeg to perform a lot of functions. The code is written in C language and optimized for best performance. It commands are simple to run. Once you get familiar with the concepts, it is very flexible to all the filters and options you can use to fit your needs.

FFmpeg supports a wide range of codes, formats, devices, and protocols, making it an ideal choice as a transcoding engine. Unlike many discontinued projects, it is still in active development for more than 20 years. There’s a large community of developers, users, and contributors who constantly develop new features and fixes.
FFmpeg has been used in the core processing for video platforms like YouTube and iTunes. Most of us used a media player like VLC to play video files. VLC uses FFmpeg libraries as its core. Some video editors and mobile applications also use FFmpeg under the hood.

Setting up FFmpeg in Windows, Ubuntu, and macOS

There are a few things you need to know before setting up FFmpeg. There’s no official build for any operating systems. But it is an open-source project, which you can get hold of the FFmpeg source code from the internet. So, you can choose to build it yourself from the source code. You can download a snapshot of the current source code here or do a checkout from their git repository and follow the instructions to compile it to your machine. This can be a little complex and time-consuming and may require you to understand and choose some configurations options. It may also need additional tools and compilers to be installed at first. So unless you have a good reason to build it yourself, you may not want to go this route.

Instead, you can choose to download and install a pre-build FFmpeg package for your operating system. There are two ways to set up these pre-build binaries. If you have a package manager like APT in Ubuntu and Homebrew in macOS, it is easier to use it since it takes care of downloading FFmpeg and its dependencies.

Let’s keep the thing as simple as possible and choose the easiest way to get started. The procedures below show how you can set up using pre-build packages.

macOS

On macOS, the easiest way to get FFmpeg installed is by using Homebrew. Make sure you installed Homebrew in your macOS, or you can follow this tutorial on how to install Homebrew.
We are using macOS 11.4 in this case.

• Open Terminal and type brew — version . You should see the brew version printed out in Terminal if Homebrew is installed successfully.

• For macOS in x86 Architecture, use the commandbrew install ffmpeg while forM1 please use the command aarch arm64 brew install ffmpeg .

• After the installation is done, you can type ffmpeg -version in Terminal. If the FFmpeg version is printed, means FFmpeg is installed successfully.

Windows

Unlike APT on Ubuntu and Homebrew on macOS, there’s no well-known package manager on Windows, which you can use to quickly install FFmpeg. For Windows, you need to down and set it up manually.

There are 4 Windows FFmpeg build variants available can be found here:

1. git full – built from a master branch with a large set of libraries.
2. git essentials – built from a master branch with commonly-used libraries.
3. release full – built from the latest release branch with a large set of libraries.
4. release essentials – built from the latest release branch with commonly-used libraries.

We will use FFmpeg release essentials in our case.

• Download FFmpeg release essentials here.
• Extract the files to a place from where you will use them in the future. In my case, I put them in my E Drive.

• Inside the bin folder is containing those executables we want.

• Add bin folder to Windows Environment Path Variable so that we can run this tool without having to specify the full path every time. Search for Environment Variables in Search Bar and click on Environment Variables…

• Click on Path in User Variable, click New, and paste the full path of the bin folder.

• Open Command Prompt and type ffmpeg -version . If you see the FFmpeg version is printed, means FFmpeg is installed successfully. We also can check ffprob -version and ffplay -version .

FFmpeg Libraries

FFmpeg has several valuable libraries that you can use directly from your application code. Each library contains different functions related to a particular area.

Some of the famous libraries are:

1. libavcodec — Contains all the encoders and decoders that FFmpeg supports.
2. libavformat — Has all the muxers and demuxers for dealing with various containers format.
3. libavfilter — Consists of many filters that you can use to modify the audio or video according to the requirements.
4. libavdevice— Support several different input and output devices.
5. libavutil — Aid portable multimedia programming.
6. libswscale — Performs highly optimized image scaling and colorspace and pixel format conversion operations.
7. libswresample — Performs highly optimized audio resampling, remixing, and sample format conversion operations.

FFmpeg Tools

Besides the libraries which libraries can use from other applications, the FFmpeg package contains a few tools on its own.

Some of the tools are:

1. FFmpeg — Main transcoding engine. It is often invoked from a command line or another process.
2. ffplay — A minimal tool for playing audio or video.
3. ffprobe — Quickly inspecting and media to extract valuable information, like how many streams there are in the media, the frame rate of a video, etc.

FFprobe Usage and Examples

Usage

ffprobe [OPTIONS] [INPUT_FILE]

• man ffprobe — Refer to the manual.
• ffprobe -h — Shows included help.
• -v error — Hide logs unless there’s an error.
• -show_format — Show formatted video metadata using Tag.
• show_streams — Show all the streams in the file (both video and audio).
• -print_format —Print metadata in specific format (CSV/XML/JSON).
• -select_streams
• -show_entries stream= — Show only the specific entity and its nested dependencies.
• -show_entries stream= default-noprint_wrappers=1 — Show only the specific entity.

You can find out more FFprob usage here.

Basic Examples

• Show all video metadata.

$ ffprobe -v error -show_format -show_streams input.mp4

Output:
[STREAM]
index=0
codec_name=h264
codec_long_name=H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
profile=High
codec_type=video
codec_time_base=1/50
codec_tag_string=avc1
codec_tag=0x31637661
width=320
height=240
has_b_frames=2
sample_aspect_ratio=1:1
display_aspect_ratio=4:3
pix_fmt=yuv420p
level=13
color_range=N/A
color_space=unknown
color_transfer=unknown
color_primaries=unknown
chroma_location=left
timecode=N/A
refs=4
is_avc=1
nal_length_size=4
id=N/A
r_frame_rate=25/1
avg_frame_rate=25/1
time_base=1/12800
start_pts=0
start_time=0.000000
duration_ts=384000
duration=30.000000
bit_rate=34761
max_bit_rate=N/A
bits_per_raw_sample=8
nb_frames=750
nb_read_frames=N/A
nb_read_packets=N/A
DISPOSITION:default=1
DISPOSITION:dub=0
DISPOSITION:original=0
DISPOSITION:comment=0
DISPOSITION:lyrics=0
DISPOSITION:karaoke=0
DISPOSITION:forced=0
DISPOSITION:hearing_impaired=0
DISPOSITION:visual_impaired=0
DISPOSITION:clean_effects=0
DISPOSITION:attached_pic=0
TAG:language=und
TAG:handler_name=VideoHandler
[/STREAM]
[STREAM]
index=1
codec_name=aac
codec_long_name=AAC (Advanced Audio Coding)
profile=LC
codec_type=audio
codec_time_base=1/44100
codec_tag_string=mp4a
codec_tag=0x6134706d
sample_fmt=fltp
sample_rate=44100
channels=1
channel_layout=mono
bits_per_sample=0
id=N/A
r_frame_rate=0/0
avg_frame_rate=0/0
time_base=1/44100
start_pts=-1024
start_time=-0.023220
duration_ts=1324024
duration=30.023220
bit_rate=56517
max_bit_rate=N/A
bits_per_raw_sample=N/A
nb_frames=1293
nb_read_frames=N/A
nb_read_packets=N/A
DISPOSITION:default=1
DISPOSITION:dub=0
DISPOSITION:original=0
DISPOSITION:comment=0
DISPOSITION:lyrics=0
DISPOSITION:karaoke=0
DISPOSITION:forced=0
DISPOSITION:hearing_impaired=0
DISPOSITION:visual_impaired=0
DISPOSITION:clean_effects=0
DISPOSITION:attached_pic=0
TAG:language=und
TAG:handler_name=SoundHandler
[/STREAM]
[FORMAT]
filename=input.mp4
nb_streams=2
nb_programs=0
format_name=mov,mp4,m4a,3gp,3g2,mj2
format_long_name=QuickTime / MOV
start_time=-0.023220
duration=30.024000
size=368644
bit_rate=98226
probe_score=100
TAG:major_brand=isom
TAG:minor_version=512
TAG:compatible_brands=isomiso2avc1mp41
TAG:title=FFprobe Tips
TAG:encoder=Lavf56.15.101
[/FORMAT]

• From the above example, if you would only want duration=30.024000 .

$ ffprobe -v error -show_entries format=duration -of default=noprint_wrappers=1 input.mp4

Output: size=368644

• If you would only want the value with no key.

$ ffprobe -v error -show_entries format=size -of default=noprint_wrappers=1:nokey=1 input.mp4

Output: 368644

• Getting width and height (resolution) of video.

$ ffprobe -v error -select_streams v:0 -show_entries stream=height,width -of csv=s=x:p=0 input.mp4

Output: 1280x720

FFplay Usage and Examples

Usage

ffplay [OPTIONS] [INPUT_FILE]

• man ffplay — Refer to the manual.
• ffplay -h — Shows included help.
• ffplay  — Play the video on full screen.
• -x -y  — Play the video with specific height and width popup windows size in center. It will automatically fill the extra space between the video area and the window area if the aspect ratio of the popup window and the video are different.
• -noborder — Hide title bar while playing video in popup windows.
• -[top/left/bottom/right]  — Make popup window in specific coordinate. By default, it will pop up in the center of the screen.

You can find out more FFplay usage here.

Basic Examples

• Play a video file named video.mp4 in fullscreen.

$ ffplay video.mp4

• Play a video file in a popup window with sizes 600 widths, 600 height, and no title bar.

$ ffplay video.mp4 -x 600 -y 600 -noborder



• Play a video file in a popup window in the top left corner.

$ ffplay video.mp4 -x 600 -y 600 -noborder -top 0 -left 0



 

Conclusion

We had covered all the basics in FFmpeg. Feel free to read their official documentation here if you want to have a deeper understanding!

References

• https://ffmpeg.org/documentation.html
• https://trac.ffmpeg.org/wiki/FFprobeTips
• https://ffmpeg.org/ffplay-all.html
• https://ffmpeg.org/ffprobe.html
• FFmpeg Complete Guide by Syed Andaleeb Roomy

About Author:

This article is written by Han Sheng, an AI Full Stack Application Developer from Cameron Highlands, Malaysia. He has a passion for Deep Learning, Computer Vision, and also Edge Devices. He did several AI-based Web/Mobile Applications to help clients solving real-world problems. Feel free to read about him via his portfolio.

+