Use of the MPEG4IP Server Side Tools

The MPEG4IP Server Side Tools allow for the conversion of raw audio and video into MPEG-4 compressed media stored in an MP4 file which can be streamed by the Darwin Streaming Server. It also includes tools for displaying information about the streams.

If you are using a Linux system, the easiest method is to use mp4live. Hook your inputs to your capture card, set your audio and video settings (remember to use AAC to create ISMA-compatible content), and start.

On other OS machines, or on linux without a capture card, you will need to use encoders that are not included with this package, like xvid, faac, lame, or ffmpeg to encode content. You can then use mpeg4ip's tools to put that content into formats that can be used for streaming or interoperability.

The basic steps to create content are:

  1. Get raw content in form that encoder can understand. This is probably YUV 4:2:0 format for video, or 16 bit PCM for audio.
    It may also be another format for transcoders, like VLC, ffmpeg or virtualdub.
  2. Encode the content into elementary streams or another format that mpeg4ip understands.
  3. Assemble the content into mp4 file container
  4. Hint the files for streaming, if needed

Tools available

avi2raw Extracts audio/video tracks from an AVI file
lboxcrop Vertically crops raw video to a new aspect ratio
rgb2yuv Convert 24 bit RGB data file to a YUV 4:2:0 file
mp4venc Encodes raw video into MPEG-4 encoded video using ISO codec - not available on all systems
mp4creator Creates and hints audio/video tracks to an mp4 file. Can also be used to extract tracks and reconstruct AAC headers, and can encrypt to the ISMACryp specification. It will also assemble 3GPP files, but not hint them.
xvidenc Encodes raw video into MPEG-4 encoded video using the Xvid codec. Not available with xvid 1.0 (yet)
mp4dump Utility to dump MP4 file meta-information in text form
mp4trackdump Utility to dump MP4 file track information in text form
mp4info Utility to display MP4 file summary
avidump Utility to display AVI file summary
yuvdump Utility to display a raw video file on the screen

Note: the mp4encode script is no longer available. Since we recommend getting the encoders from those sites, supporting it was made unwieldy.

Getting Raw Content

There are many ways of getting raw content. Most likely, the content will come in a couple of different forms - raw YUV, RGB or PCM files, or .avi, .mp4 or .mov containers containing raw data.

If you have an .avi container, you can use avidump to display information about the contents. The "Video Compressor" and "Audio format" fields display information about what format the audio and video are in. I420 is YUV 4:2:0, DIB is RGB. Other codes might indicate that the content is already encoded, so you can skip the next step.

avidump will also give you information about video frame rate, audio sampling rates and channels that you will need for encoding.

For .mp4 and .mov containers, you can use mp4info to display information about what audio and video tracks are in those container files. .mov codes for raw video are "raw " for RGB and "Yuv2" for YUV 4:2:2. mp4live will store raw YUV with a special code only used for mpeg4ip.

If you have RGB data, you will need to use the rgb2yuv utility to convert the RGB data. The RGB data needs to be in 24 bit packed format to work with this utility. RGB24 is a packed format with 1 byte for each R, G and B.

You can also use the lboxcrop to crop the YUV data.

For the most part, the encoders that need YUV data will need YUV12, aka YV12. (FYI: This is a planar format sampled at 4:2:0)

Our yuvdump utility is a good method of checking that your YUV data is correct and in the correct format. The height and width must be specified, but if the data is correct, you should see the video that you are trying to encode.

Encoding Content

When you have raw audio and raw video, you will need to use an encoder to put the audio and video in the correct encode format for your application.

mpeg4ip contains the ISO encoder, which compiles only on the windows platform. If you want to use this encoder, you will need to create an executable and use a file where the many configuration parameters are given. An template parameter file, 'mp4venc_template.par', is located in the server/util/mp4encode directory. We do not support this encoder, and don't know how to set the parameters, so don't ask.

How you encode your audio and video depends mainly on your available encoder options, the bandwidth that you wish to use, and the interoperability level you wish to maintain.

We recommend released versions of ffmpeg or xvid for encoding video, and faac or lame for encoding audio, although any standard encoder should be okay.

For the most part, these stand-alone encoders will create elementary streams, but may create mpeg files, or avi files, or even .mp4 files. mp4player should be able to play most of the elementary streams.

Assembling the content

The mpeg4ip project's crown jewel is our mpeg-4 file container library, mp4v2, and our ability to store various content into files.

The mpeg-4 (mp4) file container format is defined in ISO-IEC 14496-12 and 14496-14, and is based on the Quicktime .mov format. One of the main features of this container format is that it is really content agnostic, and can be used to stream pretty much anything with the addition of standard hint tracks. To paraphrase the movie "Apollo 13", you could stream a washing machine if you could figure out how to hint it.

The mp4 file container can contain any number of streams (also called tracks). A fully hinted ISMA mp4 container can contain 6 different tracks.

The mpeg4ip project was built mainly on creating files interoperable with the ISMA. These files are mpeg-4 simple and advanced simple profile encoded with AAC low complexity audio in a mp4 container. (ISO 14496-2 video, 14496-3 audio, 14496-14 file format).

We've since added other encoded formats, such as mpeg1 or mpeg2 support for streaming servers using the mpeg-4 file container, and have received contributed code for 3GP support (h.263 video and AMR audio). We're also working to try to create .mov files compatible with Quicktime.

We've also put an emphasis on flexibility, so non-standard files (such as mpeg-4 video with mp3 audio, or mpeg-2 video with aac audio) can be supported.

To assemble content using this library requires the use of the mp4creator utility. This utility has several functions. For the most up-to-date information, use man mp4creator . These are:

For mp4 containers (and .mov files, as well), 3 utilities are provided that can be used at any time in the creation process. These are:

Each utility has a man page.

Adding Streams to file

Adding audio or video elementary streams to mp4 containers is very simple. Simply use the --create option:

mp4creator [-create] <es file> <mp4 file>

Supported elementary streams are: mpeg-4, H.263, H.264 (video), AAC, MP3, AMR (audio).

Video elementary streams will need to specify the frame rate. Mpeg-4 video can use variable frame rate information stored in the headers using the -variable-frame-rate option; however, most encoders do not put information that would be sufficient to correctly use this function.

mp4creator will also use .mpeg files to create mp4 file containers; the first audio and first video tracks will be added at the same time.

mp4creator used to support creation from avi files; however, avi files do not support B frames very well, and various creation tools such as divx and xvid have created different ways to store b-frames in avi files.

Therefore, avi support is deprecated. We recommend using avi2raw for audio and video in the avi file, then using the elementary stream file to create the content.

Hinting tracks file

To hint tracks, use the -hint=<track> option to mp4creator. The -hint option can also be used when adding the file to the track; however, sometimes it seems to fail - we recommend using 2 steps. Use the mp4info utility to display the track number

mp4creator can add hint tracks for mpeg-1, mpeg-2, mpeg-4, H.263 and H.264 video, and mpeg(1,2,3), aac audio.

Extracting tracks from a file

To extract track, simply use the -extract=<track> option.

Deleting tracks from a file

To delete tracks, simply use the -extract=<track> option.

Encrypting tracks

TBA

Valid HTML 4.01!