The MPEG4IP project provides a standards-based system for encoding, streaming, and playing encoded audio, video and text. To achieve this we've integrated a number of existing open source packages, and created some original code to fill in the gaps.
Please note this project is intended for developers who are interested in standards based audio and video, and Internet streaming. It is not intended for end-users. Please read all the legal information in the file COPYING!
Also note that the primary development focus of this project is the Linux platform. If you're going to use the package on other platforms, especially non-UNIX platforms, you'll probably have some work to do.
Please use the SourceForge site to report problems, suggest enhancements, ask questions, etc. The URL is http://www.sourceforge.net/projects/mpeg4ip
There is also a project web site at http://www.mpeg4ip.net/ that has some general information on MPEG4IP.
We also have a guide to MPEG4IP donated by everwicked. See doc/MPEG4IP_Guide.pdf.
There are two ways to use MPEG4IP to create content:
The older method assumes that you've somehow managed to capture raw audio and/or video into a file. That's the starting point from which you can use the MPEG4IP encoding tools to create an MP4 file. The simplest method being to use the 'mp4encode' script. Detailed instructions for this script and the individual tools that it uses are in doc/encoding.htm
Note that this might be a bit out of date; we're no longer including some of the packages like xvid and faac - they are seperate downloads, so the command lines may change. See those packages for specifics on command line.
The newer method is an integrated live encoding tool called mp4live. This program is designed to make it easy to create MP4 files or transmit live audio/video streams over the network. It can even do both things simultaneously! The key requirement to use this tool is to have Linux, a video capture device and a Video for Linux (v4l) or Video for Linux2 (v4l2) driver for it. So far we've tested with the bttv driver for Brooktree based video capture cards, and the qce driver for Logitech QuickCam Express USB webcams. Both of these solutions can be acquired for about $50 US! Please see the file MP4LIVE_README.html for more information about mp4live.
Once an MP4 file is prepared, it can be placed in the content directory of a streaming server. We typically use Apple's Darwin Streaming Server, but any server that understands MP4 files (or hinted Quicktime files) can be used.
When the content is encoded and available on the server, you can run
the player. Start 'gmp4player' and then enter the RTSP URL to the server
and the content, (command line works too). E.g.
Although we're focused on streaming, the player will also playback from
a local file. E.g
That's not all! The player is not limited to local playback of MP4 files.
It can also read AVI, CMP, DIVX, AAC, MP3, MPEG 1/2 and WAV. This is useful for
debugging since the encoded data can be check independently of the MP4
file container, and known good content, such as your favorite MP3, can be
used to verify that the player is working correctly with your hardware. E.g.
For the latest list of playback codecs, see the feature list.
And that's still not all... You can set up your player to run a playlist by creating a simple text file with the extension .mp4plist, .m4u, or .mxu, with each item (file or stream) on it's own line.
Note: If you prefer a no UI version of the player, or your system does not include GTK or GLIB, just the bare video window, 'mp4player' is available to fit that need.
Please see the file COPYING.
Darwin Quicktime Streaming Server
Please note that the Apple Darwin Quicktime Streaming Server is NOT distributed with mpeg4ip. It can be downloaded from Apple, as either source or pre-built binaries. For those who choose the source option, please see the documentation that accompanies the server on how to build and configure it for your environment.
Note the default content directory is /usr/local/movies and that you can configure the darwin server by pointing your browser at port 1220. (ie: localhost:1220)
There are some sample mp4 files included with the Darwin Streaming server. Envivio, also has some sample mp4 files. We suggest first downloading one of these samples and try opening the file with gmp4player. If that works, then try copying the file to the streaming server's content directory (e.g. /usr/local/movies), and enter the appropriate RTSP URL in gmp4player. You may have to "hint" the files before they will stream.
If you should have problems with DSS, please feel free to contact the developers over at Apple - they're a friendly bunch. They have 2 mailing lists - one for users and the other for developers. Give them a try.
There is a known problem (November, 2004) with running DSS and mp4live on the same system. Hopefully it will be fixed soon, or please see the sourceforge forums for more information
Prerequisites for Building
This section will discuss prerequisites for building mpeg4ip (ie: packages that you will first want to install to build correctly). This does not include Windows. To build on Windows, see these instructions.
Specifics on how to build using these libraries will be discussed below.
All builds require the SDL development libraries. SDL version 1.2.7 or later is required; please be sure to install the development library, if you are using RPMs.
If you have a x86 based platform, you will want to install nasm. If you do not install it, you may not be able to decode or encode at the frame rates desired. See the nasm site for information.
If you are building either the player, or mp4live, you will want to install GTK-2. We no longer support GTK-1.2. Make sure to install the development packages, as well.
If you are building the player, you may want the following packages:
The following packages are optional for mp4live, but will enable more functionality:
The developer recommends ffmpeg for mpeg-4 encoding. Please see the instructions below for how to install with ffmpeg.
If you are going to encode mpeg-4 or H.264, you will also want to
include faac for aac encoding; most other players will not play
other types of audio with those video encodings.
Building and Installing
Please read the section on prerequisites and this section completely before building, or you may have to do it twice; mpeg4ip has a number of optional packages that should be installed before starting the build process.
If you are building on Windows, please see these instructions.
On a linux or unix base system, you should only have to issue the following:
Several systems have differing requirements. Please see the section on Operating Systems Supported before continuing.
If you wish to build mpeg4ip without mp4live, you can give the
Mp4live also works best with the V4L2 interface, which is not installed on most linux distributions with a 2.4 kernel. mp4live will work without the V4L2 interface, but you will have long term audio/video synchronization problems (where long term could be minutes or hours). We give a little information on how we've installed V4L2. 2.6 series kernels should have V4L2 built in.
Fedora Core 3 and below do not include the correct include files to activate V4L2. At this point, we believe that the V4L2 is supported after installing the videodev.h and videodev2.h files, but have not verified it. We have modified the configure so it will print out a message during configure that specifies the directory where you might locate the correct files.
To install with ffmpeg version 0.4.8 or earlier,
enter the argument
For version 0.4.9 or later, we recommend that you install the ffmpeg libraries and
include files. When installing ffmpeg, use the additional
If you install ffmpeg, it is not recommend to link it with any external libraries other than the AMR libraries below. If you do, you will have to add the extra libraries to the Makefiles that use ffmpeg (in player/plugin/video/ffmpeg, player/plugin/audio/ffmpeg, and server/mp4live).
Note: as of April, 2006, the
If you wish AMR decoding, you will want to build ffmpeg with AMR support - see their documentation for that. (Basically, use their --enable-amr_wb and --enable-amr_nb code, get the code and unzip it in the correct sub-directory - for the amr_float directory, you'll have to move the code from c-code to amr_float).
To build ffmpeg on Mac OS X, build with normal options. Edit the
config.mak, and change LDFLAGS to a NULL string - i.e.
If you get an error when linking, remove the -mdynamic-no-pic from OPTFLAGS in config.mak, and rebuild ffmpeg entirely
You may also have to run
We haven't been successful linking ffmpeg builds of 2007 on the Mac.
We've tested with ffmpeg version 0.4.7, 0.4.8 and 0.4.9-pre1. See the ffmpeg site for information on how to download. Use of CVS versions are at the discretion of the user. If you have problems with compiling with a CVS version of ffmpeg, please use a released version. If you would like to do mpeg2 encoding, you will need ffmpeg.
If you want to install faad2, please do it after you install mpeg4ip. If you have previously installed it, you must uninstall it (and most likely uninstall faac, as well). The faad2 plugin will overwrite the aac_plugin.
If you don't have root privileges, but still wish to install the distribution to a directory to which you do have write permission, then here is an example of how to do that:
You will also have to add ~/bin to your path, and ~/lib to your LD_LIBRARY_PATH
If this process works for you, you can skip down to the next section on Bootstrap Options. If this does not work, or you have downloaded from CVS, the below applies.
We've built the distribution using GNU autoconf, automake and libtool. We have attempted to follow the GNU conventions for open source packages. This is complicated by the fact that we build on many other packages. Where the package was already using the GNU tools, we left things alone. Where the package was using it's own Makefile, we left things alone if it was a complicated Makefile. If it was straightforward we replaced it with an equivalent automake file.
If you have to install any of these tools, find out where the others are installed
(which autoconf). If the path does not start with /usr/local, use the
For example, if autoconf is in /usr/bin, use the
Note: libtool is sometimes not installed on Linux systems (autoconf, and automake generally are). You can download libtool from a GNU mirror site.
If you don't have these tools and are trying to compile from CVS, don't complain - update to these tools, then let us know. There may be problems or warnings with later tools - we don't keep up with the latest and greatest tools most of the time. Please try to fix these yourself before continuing, and feed back the changes using the Sourceforge forums.
Be sure to read the OS specific section later before continuing here.
In general, the code should be portable, but as someone once said "There's no such thing as portable code, just code that has been ported." When you find problems please be sure to use the SourceForge site to tell us what you encountered, and hopefully how you fixed it.
The bootstrap script will pass any arguments to the configure scripts.
For the curious, the bootstrap script invokes the configure scripts of the included packages that have them, and then our own top level configure script is generated and run. At the end of this process all the Makefile's are ready, and setup in the correct hierarchy. If this doesn't work for you, you're free to hack as needed ;-)
If you are using cvs_bootstrap and are running into problems where your GTK configurations are not found, you will need to find the aclocal files associated with gtk and glib (in particular glib-2.0.m4, and gtk-2.0.m4). These are usually in a share/aclocal directory somewhere on your system.
We currently search /usr/local, /usr/contrib, /opt/gnome /usr /opt/local. If these do not work for you, find the above files (find . -name glib-2.0.m4 -print), and add the path (without the share/aclocal) to cvs_bootstrap.
Two options of potential interest are
If you are building on a system with an Intel x86 CPU clone, you may
need to specify the configuration option
To build with IPv6 support, use the
Currently, we have compiled and tested on the following platforms: linux, freeBSD, BSD/OS, Solaris and windows. For all varieties of *nux, X11 is required.
To date we've built on Red Hat Linux 6.1 to 9.0 with the native compilers, on 7.1 and 7.2 with gcc 3.0, and on 7.3 and later with gcc 3.1. We also support 9.0 with the native compiler (gcc 3.2) and gcc 3.3.
As of January, 2004, we're migrating to SuSE 9.0 due to problems we're seeing with RedHat 9 and the update process.
If you have built a previous version of mpeg4ip, do a make uninstall first, or go in and remove libsndfile from your shared libraries directory, unless you have another version installed.
libSRTP: After building and installing the libSRTP pkg according to its intructions, run cvs_bootstrap or configure so mpeg4ip will find and (statically) include the libSRTP pkg. If it is not found or if --disable-srtp is passed to configure, the libSRTP dependent calls with be conditionally compiled out.
When building with SuSE 9.0, you need to install a number of packages before you start.
If you're installing with SuSE 9.0 professional, you can install everything you need off the CD's. For a list of packages that I used, see the suse90pro.installed file in our distribution, or in CVS. These include packages that I needed to install the 2.6.1 kernel and use the make gconfig configuration.
If you're installing SuSE 9.0 personal edition, you should include all compiler packages, and the following packages. I recommend to install them in this order:
Be sure to install the GLIB/GTK family over the existing installs - for the most part, be sure to use configure --prefix=/opt/gnome.
If you want to run gmp4player with sound, you will need to disable the aRTs sound server. Start the Control Center, select Sound & Multimedia, then select Sound System. Uncheck the box that says "Start aRts soundserver on KDE startup", and hit Apply. You will need to log out, then log back in.
If you have any other way to get sound with gmp4player without the above, let us know.
One note with SuSE 9.0 - it looks like they tried to install V4L2, but it does not work. They installed an incorrect version of videodev2.h. We have only installed V4L2 as part of the 2.6 kernel (see the guide here).
When building on Slackware-8.0, you need to consider the following (from maersk):
Note that you need to be root to do this.
When building on Solaris, libtool and gnu make must be installed and used. If libtool is installed, you make get a warning message that common libraries made with gcc less than 3.0 might have problems, ignore it, but don't take binary libraries from any other machines.
There looks to be an error when building on FreeBSD 5.1 with gcc 3.2. There is a warning in the fstream.h include file. To build, you will need to remove the -Werror from BILLSCPPWARNING in configure.in, then rebuild.
Please see the FAQ included with the package (mp4liveOnDebian.FAQ). This was contributed by hell at netexpert dot hu.
NOTE: We don't usually support the Windows code in CVS. So, unless you're explicitly told to use CVS, don't.
As of 1.0, we're supporting Windows less and less. You can read the code yourself - if you have trouble doing so, our windows code is not for you.
For Windows, we now require Visual Studios 2005, with the Windows SDK and Direct X.
We also have the Visual Studio 6.0 projects included, but they most likely do not work. You will need to install nasm in the VC98/bin directory before compiling. You should get nasm-0.98.22-win32.zip from the nasm web site. You will need to rename nasmw.exe to nasm.exe. We require Service Pack 5.
We also require you to install the SDL project into where ever you are compiling the mpeg4ip project. Download the latest SDL Development Library, Windows Visual C++ zip file and extract it into the mpeg4ip top level directory. You must rename it SDL, instead of SDL-1.2.x. You should get docs, an include directory and a lib directory. You may also have to install lib/SDL.dll into a Windows directory (c:\winnt\system32 or the equivalent.
If you're looking at these directions, and going "Huh?" - this project may not be right for you. Don't contact us. Like I said, we're supporting Windows less and less.
If you're using Visual Studios 2005, use the tools.sln or player.sln solutions. The executables and DLLs will be built into the Debug and Release directories under mpeg4ip.
We recommend installing DirectX 8.1 or later. If that is not possible, and you have problems with video, try uncommenting out #define OLD_SURFACE in player/src/video.cpp. Other than that, you'll have to figure it out yourself - libsdl.org is a good resource.
For Visual Studio 6 (which no longer works), use the encoding60.dsw project for encoding tools, and the player/src/player60.dsw for the player. You may have problems with certain projects not being able to load. This is due to a problem with the way Windows ends lines in their files (using a Ctrl-M, instead of a CR-LF like DOS). For some reason, I have never had these problems. You can use a unix to DOS file converter, or just open the .dsp or .dsw in Wordpad and save it.
To run mp4player other than in Visual Studio, install mp4player.exe into a directory on your Window's path (install them into the same directory). You will also need to install all the plugin .dlls into the same directory as well. Look for the *_plugin projects in player60.dsw.
In encoding60.dsw and player60.dsw, if you do a batch build, you will see several projects that do not build correctly - these common with IPv6. This is as designed. If you want these to build, please see the appropriate package creator for the correct steps - they are not supported in mpeg4ip.
For windows GUI player, see the player README.
We only support 10.3.2 (Panther) and later. You should have the tools installed.
You should install SDL-1.2.7 (or greater) from source, not from the binaries. We require the sdl-config script, which is not installed with the binaries.
We will build the version native to your platform. To build
universal binarys, use the
You may get an error that libtoolize is not installed correctly. Mac OSX comes with libtool installed as glibtool. Automake (which doesn't come included with OSX) does not correct its scripts.
To fix this problem, do a link of libtoolize to glibtoolize (ie:
The player works on MacOSX, but we have not developed a GUI for it - there is some problem with SDL that requires the video to be in the main thread, like windows.
Right now, it works well enough for testing.
If you ran 'make install' with the defaults, then all the MPEG4IP executables will end up in /usr/local/bin.
For server side (or content creation) tools, you get:
These are described in more detail in doc/encoding.htm
A few debugging tools are also included:
For playback, you get:
We do not have the complete version of FAAD, so the faad standalone decoder will not build. FAAD as of faad2-2.0-rc3 contains a gmp4player plugin - we recommend download and using that. We've included the latest version of the plugin code in player/plugin/audio/faad2, if you have compile problems. Copy the files into the faad/plugins/mpeg4ip directory. You may have to remove the rpm rule in the top level Makefile to get it to make.
If you're going to start hacking, a map of the territory may prove useful:
mpeg4ip - top level project directory
config - autoconf files
doc - the minimal doc we've written so far
include - project wide includes
common - shared code
lib - project wide libraries
player - player specific code
server - server specific code
util - generally useful utilities
We're not only supporters of open source, we're supporters of open standards! We've attempted to use the publically defined standards as much as possible. Here's what we believe we are following. If you find something non-compliant, please let us know. We certainly will want to fix it.
Here are the citations:
ISO/IEC 14496-1:2001 MPEG-4 Systems (includes MP4 file format)
The IETF RFCs can be found in doc/ietf, or online at the ietf
The MP4 file format is derived from Apple's QuickTime file format. That specification is Apple Computer QuickTime File Format, June 28 2000 and is available through the QT developer site.
We're also involved in the Internet Streaming Media Alliance (ISMA) which seeks to standardize the protocols and formats used for streaming. We believe MPEG4IP is interoperable with that organization's 1.0 and 2.0 Technical Specification. For more information see http://www.isma.tv
Latency in mpeg4ip
There have been enough questions to merit a place in the README about the latency in mpeg4ip.
First of all, mpeg4ip is not designed for video conferencing or real time display of data. Most streaming products are not (look at Real or Quicktime's buffering schemes - QT buffers 3.0 seconds of data, while Real can buffer up to 30-40 seconds).
When you need to look at latency in a streaming environment, you need to look at each potential piece. In the simplest case (mp4live to mp4player), there are 5 potential places where latency can take place - at mp4live, in the kernel IP transmit stack, in the network, in the kernel IP receive stack and in the player.
Mp4live has very little latency. There is no latency from when the video frame is encoded with mpeg4. With MPEG-2, the B frames require a 2 frame latency period. In audio, mpeg style codecs usually require 2 or 3 frames to begin generating output. (if using AAC, frames tend to be 1024 samples. MP3 tends to be 1152 samples, but can change based on sampling frequency and bit rate. Samples can be converted to seconds by dividing by the sample rate).
You can remove some of this latency by not bundling multiple frames in an RTP packet. This can be set using the rtpMaxFramesPerPacket in the Audio Profile configuration file in mp4live. You can also use G.711 or AMR codecs.
The kernel IP stacks can have some latency built in, as well. Probably not too much, but you should be aware that it can exist (look at a sniffer trace of DSS output and you'll see what I mean).
Network latency can occur, as well. Collisions, etc, can happen, especially if server and player are not on the same network.
If you're using a streaming server as a relay, latency will be introduced there, as well.
Finally, the player has latency as well. For streaming, we tend to buffer 2 seconds for each stream. This is changeable by setting the RtpBufferTimeMsec variable in the .gmp4player_rc file (for Windows users, you'll have to change the registry to do so). The value is in milliseconds, and you need to set it to a non-zero value (0 indicates the default value).
See the PLAYER_README.html in doc.
We welcome all contributions back to mpeg4ip. Email is the best way to accomplish this.
Please do a
Please understand that we are trying to keep this a multi-platform project, so compiling on multiple platforms is very much welcome.
You can use the forums or mailing list before you start to work out how to do a certain project.
=== END OF README ===