The architecture of VLC media framework

 VLC is a free and open source cross-platform multimedia player and framework that plays most multimedia files as well as DVD, Audio CD, VCD, and various streaming protocols. Technically it is a software package that handles media on a computer and through a network. It offers an intuitive APIand a modular architecture to easily add support for new codecs, container formats and transmission protocols. Most known package that comes from VLC is the player which is commonly used on Windows, Linux and OSX.

According Videolan website, this project is quite complex to understand:

VLC media player is a large and complex piece of software. It also uses a large number of dependencies.
Being open source allows VLC development to benefit from a large community of developers worldwide. 
However, entering a project such as VLC media player can be long and complex for new developers.

Code is written by expert C hackers and sometimes it is very hard to understand. Probably a book is needed to explain how VLC works. I’ll try in few words to summarize what I found in my mining. Let’s start by the highlevel architecture:

VLC has a core and a lot of modules (between 200 and 400 depending on the build). VLC cannot do much without modules, since modules provide most of the functionalities you expect. There are following modules:

• libVLCcore is the central core of the framework. It provides an object oriented layer to C, module loading/unloading and a set of abstract functionalities regarding multimedia: input, multiplexing, demultiplexing, audio output, video output.

• modules provide concrete functionalities of the framework. Modules are categorized according their capabilities. There are modules for input management (file, network, cd), modules for codecs (mp3, divx, …), modules for gui (textual, web, telnet, qt-based, macosx native).

• External libraries As there are lot of modules, there are lot of external dependencies. There is a page onVLC Developer’s Wiki that tries to maintains these deps.

• vlc (main) – is the main of the player. It initializes libVLC and launch user interface.

Modules

As previously said VLC without modules is useless. Modules can be categorized according their capabilities. Acapability should be thought as something similar to an Object Oriented interface. Module loading is simple to explain: when VLC needs a module with a particular capability (for example when it needs a “decoder”), it opens all modules for that capability, until one matches. It opens them in decreasing score order (bigger score first, smaller ones at the end), and runs the Open() function of the module. When one module returns OK, VLC uses this module. You can read further here. Following picture represents principal module capabilities implemented inVLC

VLC can have different flavors of user interfaces. Every kind of interface is a module (here you can find a basic sound desing principle: “separate user logic from user interface”). In actual distribution there are UIs based on QTfor Windows/Linux, on Cocoa/Objective-C for OSX, on NCurses for console UI and one for Maemo device.

Just to have a picture on how modules are physically structured, if you open modules/audio_output directory you find something that looks like

where there are implementations for several audio output technologies: ALSA, Pulse and OSS for Linux, DirectXfor Windows

Object Oriented Layer – VLC Object

VLC defines an Object Oriented Layer which is used to manage models. At the basis there is a VLC Object which is defined as the following:

/***************************************************************************** * The vlc_object_t type. Yes, it's that simple :-) *****************************************************************************//** The main vlc_object_t structure */struct vlc_object_t{    VLC_COMMON_MEMBERS};

VLC_COMMON_MEMBERS is a C “#define” coded as the following:

/* VLC_COMMON_MEMBERS : members common to all basic vlc objects */#define VLC_COMMON_MEMBERS                                       \/** \name VLC_COMMON_MEMBERS                                     \ * these members are common for all vlc objects                  \ */                                                              \/**@{*/                                                          \  const char *psz_object_type;                                   \                                                             \  /* Messages header */                                          \  char *psz_header;                                              \  int  i_flags;                                                  \                                                             \  /* Object properties */                                        \  volatile bool b_die;                /**< set by the outside */ \  bool b_force;   /**< set by the outside (eg. module_need()) */ \                                                             \  /* Stuff related to the libvlc structure */                    \  libvlc_int_t *p_libvlc;         /**< (root of all evil) - 1 */ \                                                             \  vlc_object_t *  p_parent;                   /**< our parent */ \                                                             \/**@}*/

If you are used to work with Object Oriented Languages you find this structure quite naïve. In src/misc/objects.cthere are methods to create, destroy and find VLC objects: vlc_custom_create(..),vlc_object_destroy(...), vlc_object_find(...). For example this piece of code creates a new vlc object:

p_libvlc = vlc_custom_create( (vlc_object_t *)NULL, sizeof (*priv),VLC_OBJECT_GENERIC, "libvlc" );

LibVLC instance

Root class of the library is the libvlc instance which is defined as:

struct libvlc_instance_t{  libvlc_int_t *p_libvlc_int;  libvlc_vlm_t  libvlc_vlm;  unsigned      ref_count;  int           verbosity;  vlc_mutex_t   instance_lock;  struct libvlc_callback_entry_list_t *p_callback_list;};

where libvlc_int_t is a vlc object. During boot it is linked to a libvlc_priv_t type which contains main data and structures of the library. For example playlist_t (src/playlist/) is a central abstraction and it represents a music playlist. vlm_t is the root object of the server core that allows streaming of multiple media at the same time.VLC can have different user interfaces (graphical, textual, irc-based, …). These are modelled by p_intf variable which points to the user interface root object.

/** * Private LibVLC instance data. */typedef struct libvlc_priv_t{    . . . . . .    . . . . . .     bool               playlist_active;    /* Messages */    msg_bank_t        *msg_bank;    ///< The message bank    /* Singleton objects */    module_t          *p_memcpy_module;  ///< Fast memcpy plugin used    playlist_t        *p_playlist; //< the playlist singleton    vlm_t             *p_vlm;  ///< the VLM singleton (or NULL)    vlc_object_t      *p_dialog_provider; ///< dialog provider    httpd_t           *p_httpd; ///< HTTP daemon (src/network/httpd.c)    . . . . . .     . . . . . .    /* Interfaces */    struct intf_thread_t *p_intf; ///< Interfaces linked-list} libvlc_priv_t;

Module abstraction and communication

As previously said VLC is made by a large set of modules. On developer’s website there is an how-to guide to develop your own module. We can say that a module has to respect two contracts:

• from the initialization side it has to implement open() and close() method. These methods are used to initialize/deinitialize module.
• from the functionality side it has to implement methods according capabilities.

For example a module of the audio output category has to implement following methods:

..................int aout_VolumeDown( vlc_object_t * p_object, int i_nb_steps,audio_volume_t * pi_volume )int aout_VolumeUp( vlc_object_t * p_object, int i_nb_steps, audio_volume_t * pi_volume )int aout_VolumeGet( vlc_object_t * p_object, audio_volume_t * pi_volume )int aout_ToggleMute( vlc_object_t * p_object, audio_volume_t * pi_volume )..................

A module from video output category has to implement following methods:

....................picture_t *vout_CreatePicture( vout_thread_t *p_vout, bool b_progressive,bool b_top_field_first,                               unsigned int i_nb_fields )void vout_DisplayPicture( vout_thread_t *p_vout, picture_t *p_pic )void DestroyPicture( vout_thread_t *p_vout, picture_t *p_picture )void vout_DropPicture( vout_thread_t *p_vout, picture_t *p_pic  )....................

There is another fundamental concept in VLC that is build upon the “Observer-Observable” pattern and that it helps to decouple communication between modules. It is the concept of VLC variable. A VLC variable is a value that can be assigned to any VLC object. Interesting thing is that variables can fire __callbacks_ to interested viewers. Let’s see how VLC variables work.

You have first to create variable using: var_Create( p_libvlc, name, VLC_VAR_INTEGER );p_libvlc is a pointer to a vlc object, name is variable’s name while VLC_VAR_INTEGER is its type. When you want to set a value to that variable you write something like var_SetInteger( p_libvlc, name, temp_value);.temp_value is the value we want to insert. There is a var_getInteger to read value back.

For triggering callbacks there is a var_AddCallback( vlc_object_t *p_this, const char *psz_name,vlc_callback_t pf_callback, void *p_data ) where psz_name is the variable name,pf_callback is the function pointer and p_data is a generic data passed to the function pointer as argument. When you change value of a variable a TriggerCallback(..) method is called and all observers of the variable are notified. This is the most complex implementation of Observer/Observable pattern I have seen in life.

How this concept is applied concretely ? If we look at the ncurses textual user interface of VLC there is a piece of code that states: var_AddCallback( p_playlist, "playlist-item-append", PlaylistChanged, p_intf ); 
This means that when the “playlist-item-append” variable is changed, a PlaylistChanged() callback method is invoked (which is local to the user interface).

VLC variables is the mechanism used to decouple communication between modules.

Language/Tools Summary

Language
C for core; 
C++/Objective-C for GUICompilerGCCBuildingGNU Autotools ; BuildbotTestingno formal form Unit TestingDocumentationDoxygen