GStreamer基础教程06——媒体格式和pad的Capabilities

原文：https://gstreamer.freedesktop.org/documentation/tutorials/basic/index.html

译文原文：http://blog.csdn.net/sakulafly/article/details/21299519

原码：git clone git://anongit.freedesktop.org/gstreamer/gst-docs

编译方式：gcc basic-tutorial-6.c -o basic-tutorial-6`pkg-config --cflags --libs gstreamer-1.0`

Need help?

If you need help to compile this code, refer to the Building the tutorials section for your platform: Linux, Mac OS X or Windows, or use this specific command on Linux:

gcc basic-tutorial-6.c -o basic-tutorial-6 `pkg-config --cflags --libs gstreamer-1.0`

If you need help to run this code, refer to the Running the tutorials section for your platform: Linux, Mac OS X or Windows.

This tutorial simply displays information regarding the Pad Capabilities in different time instants.

Required libraries: gstreamer-1.0

目标

      Pad的Capabilities是一个GStreamer element的基础，因为framework大部分时间是自动处理的，所以我们几乎感觉不到它的存在。本教程比较偏向原理，介绍了：
      什么是Pad Capabilities
      如何获得这个东西
      什么时候应该去获得这个东西
      为什么你需要了解他们

介绍
Pads
      Pads允许信息进入或者离开一个element——就像曾经展示过得一样。这个Capabilities（或者简单地叫做Caps）就是指定哪些信息可以通过Pad来传输。例如：“”RGB视频，尺寸为320x200并且每秒30帧“或者”16位的音频采样，5.1声道，每秒采样44.1k“甚至可以是类似于mp3/h264之类的压缩格式。
      Pads支持多重Capabilities（比如，一个视频的sink可以支持RGB输出或者YUV输出），Capabilities可以指定一个范围而不必须是一个特定值（比如，一个音频sink可以支持从1~48000的采样率）。然而，数据从一个pad流向另一个pad的时候，必须是一个双方都能支持的格式。某一种数据形式是两个pad都能支持的，这样Pads的Capabilities就固定下来（只有一种类型，并且不再是一个数据区间了，而是固定的数值），这个过程被称为协商。下面的例子会更清楚的说明这一点。
      为了两个element可以连接，他们必须有一个共同的Capabilities子集（否则它们肯定不能互相连接）。这就是Capabilities存在的主要目的。
      作为一个应用开发者，我们通常都是用连接一个个element的方法来建立pipeline的。在这个例子中，你需要了解你使用的element的Pad的Caps。
Pad模板
      Pad是由Pad模板创建的，模板里面会列出一个Pad所有可能的Capabilities。模板对于创建几个相似的Pad是很有帮助的，但也会比较早就判断出两个element是否可以相连：如果连两个Pad的模板都不具备共同的子集的化，就没必要进行更深层的协商了。
      Pad模板检查是协商流程的第一步。随着流程的逐步深入，Pad会正式初始化，也会确定他们的Capability（除非协商失败了）。

Capabilities例子

SINK template: 'sink'  Availability: Always  Capabilities:    audio/x-raw-int               signed: true                width: 16                depth: 16                 rate: [ 1, 2147483647 ]             channels: [ 1, 2 ]    audio/x-raw-int               signed: false                width: 8                depth: 8                 rate: [ 1, 2147483647 ]             channels: [ 1, 2 ]

      这是一个element的永久sink pad。它支持2种媒体格式，都是音频的原始数据（audio/x-raw-int），16位的符号数和8位的无符号数。方括号表示一个范围，例如，频道（channels）的范围是1到2.

SRC template: 'src'  Availability: Always  Capabilities:    video/x-raw-yuv                width: [ 1, 2147483647 ]               height: [ 1, 2147483647 ]            framerate: [ 0/1, 2147483647/1 ]               format: { I420, NV12, NV21, YV12, YUY2, Y42B, Y444, YUV9, YVU9, Y41B, Y800, Y8  , GREY, Y16 , UYVY, YVYU, IYU1, v308, AYUV, A420 } 

      video/x-raw-yur表示这个source pad用YUV格式输出视频。它支持一个很广的维数和帧率，一系列的YUV格式（用花括号列出了）。所有这些格式都显示不同的图像打包和子采样程度。

最后的总结 

      你可以使用gst-inspect-0.10这个工具（在后面教程中会介绍）来查看一下element的Caps。

      记住有些element会查看底层的硬件来确定支持的格式和能提供的Pad的Caps（通常在进入READY或更后面的状态）。因此，这个显示的Caps由于平台的不同是不一样的，甚至可能在每一次运行都不相同（这种情况很少）。

      本教程会创建2个element，显示它们各自的Pad模板，连接起来并把pipeline设置成PLAY状态。在每个状态切换的过程中，sink element的Pad的Capabilities会显示出来，这样你就可以观察到协商过程是如何进行的，最后Pad的Caps是怎么定下来的。

一个简单地Pad Capabilities例子

#include <gst/gst.h>  /* Functions below print the Capabilities in a human-friendly format */static gboolean print_field (GQuark field, const GValue * value, gpointer pfx) {  gchar *str = gst_value_serialize (value);    g_print ("%s  %15s: %s\n", (gchar *) pfx, g_quark_to_string (field), str);  g_free (str);  return TRUE;}  static void print_caps (const GstCaps * caps, const gchar * pfx) {  guint i;    g_return_if_fail (caps != NULL);    if (gst_caps_is_any (caps)) {    g_print ("%sANY\n", pfx);    return;  }  if (gst_caps_is_empty (caps)) {    g_print ("%sEMPTY\n", pfx);    return;  }    for (i = 0; i < gst_caps_get_size (caps); i++) {    GstStructure *structure = gst_caps_get_structure (caps, i);        g_print ("%s%s\n", pfx, gst_structure_get_name (structure));    gst_structure_foreach (structure, print_field, (gpointer) pfx);  }}  /* Prints information about a Pad Template, including its Capabilities */static void print_pad_templates_information (GstElementFactory * factory) {  const GList *pads;  GstStaticPadTemplate *padtemplate;    g_print ("Pad Templates for %s:\n", gst_element_factory_get_longname (factory));  if (!factory->numpadtemplates) {    g_print ("  none\n");    return;  }    pads = factory->staticpadtemplates;  while (pads) {    padtemplate = (GstStaticPadTemplate *) (pads->data);    pads = g_list_next (pads);        if (padtemplate->direction == GST_PAD_SRC)      g_print ("  SRC template: '%s'\n", padtemplate->name_template);    else if (padtemplate->direction == GST_PAD_SINK)      g_print ("  SINK template: '%s'\n", padtemplate->name_template);    else      g_print ("  UNKNOWN!!! template: '%s'\n", padtemplate->name_template);        if (padtemplate->presence == GST_PAD_ALWAYS)      g_print ("    Availability: Always\n");    else if (padtemplate->presence == GST_PAD_SOMETIMES)      g_print ("    Availability: Sometimes\n");    else if (padtemplate->presence == GST_PAD_REQUEST) {      g_print ("    Availability: On request\n");    } else      g_print ("    Availability: UNKNOWN!!!\n");        if (padtemplate->static_caps.string) {      g_print ("    Capabilities:\n");      print_caps (gst_static_caps_get (&padtemplate->static_caps), "      ");    }        g_print ("\n");  }}  /* Shows the CURRENT capabilities of the requested pad in the given element */static void print_pad_capabilities (GstElement *element, gchar *pad_name) {  GstPad *pad = NULL;  GstCaps *caps = NULL;    /* Retrieve pad */  pad = gst_element_get_static_pad (element, pad_name);  if (!pad) {    g_printerr ("Could not retrieve pad '%s'\n", pad_name);    return;  }    /* Retrieve negotiated caps (or acceptable caps if negotiation is not finished yet) */  caps = gst_pad_get_negotiated_caps (pad);  if (!caps)    caps = gst_pad_get_caps_reffed (pad);    /* Print and free */  g_print ("Caps for the %s pad:\n", pad_name);  print_caps (caps, "      ");  gst_caps_unref (caps);  gst_object_unref (pad);}  int main(int argc, char *argv[]) {  GstElement *pipeline, *source, *sink;  GstElementFactory *source_factory, *sink_factory;  GstBus *bus;  GstMessage *msg;  GstStateChangeReturn ret;  gboolean terminate = FALSE;    /* Initialize GStreamer */  gst_init (&argc, &argv);     /* Create the element factories */  source_factory = gst_element_factory_find ("audiotestsrc");  sink_factory = gst_element_factory_find ("autoaudiosink");  if (!source_factory || !sink_factory) {    g_printerr ("Not all element factories could be created.\n");    return -1;  }    /* Print information about the pad templates of these factories */  print_pad_templates_information (source_factory);  print_pad_templates_information (sink_factory);    /* Ask the factories to instantiate actual elements */  source = gst_element_factory_create (source_factory, "source");  sink = gst_element_factory_create (sink_factory, "sink");    /* Create the empty pipeline */  pipeline = gst_pipeline_new ("test-pipeline");    if (!pipeline || !source || !sink) {    g_printerr ("Not all elements could be created.\n");    return -1;  }    /* Build the pipeline */  gst_bin_add_many (GST_BIN (pipeline), source, sink, NULL);  if (gst_element_link (source, sink) != TRUE) {    g_printerr ("Elements could not be linked.\n");    gst_object_unref (pipeline);    return -1;  }    /* Print initial negotiated caps (in NULL state) */  g_print ("In NULL state:\n");  print_pad_capabilities (sink, "sink");    /* Start playing */  ret = gst_element_set_state (pipeline, GST_STATE_PLAYING);  if (ret == GST_STATE_CHANGE_FAILURE) {    g_printerr ("Unable to set the pipeline to the playing state (check the bus for error messages).\n");  }    /* Wait until error, EOS or State Change */  bus = gst_element_get_bus (pipeline);  do {    msg = gst_bus_timed_pop_filtered (bus, GST_CLOCK_TIME_NONE, GST_MESSAGE_ERROR | GST_MESSAGE_EOS |        GST_MESSAGE_STATE_CHANGED);      /* Parse message */    if (msg != NULL) {      GError *err;      gchar *debug_info;          switch (GST_MESSAGE_TYPE (msg)) {        case GST_MESSAGE_ERROR:          gst_message_parse_error (msg, &err, &debug_info);          g_printerr ("Error received from element %s: %s\n", GST_OBJECT_NAME (msg->src), err->message);          g_printerr ("Debugging information: %s\n", debug_info ? debug_info : "none");          g_clear_error (&err);          g_free (debug_info);          terminate = TRUE;          break;        case GST_MESSAGE_EOS:          g_print ("End-Of-Stream reached.\n");          terminate = TRUE;          break;        case GST_MESSAGE_STATE_CHANGED:          /* We are only interested in state-changed messages from the pipeline */          if (GST_MESSAGE_SRC (msg) == GST_OBJECT (pipeline)) {            GstState old_state, new_state, pending_state;            gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);            g_print ("\nPipeline state changed from %s to %s:\n",                gst_element_state_get_name (old_state), gst_element_state_get_name (new_state));            /* Print the current capabilities of the sink element */            print_pad_capabilities (sink, "sink");          }          break;        default:          /* We should not reach here because we only asked for ERRORs, EOS and STATE_CHANGED */          g_printerr ("Unexpected message received.\n");          break;      }      gst_message_unref (msg);    }  } while (!terminate);    /* Free resources */  gst_object_unref (bus);  gst_element_set_state (pipeline, GST_STATE_NULL);  gst_object_unref (pipeline);  gst_object_unref (source_factory);  gst_object_unref (sink_factory);  return 0;}

工作流程

      这里的print_field，print_caps和print_pad_templates仅仅简单的用方便阅读的方式显示Capabilities结构。如果你想学到GstCaps数据结构的内在组织，那么请阅读以下Gstreamer文档关于Pad Caps的部分。

/* Shows the CURRENT capabilities of the requested pad in the given element */static void print_pad_capabilities (GstElement *element, gchar *pad_name) {  GstPad *pad = NULL;  GstCaps *caps = NULL;    /* Retrieve pad */  pad = gst_element_get_static_pad (element, pad_name);  if (!pad) {    g_printerr ("Could not retrieve pad '%s'\n", pad_name);    return;  }    /* Retrieve negotiated caps (or acceptable caps if negotiation is not finished yet) */  caps = gst_pad_get_negotiated_caps (pad);  if (!caps)    caps = gst_pad_get_caps_reffed (pad);    /* Print and free */  g_print ("Caps for the %s pad:\n", pad_name);  print_caps (caps, "      ");  gst_caps_unref (caps);  gst_object_unref (pad);}

      gst_element_get_static_pad()方法会从给定的element里面取得Pad的名字。这个Pad是静态的，因为它会一直存在。要进一步了解Pad请阅读GStreamer文档关于Pad的部分。

      然后我们调用gst_pad_get_negotiated_caps()方法来获得Pad当前的Capabilities，无论是否已经固定下来，都会依赖协商过程的状态。这时Capabilities可能是不存在的，如果遇到这种情况，我们就调用gst_pad_get_caps_reffed()方法来建立一个当前可以接受的Pad Capabilities。这个所谓的当前可接受的Caps时Pad Template的在NULL状态下的Caps，但后面这个可能会改变，因为还会查询实际的硬件。

      gst_pad_get_caps_reffed()通常来说比gst_pad_get_caps()快一点，而且如果我们不需要改变获得的Caps的话也足够用了。然后我们就打印出这些Capabilities。

  /* Create the element factories */  source_factory = gst_element_factory_find ("audiotestsrc");  sink_factory = gst_element_factory_find ("autoaudiosink");  if (!source_factory || !sink_factory) {    g_printerr ("Not all element factories could be created.\n");    return -1;  }    /* Print information about the pad templates of these factories */  print_pad_templates_information (source_factory);  print_pad_templates_information (sink_factory);    /* Ask the factories to instantiate actual elements */  source = gst_element_factory_create (source_factory, "source");  sink = gst_element_factory_create (sink_factory, "sink");

      在前面的教程中我们直接使用gst_element_factory_make()方法来创建element，而忽略了工厂本身，我们现在补上这一部分。一个GstElementFactory是负责通过一个工厂名来实例化一个element的。

      你可以使用gst_element_factory_find()来创建一个“videotestsrc”工厂，然后通过gst_element_factory_create()来实例化多个“videotestsrc” element。gst_element_factory_make()实际上只是这个过程的一个封装。

      通过工程，Pad模板实际上已经可以访问了，所以工厂一建立我们立刻打印这些信息。

      我们跳过pipeline的创建和启动部分，直接跳到状态切换消息的处理：

        case GST_MESSAGE_STATE_CHANGED:          /* We are only interested in state-changed messages from the pipeline */          if (GST_MESSAGE_SRC (msg) == GST_OBJECT (pipeline)) {            GstState old_state, new_state, pending_state;            gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);            g_print ("\nPipeline state changed from %s to %s:\n",                gst_element_state_get_name (old_state), gst_element_state_get_name (new_state));            /* Print the current capabilities of the sink element */            print_pad_capabilities (sink, "sink");          }          break;

      这里只是在每次pipeline状态切换的时候简单地打印当前的Pad Caps。你可以看到，在输出信息里面，初始的Caps（Pad模板的Caps）是如何逐步变化直到确定下来的（仅包含一个类型而且数值固定）。

Conclusion

This tutorial has shown:

• What are Pad Capabilities and Pad Template Capabilities.
• How to retrieve them with gst_pad_get_current_caps() or gst_pad_query_caps().
• That they have different meaning depending on the state of the pipeline (initially they indicate all the possible Capabilities, later they indicate the currently negotiated Caps for the Pad).
• That Pad Caps are important to know beforehand if two elements can be linked together.
• That Pad Caps can be found using the gst-inspect-1.0 tool described in Basic tutorial 10: GStreamer tools.

Next tutorial shows how data can be manually injected into and extracted from the GStreamer pipeline.

Remember that attached to this page you should find the complete source code of the tutorial and any accessory files needed to build it. It has been a pleasure having you here, and see you soon!