Using libavformat and libavcodec
来源:互联网 发布:python 字典查找value 编辑:程序博客网 时间:2024/05/22 17:19
The libavformat and libavcodec libraries that come with ffmpegare a great way of accessing a large variety of video file formats.Unfortunately, there is no real documentation on using these libraries in yourown programs (at least I couldn't find any), and the example programs aren'treally very helpful either.
This situation meant that, when I used libavformat/libavcodec on a recentproject, it took quite a lot of experimentation to find out how to use them.Here's what I learned - hopefully I'll be able to save others from having to go through the same trial-and-error process. There's also a small demo program that you can download. The codeI'll present works with libavformat/libavcodec as included in version 0.4.8 offfmpeg (the most recent version as I'm writing this). If you find that laterversions break the code, please let me know.
In this document, I'll only cover how to read video streams from a file;audio streams work pretty much the same way, but I haven't actually used them,so I can't present any example code.
In case you're wondering why there are two libraries, libavformat andlibavcodec: Many video file formats (AVI being a prime example) don't actuallyspecify which codec(s) should be used to encode audio and video data; theymerely define how an audio and a video stream (or, potentially, severalaudio/video streams) should be combined into a single file. This is whysometimes, when you open an AVI file, you get only sound, but no picture -because the right video codec isn't installed on your system. Thus,libavformat deals with parsing video files and separating the streamscontained in them, and libavcodec deals with decoding raw audio and videostreams.
Opening a Video File
First things first - let's look at how to open a video file and get at thestreams contained in it. The first thing we need to do is to initializelibavformat/libavcodec:
av_register_all();
This registers all available file formats and codecs with the library so theywill be used automatically when a file with the corresponding format/codec isopened. Note that you only need to call av_register_all() once, soit's probably best to do this somewhere in your startup code. If you like,it's possible to register only certain individual file formats and codecs, butthere's usually no reason why you would have to do that.
Next off, opening the file:
AVFormatContext *pFormatCtx;
const char *filename="myvideo.mpg";
// Open video file
if(av_open_input_file(&pFormatCtx, filename, NULL, 0, NULL)!=0)
handle_error(); // Couldn't open file
The last three parameters specify the file format, buffer size and formatparameters; by simply specifying NULL or 0 we ask libavformat to auto-detect the format and use a default buffer size. Replace handle_error() withappropriate error handling code for your application.
Next, we need to retrieve information about the streams contained in the file:
// Retrieve stream information
if(av_find_stream_info(pFormatCtx)<0)
handle_error(); // Couldn't find stream information
This fills the streams field of the AVFormatContext withvalid information. As a debugging aid, we'll dump this information ontostandard error, but of course you don't have to do this in a productionapplication:
dump_format(pFormatCtx, 0, filename, false);
As mentioned in the introduction, we'll handle only video streams, not audiostreams. To make things nice and easy, we simply use the first video stream wefind:
int i, videoStream;
AVCodecContext *pCodecCtx;
// Find the first video stream
videoStream=-1;
for(i=0; i<pFormatCtx->nb_streams; i++)
if(pFormatCtx->streams[i]->codec.codec_type==CODEC_TYPE_VIDEO)
{
videoStream=i;
break;
}
if(videoStream==-1)
handle_error(); // Didn't find a video stream
// Get a pointer to the codec context for the video stream
pCodecCtx=&pFormatCtx->streams[videoStream]->codec;
OK, so now we've got a pointer to the so-called codec context for our videostream, but we still have to find the actual codec and open it:
AVCodec *pCodec;
// Find the decoder for the video stream
pCodec=avcodec_find_decoder(pCodecCtx->codec_id);
if(pCodec==NULL)
handle_error(); // Codec not found
// Inform the codec that we can handle truncated bitstreams -- i.e.,
// bitstreams where frame boundaries can fall in the middle of packets
if(pCodec->capabilities & CODEC_CAP_TRUNCATED)
pCodecCtx->flags|=CODEC_FLAG_TRUNCATED;
// Open codec
if(avcodec_open(pCodecCtx, pCodec)<0)
handle_error(); // Could not open codec
(So what's up with those "truncated bitstreams"? Well, as we'll see in amoment, the data in a video stream is split up into packets. Since the amountof data per video frame can vary, the boundary between two video frames neednot coincide with a packet boundary. Here, we're telling the codec that we canhandle this situation.)
One important piece of information that is stored in theAVCodecContext structure is the frame rate of the video. To allow fornon-integer frame rates (like NTSC's 29.97 fps), the rate is stored as afraction, with the numerator in pCodecCtx->frame_rate and thedenominator in pCodecCtx->frame_rate_base. While testing the librarywith different video files, I noticed that some codecs (notably ASF) seem to fill these fields incorrectly (frame_rate_base contains 1 instead of1000). The following hack fixes this:
// Hack to correct wrong frame rates that seem to be generated by some
// codecs
if(pCodecCtx->frame_rate>1000 && pCodecCtx->frame_rate_base==1)
pCodecCtx->frame_rate_base=1000;
Note that it shouldn't be a problem to leave this fix in place even if the bug is corrected some day - it's unlikely that a video would have a frame rate ofmore than 1000 fps.
One more thing left to do: Allocate a video frame to store the decoded imagesin:
AVFrame *pFrame;
pFrame=avcodec_alloc_frame();
That's it! Now let's start decoding some video.
Decoding Video Frames
As I've already mentioned, a video file can contain several audio and videostreams, and each of those streams is split up into packets of a particularsize. Our job is to read these packets one by one using libavformat, filterout all those that aren't part of the video stream we're interested in, andhand them on to libavcodec for decoding. In doing this, we'll have to takecare of the fact that the boundary between two frames can occur in the middleof a packet.
Sound complicated? Lucikly, we can encapsulate this whole process in a routinethat simply returns the next video frame:
bool GetNextFrame(AVFormatContext *pFormatCtx, AVCodecContext *pCodecCtx,
int videoStream, AVFrame *pFrame)
{
static AVPacket packet;
static int bytesRemaining=0;
static uint8_t *rawData;
static bool fFirstTime=true;
int bytesDecoded;
int frameFinished;
// First time we're called, set packet.data to NULL to indicate it
// doesn't have to be freed
if(fFirstTime)
{
fFirstTime=false;
packet.data=NULL;
}
// Decode packets until we have decoded a complete frame
while(true)
{
// Work on the current packet until we have decoded all of it
while(bytesRemaining > 0)
{
// Decode the next chunk of data
bytesDecoded=avcodec_decode_video(pCodecCtx, pFrame,
&frameFinished, rawData, bytesRemaining);
// Was there an error?
if(bytesDecoded < 0)
{
fprintf(stderr, "Error while decoding frame/n");
return false;
}
bytesRemaining-=bytesDecoded;
rawData+=bytesDecoded;
// Did we finish the current frame? Then we can return
if(frameFinished)
return true;
}
// Read the next packet, skipping all packets that aren't for this
// stream
do
{
// Free old packet
if(packet.data!=NULL)
av_free_packet(&packet);
// Read new packet
if(av_read_packet(pFormatCtx, &packet)<0)
goto loop_exit;
} while(packet.stream_index!=videoStream);
bytesRemaining=packet.size;
rawData=packet.data;
}
loop_exit:
// Decode the rest of the last frame
bytesDecoded=avcodec_decode_video(pCodecCtx, pFrame, &frameFinished,
rawData, bytesRemaining);
// Free last packet
if(packet.data!=NULL)
av_free_packet(&packet);
return frameFinished!=0;
}
Now, all we have to do is sit in a loop, calling GetNextFrame() untilit returns false. Just one more thing to take care of: Most codecs returnimages in YUV 420 format (one luminance and two chrominance channels, with thechrominance channels samples at half the spatial resolution of the luminancechannel). Depending on what you want to do with the video data, you may wantto convert this to RGB. (Note, though, that this is not necessary if all youwant to do is display the video data; take a look at the X11 Xvideo extension,which does YUV-to-RGB and scaling in hardware.) Fortunately, libavcodecprovides a conversion routine called img_convert, which doesconversion between YUV and RGB as well as a variety of other image formats.The loop that decodes the video thus becomes:
while(GetNextFrame(pFormatCtx, pCodecCtx, videoStream, pFrame))
{
img_convert((AVPicture *)pFrameRGB, PIX_FMT_RGB24, (AVPicture*)pFrame,
pCodecCtx->pix_fmt, pCodecCtx->width, pCodecCtx->height);
// Process the video frame (save to disk etc.)
DoSomethingWithTheImage(pFrameRGB);
}
The RGB image pFrameRGB (of type AVFrame *) is allocatedlike this:
AVFrame *pFrameRGB;
int numBytes;
uint8_t *buffer;
// Allocate an AVFrame structure
pFrameRGB=avcodec_alloc_frame();
if(pFrameRGB==NULL)
handle_error();
// Determine required buffer size and allocate buffer
numBytes=avpicture_get_size(PIX_FMT_RGB24, pCodecCtx->width,
pCodecCtx->height);
buffer=new uint8_t[numBytes];
// Assign appropriate parts of buffer to image planes in pFrameRGB
avpicture_fill((AVPicture *)pFrameRGB, buffer, PIX_FMT_RGB24,
pCodecCtx->width, pCodecCtx->height);
Cleaning up
OK, we've read and processed our video, now all that's left for us to do isclean up after ourselves:
// Free the RGB image
delete [] buffer;
av_free(pFrameRGB);
// Free the YUV frame
av_free(pFrame);
// Close the codec
avcodec_close(pCodecCtx);
// Close the video file
av_close_input_file(pFormatCtx);
Done!
Sample Code
A sample app that wraps all of this code up in compilable form is here. If you have any additional comments, please contact me at boehme@inb.uni-luebeckREMOVETHIS.de. Standard disclaimer: I assume no liability for the correct functioning of the codeand techniques presented in this article.
- Using libavformat and libavcodec
- Using libavformat and libavcodec
- Using libavformat and libavcodec
- Using libavformat and libavcodec
- Using libavformat and libavcodec(1)
- Using libavformat and libavcodec(2)
- Using libavformat and libavcodec(3)
- 【zz】Using libavformat and libavcodec
- 【转载】How to use libavformat and libavcodec
- libavformat/libavcodec学习
- libavformat/libavcodec学习
- libavformat/libavcodec 学习
- libavformat/libavcodec学习
- libavformat/libavcodec学习(mplayer)
- libavformat/libavcodec 学习
- libavformat/libavcodec学习(转)
- libavformat/libavcodec学习
- libavformat/libavcodec 学习
- 调试freescale CSI sa7113 camera的心得
- 通过崩溃地址找错误行数之Delphi版
- 书上实现连接两个字符串的代码,分享下!
- 数据挖掘笔记(三):数据模型(1)
- asp.net用户登录代码
- Using libavformat and libavcodec
- 从头学CodeIgniter和Doctrine 用户注册【翻译】下(表单辅助函数-表单验证-测试表单)流程详解
- Linux环境进程间通信---共享内存
- $this->db->query($sql) 在一些php框架中,这种方式是怎么实现
- Android入门前言(一)之------Android应用开发入门五问
- $this->form_validation->run();
- php处理中文标点
- Get和Post的区别
- Oracle 重建所有表的索引的存储过程 (重建索引)