FLV文件中VideoPacket的详解

FLV文件中VideoPacket的详解

众所周知，FLV文件体主要有三种Tag组成：AUDIODATA、VIDEODATA和SCRIPTDATA，其中SCRIPTDATA中包含了MetaData，通过解析这些元数据，可以得到解码器的初始化信息，如音频解码元数据audiocodecid、audiodatarate、audiodelay、audiosamplerate、audiosamplesize、stereo，视频解码元数据videocodecid、framerate、height、width、videodatarate等信息。这些信息里面有的对于初始化解码器非常有用，例如解码器必须知道codecid，才能进行初始化从操作，视频解码器初始化时有的需要知道图像宽高信息，才能对解码重建Buffer进行初始化。

某些FLV文件，在SCRIPTDATA的MetaData中并未给出相应信息，这样就需要对AUDIODATA、VIDEODATA进行解析。Adobe Flash Video File Format Specification中只是简单交代了AVCVIDEOPACKET的格式。对其他的codecID对应的VideoPacket并未详细介绍。本文专门针对视频解码初始化过程中需要的width和height信息的深入解析为例，详细介绍一些VideoPacket。因为FLV的VideoPacket的格式和SWF文件离得VideoPacket格式基本一致，也可以参考SWF File Format Specification。详见http://www.adobe.com/go/swf_file_format。

1、VideoTagBody

IF FrameType == 5

UI8

ELSE (

IF CodecID == 2

H263VIDEOPACKET                            //Sorensen H263

IF CodecID == 3

SCREENVIDEOPACKET              //Screen

IF CodecID == 4

VP6FLVVIDEOPACKET               //On2 Vp6

IF CodecID == 5

VP6FLVALPHAVIDEOPACKET  //On2 VP6 alpha

IF CodecID == 6

SCREENV2VIDEOPACKET                  //Screen V2

IF CodecID == 7

AVCVIDEOPACKET                     //AVC

)

2、H263VIDEOPACKET

这里的H263是Sorenson H263，对ITU-T的H.263进行了订制，所以在picture层和MB层的header的格式发生了改动，且GOB层被删除。其具体格式为：

表格 1 H263VIDEOPACKET

Field

Type

Comment

PictureStartCode

UB[17]

Similar to H.263 5.1.1
0000 0000 0000 0000 1

Version

UB[5]

Video format version
Flash Player 6 supports 0 and 1

TemporalReference

UB[8]

See H.263 5.1.2

PictureSize

UB[3]

000: custom, 1 byte
001: custom, 2 bytes
010: CIF (352x288)
011: QCIF (176x144)
100: SQCIF (128x96)
101: 320x240
110: 160x120
111: reserved

CustomWidth

If PictureSize = 000, UB[8]
If PictureSize = 001, UB[16]
Otherwise absent
Note: UB[16] is not the same
as UI16; there is no byte
swapping.

Width in pixels

CustomHeight

If PictureSize = 000, UB[8]
If PictureSize = 001, UB[16]
Otherwise absent
Note: UB[16] is not the same
as UI16; there is no byte
swapping.

Height in pixels

PictureType

UB[2]

00: intra frame
01: inter frame
10: disposable inter frame
11: reserved

DeblockingFlag

UB[1]

Requests use of deblocking
filter (advisory only, Flash
Player may ignore)

Quantizer

UB[5]

See H.263 5.1.4

ExtraInformationFlag

UB[1]

See H.263 5.1.9

ExtraInformation

If ExtraInformationFlag = 1,
UB[8]
Otherwise absent

See H.263 5.1.10

...

The ExtraInformationFlag-
ExtraInformation sequence
repeats until an
ExtraInformationFlag of 0 is
encountered

Macroblock

MACROBLOCK

See following

PictureStuffing

varies

See H.263 5.1.13

从上表可以得出，不用深入到macroblock层，只需解析头部，就可很简单的得到width和height信息。

 

2、SCREENVIDEOPACKET和SCREENV2VIDEOPACKET

Screen Video是一个简单的无损压缩视频格式，它利用帧间编码传递位图，其pixel_froamt是BGR24，专用于电脑截屏视频传输。像素数据使用了Zlib开放标准进行了压缩。其具体格式为：

表格 2 SCREENVIDEOPACKET

Field

Type

Comment

BlockWidth

UB[4]

Pixel width of each block in the grid. This value is stored as (actualWidth / 16) - 1, so possible block sizes are a multiple of 16 and not more than 256.

ImageWidth

UB[12]

Pixel width of the full image.

BlockHeight

UB[4]

Pixel height of each block in the grid. This value is stored
as (actualHeight / 16) - 1, so possible block sizes are a
multiple of 16 and not more than 256.

ImageHeight

UB[12]

Pixel height of the full image.

ImageBlocks

IMAGEBLOCK[n]

Blocks of image data. See preceding for details of how
to calculate n. Blocks are ordered from bottom left to
top right, in rows.

表格 3 SCREENV2VIDEOPACKET

Field

Type

Comment

BlockWidth

UB[4]

Pixel width of each block in the grid. This value is stored as (actualWidth / 16) - 1, so possible block sizes are a multiple of 16 and not more than 256.

ImageWidth

UB[12]

Pixel width of the full image.

BlockHeight

UB[4]

Pixel height of each block in the grid. This value is stored as (actualHeight / 16) - 1, so possible block sizes are a multiple of 16 and not more than 256.

ImageHeight

UB[12]

Pixel height of the full image.

Reserved

UB[6]

Must be 0

HasIFrameImage

UB[1]

If 1, has IFrameImage

HasPaletteInfo

UB[1]

If 1, has PaletteInfo

PaletteInfo

If HasPaletteInfo,
IMAGEBLOCK

One block of data to describe the palette.

ImageBlocks

IMAGEBLOCKV2[n]

Blocks of image data. See Block format for details of how to calculate n. Blocks are ordered from bottom left to top right in rows and can be a combination of keyblocks and interblocks.

IFrameImage

If HasIFrameImage,
IMAGEBLOCKV2[n]

Blocks of image data representing interblocks that must be combined with the previous keyblocks to produce the image. See Block format for details of how to calculate n. Blocks are ordered from bottom left to top right in rows.

从上表可以得出，不用深入到macroblock层，只需解析头部，就可很简单的得到width和height信息。

3、VP6FLVVIDEOPACKET和VP6FLVALPHAVIDEOPACKET

         VP6FLVVIDEOPACKET和VP6FLVALPHAVIDEOPACKET比较简单，只是简单的将VP6或VP6A的一个视频帧封装到VideoPacket即可。

表格 4 VP6FLVVIDEOPACKET

Field

Type

Comment

HorizontalAdjustment

UB[4]

Number of pixels to subtract from the total width. The resulting width is used on the stage, and the rightmost pixels of the video is cropped.

VerticalAdjustment

UB[4]

Number of pixels to subtract from the total height. The resulting height is used on the stage, and the rightmost pixels of the video is cropped.

Data

UI8[n]

Raw VP6 video stream data

表格 5 VP6FLVALPHAVIDEOPACKET

Field

Type

Comment

HorizontalAdjustment

UB[4]

Number of pixels to subtract from the total width. The resulting width is used on the stage, and the rightmost pixels of the video is cropped.

VerticalAdjustment

UB[4]

Number of pixels to subtract from the total height. The resulting height is used on the stage, and the rightmost pixels of the video is cropped.

OffsetToAlpha

UI[24]

Offset in bytes to the alpha channel video data.

Data

UI8[n]

Raw VP6 video stream data.

AlphaData

UI8[n]

Raw VP6 video stream data representing the alpha channel

这样如果MetaData中没有宽高信息的话，就需要深入到Data层进行header parse，解析得到宽高信息，过程比较麻烦，不过可以参考FFmpeg的avcodec中的vp6_parse_header()函数，亦或是参考VP6或VP6Alpha的codec文档吧，本文暂不做详解。

4、AVCVIDEOPACKET

       FLV将AVC的SPS(序列参数集)和PPS(图像参数集)和其他的配置信息封装到AVCDecoderConfigurationRecord，其具体格式为：

AVCDecoderConfigurationRecord {
         unsigned int(8) ConfigurationVersion = 1;
         unsigned int(8) AVCProfileIndication;
         unsigned int(8) ProfileCompatibility;
         unsigned int(8) AVCLevelIndication;
         bit(6) reserved = ‘111111’b;
         unsigned int(2) LengthSizeMinusOne;
         bit(3) reserved = ‘111’b;
         unsigned int(5) numOfSequenceParameterSets;
         for (i=0; i< numOfSequenceParameterSets;  i++) {
                   unsigned int(16) sequenceParameterSetLength ;
                   bit(8*sequenceParameterSetLength) sequenceParameterSetNALUnit;   //SPS
         }
         unsigned int(8) numOfPictureParameterSets;
         for (i=0; i< numOfPictureParameterSets;  i++) {
                   unsigned int(16) pictureParameterSetLength;
                   bit(8*pictureParameterSetLength) pictureParameterSetNALUnit;              //PPS
         }
}

         图像的宽高信息在SPS中，所以需要得到宽高信息，需要解析H.264的SPS，详见H.264的文档，当然也可以参考FFmpeg的avcodec中的ff_h264_decode_seq_parameter_set()函数。

5、一点点个人想法

流媒体播放中，多媒体音视频包经过网络到达终端，脱去一层层网络打包外衣，而后容器的外衣、再次是codec的header，经过艰辛的历程到达解码器。最终的目的就是节省网络带宽。随着互联网免费服务潮的到来，那些效果不错，但是收费高昂的标准或技术，遇到了很大的挑战。Google主推的VP8，较之ITU-T的H264，虽在编码效果上稍微逊色，但是其最大的杀手锏——免费，抓住了H264专利费高昂的七寸。Google强大的号召力或许真的会使VPx系列的codec最后胜出。一般公司玩技术、牛公司玩标准、超牛公司玩潮流，不得不佩服美帝的技术帝国。何种这些风潮才能Made in China?