X264码率控制流程分析

码率控制的理论知识：码率控制的目的和意义：图像通信中码率控制的目的：通过调节编码参数，控制单位时间内的编码视频流的数据量，以使产生的比特流符合各种应用的需求。视频压缩的效率和视频内容有很大的关系，对于变化多样的画面，视频编码的输出的码流变化较大，在信道环境不好的时候就容易导致解码端显示的质量的不稳定。率失真理论：由于传输带宽和存储空间的限制，视频应用对压缩比有较高的要求。而无损编码较低的压缩比无法满足视频在实际应用中的需求。但如果给视频引入一定程度的失真，通常可以获得较高的压缩比。率失真理论对有损压缩编码下的失真和编码性能之间的关系的描述，为码率控制的研究提供了坚实的理论依据。率失真理论主旨是描述编码失真度和编码数据速率的关系。该理论建立在图像是连续的基础上的，在有限数据速率下，由于存在量化误差，必然存在失真。当使用有损编码方法时，重建图像g(x,y)和原始图像f(x,y)之间存在差异，失真度D的函数形式在理论上是可以根据需要自由选取的，在图像编码中，D常用均方差形式表示的，典型的率失真曲线。R(D)为D的凸减函数。对于怎么选择哪个函数的率失真效果更好，则是比较哪个函数的率失真函数更为接近典型的率失真函数的曲线。x264码率控制方法：采用的码率控制算法并没有采用拉格朗日代价函数来控制编码，而是使用一种更简单的方法，即利用半精度帧的SATD(sum of absolute transformed difference)作为模式选择的依据。SATD即将残差经哈德曼变换的4×4块的预测残差绝对值总和，可以将其看作简单的时频变换，其值在一定程度上可以反映生成码流的大小。SATD是将残差经哈达曼变换4*4块的预测残差绝对值总和。自适应宏块层码率控制策略：X264的宏块没有任何码率控制的机制，其在帧层得到一个QP后，属于该帧的所有宏块都用着统一的QP进行量化。码率控制性能测度：1、比特率误差|ABR-TBR|/TBR ，越小越好。2、编码器性能。3、缓冲区满度与TBL的匹配程度。4、跳帧数。5、PSNR波动越小越好。x264中码率控制的流程(对于重点函数在下面有注释)：1.在进行编码时，Encode--->x264_encoder_open(主要是进行参数的修订设置，进行初始化)---->x264_ratecontrol_new2.encode--->Encode_frame--->x264_encoder_encode--->x264_ratecontrol_slice_type3.encode--->Encode_frame--->x264_encoder_encode--->x264_ratecontrol_start**************4.encode--->Encode_frame--->x264_encoder_encode--->x264_ratecontrol_qp5.encode--->Encode_frame--->x264_encoder_encode--->x264_slices_write--->x264_slice_write--->x264_ratecontrol_mb********************6.encode--->Encode_frame--->x264_encoder_encode--->x264_ratecontrol_end(在编完一帧过后)7.在编完过后，encode--->x264_encoder_close---->ratecontrol summary/x264_ratecontrol_delete函数注释：在编码中所用的编码方式：#define X264_RC_CQP 0#define X264_RC_CRF 1#define X264_RC_ABR 21.x264_ratecontrol_new( x264_t *h ){ // 获取RC方式，FPS，bitrate,rc->buffer_rate,rc->buffer_size// 在码率控制的时候会出现2pass，参数的初始化rc = h->rc;rc->b_abr = h->param.rc.i_rc_method != X264_RC_CQP && !h->param.rc.b_stat_read;rc->b_2pass = h->param.rc.i_rc_method == X264_RC_ABR && h->param.rc.b_stat_read;..........if( h->param.rc.b_mb_tree )//这里设置mb_tree{h->param.rc.f_pb_factor = 1;rc->qcompress = 1;}elserc->qcompress = h->param.rc.f_qcompress;..............rc->ip_offset = 6.0 * log(h->param.rc.f_ip_factor) / log(2.0);rc->pb_offset = 6.0 * log(h->param.rc.f_pb_factor) / log(2.0);rc->qp_constant[SLICE_TYPE_P] = h->param.rc.i_qp_constant;rc->qp_constant[SLICE_TYPE_I] = x264_clip3( h->param.rc.i_qp_constant - rc->ip_offset + 0.5, 0, 51 );rc->qp_constant[SLICE_TYPE_B] = x264_clip3( h->param.rc.i_qp_constant + rc->pb_offset + 0.5, 0, 51 );}2.int x264_ratecontrol_slice_type( x264_t *h, int frame_num ){//根据不同类型来获取不同的qp_constanth->param.rc.i_qp_constant = (h->stat.i_frame_count[SLICE_TYPE_P] == 0) ? 24: 1 + h->stat.f_frame_qp[SLICE_TYPE_P] / h->stat.i_frame_count[SLICE_TYPE_P];rc->qp_constant[SLICE_TYPE_P] = x264_clip3( h->param.rc.i_qp_constant, 0, 51 );rc->qp_constant[SLICE_TYPE_I] = x264_clip3( (int)( qscale2qp( qp2qscale( h->param.rc.i_qp_constant ) / fabs( h->param.rc.f_ip_factor )) + 0.5 ), 0, 51 );rc->qp_constant[SLICE_TYPE_B] = x264_clip3( (int)( qscale2qp( qp2qscale( h->param.rc.i_qp_constant ) * fabs( h->param.rc.f_pb_factor )) + 0.5 ), 0, 51 );}3.x264_ratecontrol_start( h, h->fenc->i_qpplus1, overhead*8 );这个函数的目的就是在一帧的编码前就选择QP/* Init the rate control *//* FIXME: Include slice header bit cost. */x264_ratecontrol_start( h, h->fenc->i_qpplus1, overhead*8 );对x264_ratecontrol_start函数的解析如下：x264_zone_t *zone = get_zone( h, h->fenc->i_frame );//找到h->fenc->i_frame所在的zone....................//由各种不同的slice类型，vbv等等参数获取的q值if( i_force_qp ){q = i_force_qp - 1;//}else if( rc->b_abr ){q = qscale2qp( rate_estimate_qscale( h ) );//下面有注解}else if( rc->b_2pass ){rce->new_qscale = rate_estimate_qscale( h );q = qscale2qp( rce->new_qscale );}else /* CQP */{if( h->sh.i_type == SLICE_TYPE_B && h->fdec->b_kept_as_ref )q = ( rc->qp_constant[ SLICE_TYPE_B ] + rc->qp_constant[ SLICE_TYPE_P ] ) / 2;elseq = rc->qp_constant[ h->sh.i_type ];if( zone ){if( zone->b_force_qp )q += zone->i_qp - rc->qp_constant[SLICE_TYPE_P];elseq -= 6*log(zone->f_bitrate_factor)/log(2);}///////////////////////////////////////////////////////////////////* Terminology:* qp = h.264's quantizer* qscale = linearized quantizer = Lagrange multiplier*/static inline double qp2qscale(double qp){return 0.85 * pow(2.0, ( qp - 12.0 ) / 6.0);}static inline double qscale2qp(double qscale){return 12.0 + 6.0 * log(qscale/0.85) / log(2.0);}////////////////////////////////////////////////////////////////////////rate_estimate_qscale( h )// update qscale for 1 frame based on actual bits used so far(即根据所需BIT来计算qscale)static float rate_estimate_qscale( x264_t *h ){//这里是分别针对B，P帧分别进行,因为I帧是已经设定if( pict_type == SLICE_TYPE_B ){//这里B帧的q的大小是由参考帧求的..........................................// 由predict_size获得帧的sizercc->frame_size_planned = predict_size( rcc->pred_b_from_p, q, h->fref1[h->i_ref1-1]->i_satd );x264_ratecontrol_set_estimated_size(h, rcc->frame_size_planned);//////////////////////////void x264_ratecontrol_set_estimated_size( x264_t *h, int bits ){x264_pthread_mutex_lock( &h->fenc->mutex );h->rc->frame_size_estimated = bits;///***********x264_pthread_mutex_unlock( &h->fenc->mutex );}////////////////////////////}////P帧的q值获取else{//这里的分有1pass和2pass的选择...................选择predicted_bits，求出diffdiff = predicted_bits - (int64_t)rce.expected_bits;q /= x264_clip3f((double)(abr_buffer - diff) / abr_buffer, .5, 2);}}4.int x264_ratecontrol_qp( x264_t *h ){return h->rc->qpm;}5.void x264_ratecontrol_mb( x264_t *h, int bits ){//这个函数主要是针对一行的bitsif( h->sh.i_type == SLICE_TYPE_B ){//由参考的图像求对应的行的qp，有已编码的bits获得此行的bits和qpint avg_qp = X264_MIN(h->fref0[0]->i_row_qp[y+1], h->fref1[0]->i_row_qp[y+1])+ rc->pb_offset * ((h->fenc->i_type == X264_TYPE_BREF) ? 0.5 : 1);rc->qpm = X264_MIN(X264_MAX( rc->qp, avg_qp), 51); //avg_qp could go higher than 51 due to pb_offseti_estimated = row_bits_so_far(h, y); //FIXME: compute full estimated sizeif (i_estimated > h->rc->frame_size_planned)x264_ratecontrol_set_estimated_size(h, i_estimated);}//I, p，这里还要参考缓冲区的状态else{//对I,P帧在考虑VBV的情况下求的bits和qp}}6./* After encoding one frame, save stats and update ratecontrol state */int x264_ratecontrol_end( x264_t *h, int bits ){///统计ipb类型的Mb的个数，并计算平均QPh->fdec->f_qp_avg_rc = rc->qpa_rc /= h->mb.i_mb_count;h->fdec->f_qp_avg_aq = rc->qpa_aq /= h->mb.i_mb_count;}7.void x264_ratecontrol_summary( x264_t *h ){x264_ratecontrol_t *rc = h->rc;//ABRif( rc->b_abr && h->param.rc.i_rc_method == X264_RC_ABR && rc->cbr_decay > .9999 ){double base_cplx = h->mb.i_mb_count * (h->param.i_bframe ? 120 : 80);double mbtree_offset = h->param.rc.b_mb_tree ? (1.0-h->param.rc.f_qcompress)*13.5 : 0;x264_log( h, X264_LOG_INFO, "final ratefactor: %.2f/n",qscale2qp( pow( base_cplx, 1 - rc->qcompress )* rc->cplxr_sum / rc->wanted_bits_window ) - mbtree_offset );}}/////////////////////////////void x264_ratecontrol_delete( x264_t *h )///////释放RC开辟的空间通过以上的流程总结x264码率控制的过程基本是有以下三步：1.对码率控制的相关变量进行初始化，如，I,P,B的初始QP值，RC的方式，VBV的初始状态等等；2.获取编码帧的复杂度，x264用SATD表示，对于采用的不同参数的码率控制的方式，由前面已编码的Bits,复杂度，目标比特的设置等一些条件来获取编码当前帧的qp值。3.在编码过程中，由获得qp值得到预测的bits;实验部分：1.简单参数设置：参数设置：--frames 10 --qp 26 -o test.264 F:/......./akiyo_qcif.yuv 176x144其他的参数采用默认设置（在默认设置时采用的码率控制模型是X264_RC_CQP），所得的实验结果：x264 [info]: 176x144 @ 25.00 fpsx264 [info]: using cpu capabilities: MMX2 SSE2 Cache64 Slow_mod4_stackx264 [info]: profile High, level 1.1x264 [info]: frame I:1 Avg QP:23.00 size: 4189x264 [info]: frame P:3 Avg QP:26.00 size: 62x264 [info]: frame B:6 Avg QP:28.00 size: 38x264 [info]: consecutive B-frames: 11.1% 0.0% 0.0% 88.9%x264 [info]: mb I I16..4: 3.0% 41.4% 55.6%x264 [info]: mb P I16..4: 0.0% 0.0% 0.0% P16..4: 3.0% 1.3% 1.7% 0.0% 0.0% skip:93.9%x264 [info]: mb B I16..4: 0.0% 0.0% 0.0% B16..8: 4.4% 0.2% 0.3% direct:0.7% skip:94.4% L0:56.0% L1:40.5% BI: 3.4%x264 [info]: 8x8 transform intra:41.4% inter:25.9%x264 [info]: coded y,uvDC,uvAC intra: 83.6% 81.8% 68.7% inter: 1.1% 0.1% 0.0%x264 [info]: i16 v,h,dc,p: 100% 0% 0% 0%x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 25% 30% 15% 4% 4% 4% 7% 5% 6%x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 35% 18% 8% 5% 5% 10% 5% 8% 6%x264 [info]: ref P L0: 88.9% 0.0% 11.1%x264 [info]: kb/s:92.08encoded 10 frames, 24.33 fps, 92.08 kb/s2.改变码率控制的模型：--frames 10 --qp 26 --crf 2 -o test.264 F:/....../akiyo_qcif.yuv 176x144x264 [info]: 176x144 @ 25.00 fpsx264 [info]: using cpu capabilities: MMX2 SSE2 Cache64 Slow_mod4_stackx264 [info]: profile High, level 1.1x264 [info]: frame I:1 Avg QP:10.00 size: 10246x264 [info]: frame P:3 Avg QP:11.48 size: 847x264 [info]: frame B:6 Avg QP:12.10 size: 172x264 [info]: consecutive B-frames: 11.1% 0.0% 0.0% 88.9%x264 [info]: mb I I16..4: 1.0% 44.4% 54.5%x264 [info]: mb P I16..4: 0.0% 0.0% 0.0% P16..4: 30.0% 3.0% 4.7% 0.0% 0.0% skip:62.3%x264 [info]: mb B I16..4: 0.0% 0.0% 0.0% B16..8: 6.9% 1.0% 1.2% direct:4.0% skip:86.9% L0:34.7% L1:55.6% BI: 9.7%x264 [info]: 8x8 transform intra:44.4% inter:34.8%x264 [info]: coded y,uvDC,uvAC intra: 100.0% 99.0% 94.9% inter: 11.6% 7.6% 4.9%x264 [info]: i16 v,h,dc,p: 100% 0% 0% 0%x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 30% 33% 16% 3% 2% 3% 4% 4% 5%x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 28% 12% 11% 5% 7% 11% 6% 12% 8%x264 [info]: ref P L0: 95.8% 1.6% 2.7%x264 [info]: ref B L0: 96.3% 3.7%x264 [info]: kb/s:276.36encoded 10 frames, 14.27 fps, 276.36 kb/s针对1，2两个实验，所采用的RC模型不一样，1：X264_RC_CQP，2：X264_RC_CRF，其他参数的设置一样，从IPB的平均QP，编码Bits可以看出和对于实际的应用来说，CRF的效果不如CQP。3.--frames 10 --qp 26 --pass 1 -o test.264 F:/...../bin/akiyo_qcif.yuv 176x144x264 [info]: 176x144 @ 25.00 fpsx264 [info]: using cpu capabilities: MMX2 SSE2 Cache64 Slow_mod4_stackx264 [info]: profile Main, level 1.1x264 [info]: frame I:1 Avg QP:23.00 size: 4068x264 [info]: frame P:3 Avg QP:26.00 size: 59x264 [info]: frame B:6 Avg QP:28.00 size: 31x264 [info]: consecutive B-frames: 11.1% 0.0% 0.0% 88.9%x264 [info]: mb I I16..4: 15.2% 0.0% 84.8%x264 [info]: mb P I16..4: 0.0% 0.0% 0.0% P16..4: 7.1% 0.0% 0.0% 0.0% 0.0% skip:92.9%x264 [info]: mb B I16..4: 0.0% 0.0% 0.0% B16..8: 1.2% 0.0% 0.0% direct:1.5% skip:97.3% L0:100.0% L1: 0.0% BI: 0.0%x264 [info]: coded y,uvDC,uvAC intra: 87.4% 77.8% 68.7% inter: 1.1% 0.1% 0.0%x264 [info]: i16 v,h,dc,p: 47% 20% 27% 7%x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 29% 27% 10% 5% 4% 8% 5% 6% 5%x264 [info]: kb/s:88.58encoded 10 frames, 52.63 fps, 88.58 kb/s4.--frames 10 --qp 26 --pass 2 -o test.264 F:/...../bin/akiyo_qcif.yuv 176x144x264 [info]: 176x144 @ 25.00 fpsx264 [info]: using cpu capabilities: MMX2 SSE2 Cache64 Slow_mod4_stackx264 [info]: profile High, level 1.1x264 [info]: frame I:1 Avg QP:23.00 size: 4189x264 [info]: frame P:3 Avg QP:26.00 size: 62x264 [info]: frame B:6 Avg QP:28.00 size: 38x264 [info]: consecutive B-frames: 11.1% 0.0% 0.0% 88.9%x264 [info]: mb I I16..4: 3.0% 41.4% 55.6%x264 [info]: mb P I16..4: 0.0% 0.0% 0.0% P16..4: 3.0% 1.3% 1.7% 0.0% 0.0% skip:93.9%x264 [info]: mb B I16..4: 0.0% 0.0% 0.0% B16..8: 4.4% 0.2% 0.3% direct:0.7% skip:94.4% L0:56.0% L1:40.5% BI: 3.4%x264 [info]: 8x8 transform intra:41.4% inter:25.9%x264 [info]: coded y,uvDC,uvAC intra: 83.6% 81.8% 68.7% inter: 1.1% 0.1% 0.0%x264 [info]: i16 v,h,dc,p: 100% 0% 0% 0%x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 25% 30% 15% 4% 4% 4% 7% 5% 6%x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 35% 18% 8% 5% 5% 10% 5% 8% 6%x264 [info]: ref P L0: 88.9% 0.0% 11.1%x264 [info]: kb/s:92.08encoded 10 frames, 27.70 fps, 92.08 kb/s5.--frames 10 --qp 26 --pass 3 -o test.264 F:/...../bin/akiyo_qcif.yuv 176x144x264 [info]: 176x144 @ 25.00 fpsx264 [info]: using cpu capabilities: MMX2 SSE2 Cache64 Slow_mod4_stackx264 [info]: profile High, level 1.1x264 [info]: frame I:1 Avg QP:23.00 size: 4189x264 [info]: frame P:3 Avg QP:26.00 size: 62x264 [info]: frame B:6 Avg QP:28.00 size: 38x264 [info]: consecutive B-frames: 11.1% 0.0% 0.0% 88.9%x264 [info]: mb I I16..4: 3.0% 41.4% 55.6%x264 [info]: mb P I16..4: 0.0% 0.0% 0.0% P16..4: 3.0% 1.3% 1.7% 0.0% 0.0% skip:93.9%x264 [info]: mb B I16..4: 0.0% 0.0% 0.0% B16..8: 4.4% 0.2% 0.3% direct:0.7% skip:94.4% L0:56.0% L1:40.5% BI: 3.4%x264 [info]: 8x8 transform intra:41.4% inter:25.9%x264 [info]: coded y,uvDC,uvAC intra: 83.6% 81.8% 68.7% inter: 1.1% 0.1% 0.0%x264 [info]: i16 v,h,dc,p: 100% 0% 0% 0%x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 25% 30% 15% 4% 4% 4% 7% 5% 6%x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 35% 18% 8% 5% 5% 10% 5% 8% 6%x264 [info]: ref P L0: 88.9% 0.0% 11.1%x264 [info]: kb/s:92.08encoded 10 frames, 25.64 fps, 92.08 kb/s对于3，4，5是关于Pass的实验比较：多次压缩码率控制1：第一次压缩，创建统计文件2：按建立的统计文件压缩并输出，不覆盖统计文件，3：按建立的统计文件压缩，优化统计文件在想得到建好的效果的时候采用pass 2就可以了。6.--frames 10 --qp 26 --bitrate 64 -o test.264 F:/...../bin/akiyo_qcif.yuv 176x144x264 [info]: 176x144 @ 25.00 fpsx264 [info]: using cpu capabilities: MMX2 SSE2 Cache64 Slow_mod4_stackx264 [info]: profile High, level 1.1x264 [info]: frame I:1 Avg QP:38.31 size: 1461x264 [info]: frame P:3 Avg QP:42.00 size: 18x264 [info]: frame B:6 Avg QP:45.00 size: 14x264 [info]: consecutive B-frames: 11.1% 0.0% 0.0% 88.9%x264 [info]: mb I I16..4: 15.2% 68.7% 16.2%x264 [info]: mb P I16..4: 0.0% 0.0% 0.0% P16..4: 2.0% 0.0% 0.3% 0.0% 0.0% skip:97.6%x264 [info]: mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.2% 0.0% 0.0% direct:0.0% skip:99.8% L0: 0.0% L1:100.0% BI: 0.0%x264 [info]: final ratefactor: 31.50x264 [info]: 8x8 transform intra:68.7%x264 [info]: coded y,uvDC,uvAC intra: 48.0% 61.6% 32.3% inter: 0.0% 0.0% 0.0%x264 [info]: i16 v,h,dc,p: 33% 47% 7% 13%x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 35% 17% 20% 3% 4% 7% 3% 7% 5%x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 50% 14% 8% 5% 5% 5% 3% 6% 4%x264 [info]: kb/s:31.94encoded 10 frames, 31.25 fps, 31.94 kb/s7.--frames 250 --qp 26 --bitrate 64 -o test.264 F:/...../bin/akiyo_qcif.yuv 176x144x264 [info]: 176x144 @ 25.00 fpsx264 [info]: using cpu capabilities: MMX2 SSE2 Cache64 Slow_mod4_stackx264 [info]: profile High, level 1.1x264 [info]: frame I:1 Avg QP:34.62 size: 1779x264 [info]: frame P:92 Avg QP:19.81 size: 569x264 [info]: frame B:157 Avg QP:26.76 size: 53x264 [info]: consecutive B-frames: 15.7% 0.0% 2.4% 81.9%x264 [info]: mb I I16..4: 14.1% 61.6% 24.2%x264 [info]: mb P I16..4: 0.0% 0.0% 0.0% P16..4: 25.8% 9.4% 9.9% 0.0% 0.0% skip:54.8%x264 [info]: mb B I16..4: 0.0% 0.0% 0.0% B16..8: 13.9% 0.7% 1.4% direct:1.1% skip:83.0% L0:16.6% L1:72.1% BI:11.3%x264 [info]: final ratefactor: 18.97x264 [info]: 8x8 transform intra:61.5% inter:40.4%x264 [info]: coded y,uvDC,uvAC intra: 61.3% 65.4% 34.6% inter: 8.6% 6.8% 2.8%x264 [info]: i16 v,h,dc,p: 57% 43% 0% 0%x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 32% 22% 18% 4% 2% 7% 3% 7% 4%x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 45% 10% 10% 5% 6% 7% 6% 6% 5%x264 [info]: ref P L0: 87.6% 7.6% 4.8%x264 [info]: ref B L0: 95.0% 5.0%x264 [info]: kb/s:49.92encoded 250 frames, 16.74 fps, 49.92 kb/s6，7是针对不同的编码帧数来进行比较的，在编码帧数越多，带宽利用的效果就越好。6，7是在设置了目标码率64kp/s时，采用的是ABR的RC模型，在设置了目标码率能够根据目标码率的大小改变QP大小，能够控制码率。