WEBRTC视频网络传输到解码输出

一.  视频传输到解码

VideoCodingModuleImpl类其实并没有使用

 

函数调用次顺:

VideoReceiveStream::DeliverRtp->RtpStreamReceiver::DeliverRtp->RtpReceiverImpl::IncomingRtpPacket->RTPReceiverVideo::ParseRtpPacket->

RtpStreamReceiver::OnReceivedPayloadData->VideoReceiver::IncomingPacket[进入视频编解码模块]

 

Module/video_coding/video_coding_impl.h

VideoReceiver

1.       int32_t RegisterReceiveCallback(VCMReceiveCallback*receiveCallback);

注册解码后的视频帧回调对象，通过其虚函数FrameToRender回调，在VideoStreamDecoder构造函数，注册回调对象 video_receiver_->RegisterReceiveCallback(this);

2.       Video/video_stream_decoder.h

使用者对象

VideoStreamDecoder: publicVCMReceiveCallback

1.       int32_t FrameToRender(VideoFrame&video_frame) override;  // NOLINT

 

视频解码线程

VideoReceiveStream::Start->

VideoReceiveStream::DecodeThreadFunction->VideoReceiveStream::Decode->VideoReceiver::Decode(uint16_t maxWaitTimeMs)->

VideoReceiver::Decode(const  VCMEncodedFrame&     frame)->VideoReceiveStream::Encoded[解复用完整帧通过VideoReceiveStream构造函数中VideoReceiver的构造传入VideoReceiveStream对象作为回调]->VCMGenericDecoder::Decode(constVCMEncodedFrame& frame, int64_t nowMs)->

[VCMCodecDataBase::GetDecoder->VCMGenericDecoder::RegisterDecodeCompleteCallback->

VP9DecoderImpl::RegisterDecodeCompleteCallback注册解码后数据回调对]->

VP9DecoderImpl::Decode(const EncodedImage& input_image,

                           bool missing_frames,

                           constRTPFragmentationHeader* fragmentation,

                           constCodecSpecificInfo* codec_specific_info,

                           int64_t/*render_time_ms*/) ->DecodedImageCallback::Decoded(父)->VCMDecodedFrameCallback::Decoded(VideoFrame&decodedImage) (子)->VCMDecodedFrameCallback::Decoded(VideoFrame&decodedImage,int64_t decode_time_ms)->

VideoStreamDecoder::FrameToRender(VideoFrame&video_frame)->

[接收解码对象通过VideoStreamDecoder::VideoStreamDecoder ->VideoReceiver::RegisterReceiveCallback(this)–>VCMDecodedFrameCallback::SetUserReceiveCallback]

IncomingVideoStream::OnFrame[将帧放入队列VideoRenderFrames]->

 

视频显示|录像线程

VideoReceiveStream::Start->IncomingVideoStream::Start->

IncomingVideoStream::IncomingVideoStreamThreadFun[从VideoRenderFrames队列获取帧]->IncomingVideoStream::DeliverFrame->VideoReceiveStream::OnFrame[该回调函数由VideoReceiveStream::VideoReceiveStream中注册incoming_video_stream_.SetExternalCallback(this)]->WebRtcVideoReceiveStream::OnFrame(constwebrtc::VideoFrame& frame)[ WebRtcVideoReceiveStream::WebRtcVideoReceiveStream构造函数赋值config_.renderer = this]

[设置广播接收者PeerConnection::SetRemoteDescription -> PeerConnection::OnRemoteTrackSeen-> PeerConnection::CreateVideoReceiver[new              VideoRtpReceiver]->VideoRtpReceiver::VideoRtpReceiver[provider_->SetVideoPlayout(ssrc_,true,&broadcaster_)]->WebRtcSession::SetVideoPlayout-> VideoChannel::SetSink->

WebRtcVideoChannel2::SetSink-> WebRtcVideoReceiveStream::SetSink(sink_)]

 

总结:

1. 从网络接收线程输入到显示线程输出(中间解码线程)

VideoReceiveStream::DeliverRtp[已复用的视频RTP]->

VideoReceiveStream::OnFrame[已解码的视频帧]

2. 从网络接收RTP到解复用完整帧

VideoReceiveStream::DeliverRtp[已复用的视频RTP]->

int32_t VideoReceiveStream::Encoded(

    constEncodedImage& encoded_image,

    constCodecSpecificInfo* codec_specific_info,

const RTPFragmentationHeader*fragmentation)

 

 

RtpStreamReceiver作为Video_Coding模块和RTP/RTCP 模块的中介者

#include"webrtc/video/rtp_stream_receiver.h"

RtpStreamReceiver

1.     将RTP/RTCP模块解复用的RTP包送入Video_Coding中的VideoReceiver::IncomingPacket

2. 回调RTP/RTCP中的RequestKeyFrame、SliceLossIndicationRequest、ResendPackets[关键字请求、重传帧请求]

 

RtpReceiver

#include"webrtc/modules/rtp_rtcp/source/rtp_receiver_impl.h"

 

二.  核心类关系