Red5的丢包处理

转自：http://olylakers.iteye.com/blog/1020390

在多媒体应用中，针对Client端的带宽情况，Server端对Video Data要进行不同的处理，当Client带宽比较差时，需要在Server端对不同的视频帧进行是丢弃处理，如在低带宽条件下，对于非关键帧，为了更好的用户体验性和播放的平滑性，可对其进行丢弃，而关键帧，则一般不给予丢弃。下面结合代码，详细分析下Red5是如何根据Client端的带宽进行丢包处理的。

    Red5的Video Data丢包处理的代码集中在其PlayEngine 的 PushMessage方法，代码如下：

if (body instanceof VideoData) {IVideoStreamCodec videoCodec = null;if (msgIn instanceof IBroadcastScope) {IBroadcastStream stream = (IBroadcastStream) ((IBroadcastScope) msgIn).getAttribute(IBroadcastScope.STREAM_ATTRIBUTE);if (stream != null && stream.getCodecInfo() != null) {videoCodec = stream.getCodecInfo().getVideoCodec();}}//dont try to drop frames if video codec is null - related to SN-77if (videoCodec != null && videoCodec.canDropFrames()) {if (playlistSubscriberStream.state == State.PAUSED) {// The subscriber paused the videolog.debug("Dropping packet because we are paused");videoFrameDropper.dropPacket(rtmpMessage);return;}// Only check for frame dropping if the codec supports itlong pendingVideos = pendingVideoMessages();if (!videoFrameDropper.canSendPacket(rtmpMessage, pendingVideos)) {// Drop frame as it depends on other frames that were dropped before.log.debug("Dropping packet because frame dropper says we cant send it");return;}if (!receiveVideo) {// The client disabled video or the app doesn't have enough bandwidth// allowed for this stream.log.debug("Dropping packet because we cant receive video or token acquire failed");videoFrameDropper.dropPacket(rtmpMessage);return;}// increment the number of times we had pending video frames sequentiallyif (pendingVideos > 1) {numSequentialPendingVideoFrames++;} else {// reset number of sequential pending frames if 1 or 0 are pending.numSequentialPendingVideoFrames = 0;}if (pendingVideos > maxPendingVideoFramesThreshold|| numSequentialPendingVideoFrames > maxSequentialPendingVideoFrames) {log.debug("Pending: {} Threshold: {} Sequential: {}", new Object[] { pendingVideos,maxPendingVideoFramesThreshold, numSequentialPendingVideoFrames });// We drop because the client has insufficient bandwidth.long now = System.currentTimeMillis();if (bufferCheckInterval > 0 && now >= nextCheckBufferUnderrun) {// Notify client about frame dropping (keyframe)sendInsufficientBandwidthStatus(currentItem);nextCheckBufferUnderrun = now + bufferCheckInterval;}videoFrameDropper.dropPacket(rtmpMessage);return;}videoFrameDropper.sendPacket(rtmpMessage);}}

   首先Red5根据Stream的Codec来决定是不是需要启用丢包处理。Red5共定义了三种Codec，分别为ScreenVideo，SorensonVideo和AVCVideo，其中ScreenVideo是用于屏幕截取，Red5的缺省实现是，不可以采取丢包处理，而SorensonVideo和AVCVideo分别用于手机和视频播放，Red5的缺省实现是可以采取丢包处理。如果我们需要在采用ScreenVideo的时候也启用丢包处理，则需要修改ScreenVideo的代码，如下所示：

/** {@inheritDoc} */    public boolean canDropFrames() {return false;}

    在Red5的代码中，之所有在此处直接返回false，不支持丢包，主要是因为ScreenVideo这种Codec的Video data的Decode依赖于前后帧，如果丢弃了一帧，很可能接下来的好几帧数据都不能正确Decode。当然如果网络非常差的情况下，还是通过修改ScreenVideo的canDropFrames（），使其返回true，则可在低带宽条件下启动丢包处理，只不过这样在播放端就会出现连续的花屏现象。

    在判断Codec是否支持Drop frames后，Red5主要根据Mina通道里的pendingVideos的数量来决定是否丢弃Video data。由上面的代码可以得知，pendingVideos 是通过方法pendingVideoMessages（）获取的。pendingVideoMessages的返回的是RTMPConnection里面的pendingVideos的值（当往Mina的通道里写入一个），在Red5的代码中，如果pendingVideos超过最大允许的等待帧数，则启动丢包处理：

// increment the number of times we had pending video frames sequentiallyif (pendingVideos > 1) {numSequentialPendingVideoFrames++;} else {// reset number of sequential pending frames if 1 or 0 are pending.numSequentialPendingVideoFrames = 0;}if (pendingVideos > maxPendingVideoFramesThreshold|| numSequentialPendingVideoFrames > maxSequentialPendingVideoFrames) {log.debug("Pending: {} Threshold: {} Sequential: {}", new Object[] { pendingVideos,maxPendingVideoFramesThreshold, numSequentialPendingVideoFrames });// We drop because the client has insufficient bandwidth.long now = System.currentTimeMillis();if (bufferCheckInterval > 0 && now >= nextCheckBufferUnderrun) {// Notify client about frame dropping (keyframe)sendInsufficientBandwidthStatus(currentItem);nextCheckBufferUnderrun = now + bufferCheckInterval;}videoFrameDropper.dropPacket(rtmpMessage);return;}

     这里之所以用pendingVideos和numSequentialPendingVideoFrames同时作为是否启用丢包的判断条件，主要是为了保证统计的实时性和准确性。

  上面分析了Red5如何判断是否应该丢包，及采取什么样的丢包策略，因此在实际应用的时候可以根据具体的需求，对上述流程进行调节和改动，以适用于不同的应用场景。