⑤NuPlayer播放框架之GenericSource源码分析

[时间：2017-01] [状态：Open]
[关键词：android，nuplayer，开源播放器，播放框架，GenericSource]

0 导读

GenericSource是NuPlayer::Source的一个子类，主要功能是负责本地多媒体文件的读取解析，功能类似FFmpeg的libavformt。
通常GenericSource有以下功能：

• 多媒体文件格式探测
• 多媒体文件解析

本文是我的NuPlayer播放框架的第五篇。

1 GenericSource对外接口及主要成员

NuPlayer::Source抽象类

先从父类开始，接口如下：

// code from ~/frameworks/av/media/libmediaplayerservice/nuplayer/NuPlayerSource.hstruct NuPlayer::Source : public AHandler {    enum Flags {        FLAG_CAN_PAUSE          = 1,        FLAG_CAN_SEEK_BACKWARD  = 2,  // the "10 sec back button"        FLAG_CAN_SEEK_FORWARD   = 4,  // the "10 sec forward button"        FLAG_CAN_SEEK           = 8,  // the "seek bar"        FLAG_DYNAMIC_DURATION   = 16,        FLAG_SECURE             = 32,        FLAG_PROTECTED          = 64,    };    // The provides message is used to notify the player about various events.    Source(const sp<AMessage> &notify): mNotify(notify) {}    virtual void prepareAsync() = 0; // 常规接口，播放、停止、暂停、恢复    virtual void start() = 0;    virtual void stop() {}    virtual void pause() {}    virtual void resume() {}    // Explicitly disconnect the underling data source    virtual void disconnect() {}    // Returns OK iff more data was available, an error or ERROR_END_OF_STREAM if not.    virtual status_t feedMoreTSData() = 0;    // 获取音视频格式    virtual sp<AMessage> getFormat(bool audio);    virtual sp<MetaData> getFormatMeta(bool /* audio */) { return NULL; }    virtual sp<MetaData> getFileFormatMeta() const { return NULL; }    virtual status_t dequeueAccessUnit(bool audio, sp<ABuffer> *accessUnit) = 0;    virtual status_t getDuration(int64_t * /* durationUs */) {...}    // 返回实际解析之后的节目数目及信息，这里将每个节目称为一个Track    virtual size_t getTrackCount() const {return 0;}    virtual sp<AMessage> getTrackInfo(size_t /* trackIndex */) const {...}    virtual ssize_t getSelectedTrack(media_track_type /* type */) const {...}    virtual status_t selectTrack(size_t /* trackIndex */, bool /* select */, int64_t /* timeUs*/) {...}    // seek操作的主要执行函数    virtual status_t seekTo(int64_t /* seekTimeUs */) {...}    virtual status_t setBuffers(bool /* audio */, Vector<MediaBuffer *> &/* buffers */) {...}    virtual bool isRealTime() const {return false;}    virtual bool isStreaming() const {return true;}    virtual void setOffloadAudio(bool /* offload */) {}protected:    virtual ~Source() {}    virtual void onMessageReceived(const sp<AMessage> &msg);    sp<AMessage> dupNotify() const { return mNotify->dup(); }    void notifyFlagsChanged(uint32_t flags);    void notifyVideoSizeChanged(const sp<AMessage> &format = NULL);    void notifyInstantiateSecureDecoders(const sp<AMessage> &reply);    void notifyPrepared(status_t err = OK);private:    sp<AMessage> mNotify;};

从类层次结构来看，NuPlayer::Source的子类有：GenericSource、HTTPLiveSource、RTSPSource、StreamingSource。
本文先从最简单的GenericSource开始分析。

GenericSource对外接口及主要成员

鉴于存在继承关系，这里仅给出GenericSource特有的接口，及其主要成员函数：

// 注意这里有部分代码删减，并不是全部struct NuPlayer::GenericSource : public NuPlayer::Source {    GenericSource(const sp<AMessage> &notify, bool uidValid, uid_t uid);    status_t setDataSource(const sp<IMediaHTTPService> &httpService,            const char *url, const KeyedVector<String8, String8> *headers);    status_t setDataSource(int fd, int64_t offset, int64_t length);    status_t setDataSource(const sp<DataSource>& dataSource);private:    struct Track {        size_t mIndex;        sp<IMediaSource> mSource;        sp<AnotherPacketSource> mPackets;    };    // Helper to monitor buffering status. The polling happens every second.    // When necessary, it will send out buffering events to the player.    struct BufferingMonitor : public AHandler { ... };    Vector<sp<IMediaSource> > mSources;    Track mAudioTrack; // 音频流    int64_t mAudioTimeUs;    int64_t mAudioLastDequeueTimeUs;    Track mVideoTrack; // 视频流    int64_t mVideoTimeUs;    int64_t mVideoLastDequeueTimeUs;    Track mSubtitleTrack; // 字幕流    Track mTimedTextTrack;    sp<IMediaHTTPService> mHTTPService;    AString mUri;    KeyedVector<String8, String8> mUriHeaders;    int mFd;    int64_t mOffset;    int64_t mLength;    sp<DataSource> mDataSource;    sp<NuCachedSource2> mCachedSource;    sp<DataSource> mHttpSource;    sp<WVMExtractor> mWVMExtractor;    sp<MetaData> mFileMeta;    DrmManagerClient *mDrmManagerClient;    sp<DecryptHandle> mDecryptHandle;    bool mStarted;    bool mStopRead;    int64_t mBitrate;    sp<BufferingMonitor> mBufferingMonitor;    uint32_t mPendingReadBufferTypes;    sp<ABuffer> mGlobalTimedText;    sp<ALooper> mLooper;    sp<ALooper> mBufferingMonitorLooper;};

从这里可以看到GenericSource添加了setDataSource接口，并包含多个Track和各种DataSource/NuCachedSource2。

2 NuPlayer中调用的Source接口

先回顾下NuPlayer源码解析中的调用的NuPlayer::Source接口。

• 构建部分接口——构造/析构函数/setDataSource
• 基本播放控制接口——prepareAsync/stop/start/pause/resume/seekTo/disconnect
• 节目信息相关——getTrackCount/getTrackInfo/getSelectedTrack/selectTrack/getFormat
• 辅助信息获取及设置——getDuration/isRealTime/getFormatMeta/isStreaming/setOffloadAudio/setBuffers/setBuffers/feedMoreTSData

一个典型的GenericS初始化逻辑如下：

    sp<AMessage> notify = new AMessage(kWhatSourceNotify, this);    sp<GenericSource> source = new GenericSource(notify, mUIDValid, mUID);    status_t err = source->setDataSource(fd, offset, length);

3 构建部分接口

构造函数

GenericSource的构造函数相对简单，代码如下：

void NuPlayer::GenericSource::resetDataSource() {    mHTTPService.clear();    mHttpSource.clear();    mUri.clear();    mUriHeaders.clear();    if (mFd >= 0) {        close(mFd);        mFd = -1;    }    mOffset = 0;    mLength = 0;    setDrmPlaybackStatusIfNeeded(Playback::STOP, 0);    mDecryptHandle = NULL;    mDrmManagerClient = NULL;    mStarted = false;    mStopRead = true;    if (mBufferingMonitorLooper != NULL) {        mBufferingMonitorLooper->unregisterHandler(mBufferingMonitor->id());        mBufferingMonitorLooper->stop();        mBufferingMonitorLooper = NULL;    }    mBufferingMonitor->stop();}NuPlayer::GenericSource::GenericSource(        const sp<AMessage> &notify,        bool uidValid,        uid_t uid)    : Source(notify),      mAudioTimeUs(0),      mAudioLastDequeueTimeUs(0),      mVideoTimeUs(0),      mVideoLastDequeueTimeUs(0),      mFetchSubtitleDataGeneration(0),      mFetchTimedTextDataGeneration(0),      mDurationUs(-1ll),      mAudioIsVorbis(false),      mIsWidevine(false),      mIsSecure(false),      mIsStreaming(false),      mUIDValid(uidValid),      mUID(uid),      mFd(-1),      mDrmManagerClient(NULL),      mBitrate(-1ll),      mPendingReadBufferTypes(0) {    mBufferingMonitor = new BufferingMonitor(notify);    resetDataSource();    DataSource::RegisterDefaultSniffers(); // 这部分注意下，各种格式的探测链就是在这里初始化的}

逻辑比较简单都是一些变量及参数的初始化。比较有意思的是关于Sniffer的注册。我们看一下对应代码：

// code from ~/frameworks/av/media/libstagefright/DataSource.cppMutex DataSource::gSnifferMutex;List<DataSource::SnifferFunc> DataSource::gSniffers;bool DataSource::gSniffersRegistered = false;// staticvoid DataSource::RegisterSniffer_l(SnifferFunc func) {    for (List<SnifferFunc>::iterator it = gSniffers.begin();         it != gSniffers.end(); ++it) {        if (*it == func) {            return;        }    }    gSniffers.push_back(func);}// staticvoid DataSource::RegisterDefaultSniffers() {    Mutex::Autolock autoLock(gSnifferMutex);    if (gSniffersRegistered) {        return;    }    RegisterSniffer_l(SniffMPEG4); // mpeg4    RegisterSniffer_l(SniffMatroska); // mkv    RegisterSniffer_l(SniffOgg); // ogg    RegisterSniffer_l(SniffWAV); // wav    RegisterSniffer_l(SniffFLAC); // flac    RegisterSniffer_l(SniffAMR); // amr    RegisterSniffer_l(SniffMPEG2TS); // mpeg-ts    RegisterSniffer_l(SniffMP3); // mp3    RegisterSniffer_l(SniffAAC); // aac    RegisterSniffer_l(SniffMPEG2PS); // mpeg-ps    if (getuid() == AID_MEDIA) {        // WVM only in the media server process        RegisterSniffer_l(SniffWVM);    }    RegisterSniffer_l(SniffMidi);    char value[PROPERTY_VALUE_MAX];    if (property_get("drm.service.enabled", value, NULL)            && (!strcmp(value, "1") || !strcasecmp(value, "true"))) {        RegisterSniffer_l(SniffDRM);    }    gSniffersRegistered = true;}

上面代码就是将所有支持的容器格式放到一个List中。后面会用到的。

析构函数

析构函数相对简单，直接销毁Looper，重置DataSource即可，代码如下：

NuPlayer::GenericSource::~GenericSource() {    if (mLooper != NULL) {        mLooper->unregisterHandler(id());        mLooper->stop();    }    resetDataSource();}

setDataSource接口

设置数据源的接口，有三个重载函数，这里以file_descriptor的接口为例给出，实现非常简单，就是保存下参数就算完成了。代码如下：

status_t NuPlayer::GenericSource::setDataSource(        int fd, int64_t offset, int64_t length) {    resetDataSource();    mFd = dup(fd);    mOffset = offset;    mLength = length;    // delay data source creation to prepareAsync() to avoid blocking    // the calling thread in setDataSource for any significant time.    return OK;}

4 基本播放控制接口

这一部分的接口可以认为是一个多媒体文件播放必然会调用的接口。

prepareAsync

代码里边主要是发送kWhatPrepareAsync消息，如下：

void NuPlayer::GenericSource::prepareAsync() {    sp<AMessage> msg = new AMessage(kWhatPrepareAsync, this);    msg->post();}

具体消息处理是在onPrepareAsync中。这个函数完成了格式探测和Metadata提取等操作，处理代码如下：

// 注意代码有删减void NuPlayer::GenericSource::onPrepareAsync() {    // delayed data source creation 创建DataSource    if (mDataSource == NULL) {        {            mIsWidevine = false;            mDataSource = new FileSource(mFd, mOffset, mLength);            mFd = -1;        }        if (mDataSource == NULL) {            ALOGE("Failed to create data source!");            notifyPreparedAndCleanup(UNKNOWN_ERROR);            return;        }    }    if (mDataSource->flags() & DataSource::kIsCachingDataSource) {        mCachedSource = static_cast<NuCachedSource2 *>(mDataSource.get());    }    // init extractor from data source    status_t err = initFromDataSource();    if (err != OK) {        ALOGE("Failed to init from data source!");        notifyPreparedAndCleanup(err); // 上报处理结果        return;    }    if (mVideoTrack.mSource != NULL) {        sp<MetaData> meta = doGetFormatMeta(false /* audio */);        sp<AMessage> msg = new AMessage;        err = convertMetaDataToMessage(meta, &msg);        if(err != OK) {            notifyPreparedAndCleanup(err);            return;        }        notifyVideoSizeChanged(msg); // 上报视频分辨率    }    // 上报流状态    notifyFlagsChanged(            (mIsSecure ? FLAG_SECURE : 0)            | (mDecryptHandle != NULL ? FLAG_PROTECTED : 0)            | FLAG_CAN_PAUSE            | FLAG_CAN_SEEK_BACKWARD            | FLAG_CAN_SEEK_FORWARD            | FLAG_CAN_SEEK);    finishPrepareAsync();// 上报函数调用正常结束}

这里重点关注下initFromDataSource函数，因为这里面包含多媒体文件格式探测，代码如下：

status_t NuPlayer::GenericSource::initFromDataSource() {    sp<IMediaExtractor> extractor;    String8 mimeType;    float confidence;    sp<AMessage> dummy;      CHECK(mDataSource != NULL);    {        extractor = MediaExtractor::Create(mDataSource,                mimeType.isEmpty() ? NULL : mimeType.string());    }    if (extractor == NULL) {        return UNKNOWN_ERROR;    }    if (extractor->getDrmFlag()) {        checkDrmStatus(mDataSource);    }    mFileMeta = extractor->getMetaData();    if (mFileMeta != NULL) {        int64_t duration;        if (mFileMeta->findInt64(kKeyDuration, &duration)) {            mDurationUs = duration;        }    }    int32_t totalBitrate = 0;    size_t numtracks = extractor->countTracks();    if (numtracks == 0) {        return UNKNOWN_ERROR;    }    // 读取多媒体文件的全部Track信息    for (size_t i = 0; i < numtracks; ++i) {        sp<IMediaSource> track = extractor->getTrack(i);        if (track == NULL) {            continue;        }        sp<MetaData> meta = extractor->getTrackMetaData(i);        if (meta == NULL) {            ALOGE("no metadata for track %zu", i);            return UNKNOWN_ERROR;        }        const char *mime;        CHECK(meta->findCString(kKeyMIMEType, &mime));        // 处理下音视频信息，并保存 ... 省略部分代码        mSources.push(track);        int64_t durationUs;        if (meta->findInt64(kKeyDuration, &durationUs)) {            if (durationUs > mDurationUs) {                mDurationUs = durationUs;            }        }        int32_t bitrate;        if (totalBitrate >= 0 && meta->findInt32(kKeyBitRate, &bitrate)) {            totalBitrate += bitrate;        } else {            totalBitrate = -1;        }    }    if (mSources.size() == 0) {        ALOGE("b/23705695");        return UNKNOWN_ERROR;    }    mBitrate = totalBitrate;    return OK;}

最终通过MediaExtractor::CreateFromService调用DataSource::sniff函数来判断具体类型。sniff的实现代码如下：

bool DataSource::sniff(String8 *mimeType, float *confidence, sp<AMessage> *meta) {    *mimeType = "";    *confidence = 0.0f;    meta->clear();    // 遍历，找得分最高的，注意需要遍历全部支持的格式    for (List<SnifferFunc>::iterator it = gSniffers.begin();         it != gSniffers.end(); ++it) {        String8 newMimeType;        float newConfidence;        sp<AMessage> newMeta;        if ((*it)(this, &newMimeType, &newConfidence, &newMeta)) {            if (newConfidence > *confidence) {                *mimeType = newMimeType;                *confidence = newConfidence;                *meta = newMeta;            }        }    }    return *confidence > 0.0;}

stop/start

stop函数实现相对简单，直接修改当前状态。代码如下：

void NuPlayer::GenericSource::stop() {    // nothing to do, just account for DRM playback status    setDrmPlaybackStatusIfNeeded(Playback::STOP, 0);    mStarted = false;    // ...}

start函数主要发送kWhatStart消息，代码如下：

void NuPlayer::GenericSource::start() {    mStopRead = false;    if (mAudioTrack.mSource != NULL) { // 启动音频包读取        postReadBuffer(MEDIA_TRACK_TYPE_AUDIO);    }    if (mVideoTrack.mSource != NULL) { // 启动视频包读取        postReadBuffer(MEDIA_TRACK_TYPE_VIDEO);    }    setDrmPlaybackStatusIfNeeded(Playback::START, getLastReadPosition() / 1000);    mStarted = true;    (new AMessage(kWhatStart, this))->post();}

实际消息响应函数比较简单，如下：

case kWhatStart:case kWhatResume:{  mBufferingMonitor->restartPollBuffering();  break;}

pause/resume

这两个函数跟start/pause类似，直接设置状态值，代码如下：

void NuPlayer::GenericSource::pause() {    // nothing to do, just account for DRM playback status    setDrmPlaybackStatusIfNeeded(Playback::PAUSE, 0);    mStarted = false;}void NuPlayer::GenericSource::resume() {    // nothing to do, just account for DRM playback status    setDrmPlaybackStatusIfNeeded(Playback::START, getLastReadPosition() / 1000);    mStarted = true;    (new AMessage(kWhatResume, this))->post();}

seekTo

seekTo是实现多媒体文件seek的主要函数，其实现跟kWhatSeek消息有关，代码如下：

status_t NuPlayer::GenericSource::seekTo(int64_t seekTimeUs) {    sp<AMessage> msg = new AMessage(kWhatSeek, this);    msg->setInt64("seekTimeUs", seekTimeUs);    sp<AMessage> response;    status_t err = msg->postAndAwaitResponse(&response);    if (err == OK && response != NULL) {        CHECK(response->findInt32("err", &err));    }    return err;}

实际消息响应函数在onSeek中，代码如下：

void NuPlayer::GenericSource::onSeek(sp<AMessage> msg) {    int64_t seekTimeUs;    CHECK(msg->findInt64("seekTimeUs", &seekTimeUs));    sp<AMessage> response = new AMessage;    status_t err = doSeek(seekTimeUs); // 这是实际作seek的    response->setInt32("err", err);    sp<AReplyToken> replyID;    CHECK(msg->senderAwaitsResponse(&replyID));    response->postReply(replyID);}status_t NuPlayer::GenericSource::doSeek(int64_t seekTimeUs) {    mBufferingMonitor->updateDequeuedBufferTime(-1ll);    // If the Widevine source is stopped, do not attempt to read any more buffers.    if (mStopRead) {        return INVALID_OPERATION;    }    if (mVideoTrack.mSource != NULL) { // 调整视频读取时间        int64_t actualTimeUs;        readBuffer(MEDIA_TRACK_TYPE_VIDEO, seekTimeUs, &actualTimeUs);        seekTimeUs = actualTimeUs;        mVideoLastDequeueTimeUs = seekTimeUs;    }    if (mAudioTrack.mSource != NULL) { // 调整音频读取时间        readBuffer(MEDIA_TRACK_TYPE_AUDIO, seekTimeUs);        mAudioLastDequeueTimeUs = seekTimeUs;    }    setDrmPlaybackStatusIfNeeded(Playback::START, seekTimeUs / 1000);    if (!mStarted) {        setDrmPlaybackStatusIfNeeded(Playback::PAUSE, 0);    }    // If currently buffering, post kWhatBufferingEnd first, so that    // NuPlayer resumes. Otherwise, if cache hits high watermark    // before new polling happens, no one will resume the playback.    mBufferingMonitor->stopBufferingIfNecessary();    mBufferingMonitor->restartPollBuffering();    return OK;}

disconnect

这个函数主要断开DataSource和GenericSource之间的关联，保证后续可用，代码如下：

void NuPlayer::GenericSource::disconnect() {    sp<DataSource> dataSource, httpSource;    {        Mutex::Autolock _l(mDisconnectLock);        dataSource = mDataSource;        httpSource = mHttpSource;    }    if (dataSource != NULL) {        // disconnect data source        if (dataSource->flags() & DataSource::kIsCachingDataSource) {            static_cast<NuCachedSource2 *>(dataSource.get())->disconnect();        }    } else if (httpSource != NULL) {        static_cast<HTTPBase *>(httpSource.get())->disconnect();    }}

5 节目信息相关

getTrackCount、getTrackInfo、selectTrack和getSelectedTrack

这几个函数都是跟节目选择有关的，getTrackCount返回当前Source中包含的Track数目（一个Track可以是音频、视频、字幕或者文本），getTrackInfo则返回对应索引的详细信息。getSelectedTrack则返回当前选择或者正在播放的Track信息。selectTrack则用于选定特定的读取Track，也可用于取消读取。

getFormat

这个接口用于获取音频或者视频格式，实现如下：

sp<AMessage> NuPlayer::Source::getFormat(bool audio) {    sp<MetaData> meta = getFormatMeta(audio);    if (meta == NULL) {        return NULL;    }    sp<AMessage> msg = new AMessage;    if(convertMetaDataToMessage(meta, &msg) == OK) {        return msg;    }    return NULL;}

也就是说可以看看getFormatMeta的实现逻辑，代码如下：

sp<MetaData> NuPlayer::GenericSource::getFormatMeta(bool audio) {    sp<AMessage> msg = new AMessage(kWhatGetFormat, this);    msg->setInt32("audio", audio);    sp<AMessage> response;    sp<RefBase> format;    status_t err = msg->postAndAwaitResponse(&response);    if (err == OK && response != NULL) {        CHECK(response->findObject("format", &format));        return static_cast<MetaData*>(format.get());    } else {        return NULL;    }}

主要是发送kWhatGetFormat消息，然后交给DataSource处理。

6 辅助信息获取及设置

这里面几个函数都比较简单，多数信息都是在prepareAsync函数中读取的。这里仅列出代码：

getDuration

status_t NuPlayer::GenericSource::getDuration(int64_t *durationUs) {    *durationUs = mDurationUs;    return OK;}

isRealTime、isStreaming

virtual bool isRealTime() const {    return false;}bool NuPlayer::GenericSource::isStreaming() const {    return mIsStreaming;}

setOffloadAudio/setBuffers

void NuPlayer::GenericSource::setOffloadAudio(bool offload) {    mBufferingMonitor->setOffloadAudio(offload);}status_t NuPlayer::GenericSource::setBuffers(        bool audio, Vector<MediaBuffer *> &buffers) {    if (mIsSecure && !audio && mVideoTrack.mSource != NULL) {        return mVideoTrack.mSource->setBuffers(buffers);    }    return INVALID_OPERATION;}

feedMoreTSData

// 用于测试是否处理完所有数据status_t NuPlayer::GenericSource::feedMoreTSData() {    return OK;}

7 小结

断断续续的把本文整理完成，算是基本整清楚了针对文件读取的解析过程和调用逻辑。本文代码居多，有点乱。不建议参考，如果有问题，还是直接阅读代码吧。