Android Camera HAL3中预览preview模式下的控制流

本文均属自己阅读源码的点滴总结，转账请注明出处谢谢。

欢迎和大家交流。qq:1037701636 email:[e-mail]

Software：系统源码Android5.1

Camera3研读前沿：

    当初在研读Camera1.0相关的内容时，主要围绕着CameraClient、CameraHardwareInterface等方面进行工作的开展，无论是数据流还是控制流看起来都很简单、明了，一系列的流程化操作使得整个框架学起来特别的容易。因为没有Camera2.0相关的基础，所以这次直接看3.0相关的源码时，显得十分的吃紧，再加上底层高通HAL3.0实现的过程也是相当的复杂，都给整个研读过程带来了很多的困难。可以说，自身目前对Camera3.0框架的熟悉度也大概只有70%左右，希望通过总结来进一步梳理他的工作原理与整个框架，并进一步熟悉与加深理解。

1.Camera3下的整体架构图。

整个CameraService建立起一个可用操作底层Camera device大致需要经过Camera2Client、Camera3Device以及HAL层的camera3_device_t三个部分。

从上图中可以发现Camera3架构看上去明显比camera1来的复杂，但他更加的模块化。对比起Android4.2.2 Camer系统架构图（HAL和回调处理）一文中描述的单顺序执行流程，Camera3将更多的工作集中在了Framework去完成，将更多的控制权掌握在自己的手里，从而与HAL的交互的数据信息更少，也进一步减轻了一些在旧版本中HAL层所需要做的事情。

2. Camera2Client的建立与初始化过程

在建立好Camera2Client后会进行initialize操作，完成各个处理模块的创建：

   ....mStreamingProcessor = new StreamingProcessor(this);//preview和recorder    threadName = String8::format("C2-%d-StreamProc",            mCameraId);    mStreamingProcessor->run(threadName.string());//预览与录像    mFrameProcessor = new FrameProcessor(mDevice, this);// 3A    threadName = String8::format("C2-%d-FrameProc",            mCameraId);    mFrameProcessor->run(threadName.string()); //3A    mCaptureSequencer = new CaptureSequencer(this);    threadName = String8::format("C2-%d-CaptureSeq",            mCameraId);    mCaptureSequencer->run(threadName.string());//录像，拍照    mJpegProcessor = new JpegProcessor(this, mCaptureSequencer);    threadName = String8::format("C2-%d-JpegProc",            mCameraId);    mJpegProcessor->run(threadName.string());.... mCallbackProcessor = new CallbackProcessor(this);//回调处理    threadName = String8::format("C2-%d-CallbkProc",            mCameraId);    mCallbackProcessor->run(threadName.string());

依次分别创建了：

StreamingProcessor并启动一个他所属的thread，该模块主要负责处理previews与record两种视频流的处理，用于从hal层获取原始的视频数据

FrameProcessor并启动一个thread，该模块专门用于处理回调回来的每一帧的3A等信息，即每一帧视频除去原始视频数据外，还应该有其他附加的数据信息，如3A值。

CaptureSequencer并启动一个thread，该模块需要和其他模块配合使用，主要用于向APP层告知capture到的picture。
JpegProcessor并启动一个thread，该模块和streamprocessor类似，他启动一个拍照流，一般用于从HAL层获取jpeg编码后的图像照片数据。

此外ZslProcessor模块称之为0秒快拍，其本质是直接从原始的Preview流中获取预存着的最近的几帧，直接编码后返回给APP，而不需要再经过take picture去请求获取jpeg数据。0秒快拍技术得意于当下处理器CSI2 MIPI性能的提升以及Sensor支持全像素高帧率的实时输出。一般手机拍照在按下快门后都会有一定的延时，是因为需要切换底层Camera以及ISP等的工作模式，并重新设置参数以及重新对焦等等，都需要花一定时间后才抓取一帧用于编码为jpeg图像。

以上5个模块整合在一起基本上实现了Camera应用开发所需的基本业务功能。

3. 预览Preview下的控制流

研读Camera具体的业务处理功能，一般从视频实时预览Preview入手。一般熟悉Camera架构的人，可以从一个app端的一个api一直连续打通到底层hal的一个控制命令。大致可以如下图所示：

对于preview部分到CameraService的控制流可以参考博文Android4.2.2的preview的数据流和控制流以及最终的预览显示，本文将直接从Camera2Client::startPreview() 作为入口来分析整个Framework层中Preview相关的数据流。

status_t Camera2Client::startPreview() {    ATRACE_CALL();    ALOGV("%s: E", __FUNCTION__);    Mutex::Autolock icl(mBinderSerializationLock);    status_t res;    if ( (res = checkPid(__FUNCTION__) ) != OK) return res;    SharedParameters::Lock l(mParameters);    return startPreviewL(l.mParameters, false);}

startPreview通过startPreviewL提取参数后真正的开始执行Preview相关的控制流。该函数看上去内容虽然较多，但基本采用了同一种处理方式：

status_t Camera2Client::startPreviewL(Parameters ¶ms, bool restart) {//restart == false    ATRACE_CALL();    status_t res;......   int lastPreviewStreamId = mStreamingProcessor->getPreviewStreamId();//获取上一层Preview stream id   res = mStreamingProcessor->updatePreviewStream(params);//创建camera3device stream， Camera3OutputStream.....   int lastJpegStreamId = mJpegProcessor->getStreamId();   res = updateProcessorStream(mJpegProcessor, params);//预览启动时就建立一个jpeg的outstream.....        res = mCallbackProcessor->updateStream(params);//回调处理建立一个Camera3outputstream        if (res != OK) {            ALOGE("%s: Camera %d: Unable to update callback stream: %s (%d)",                    __FUNCTION__, mCameraId, strerror(-res), res);            return res;        }        outputStreams.push(getCallbackStreamId());......    outputStreams.push(getPreviewStreamId());//预览stream......   if (!params.recordingHint) {        if (!restart) {            res = mStreamingProcessor->updatePreviewRequest(params);//request处理,更新了mPreviewrequest            if (res != OK) {                ALOGE("%s: Camera %d: Can't set up preview request: "                        "%s (%d)", __FUNCTION__, mCameraId,                        strerror(-res), res);                return res;            }        }        res = mStreamingProcessor->startStream(StreamingProcessor::PREVIEW,                outputStreams);//启动stream，传入outputStreams即stream 的id    } else {        if (!restart) {            res = mStreamingProcessor->updateRecordingRequest(params);            if (res != OK) {                ALOGE("%s: Camera %d: Can't set up preview request with "                        "record hint: %s (%d)", __FUNCTION__, mCameraId,                        strerror(-res), res);                return res;            }        }        res = mStreamingProcessor->startStream(StreamingProcessor::RECORD,                outputStreams);    }......}

(1). mStreamingProcessor->updatePreviewStream()

由预览与录像处理模块更新一个预览流，其实现过程如下：

status_t StreamingProcessor::updatePreviewStream(const Parameters ¶ms) {    ATRACE_CALL();    Mutex::Autolock m(mMutex);    status_t res;    sp<CameraDeviceBase> device = mDevice.promote();//Camera3Device    if (device == 0) {        ALOGE("%s: Camera %d: Device does not exist", __FUNCTION__, mId);        return INVALID_OPERATION;    }    if (mPreviewStreamId != NO_STREAM) {        // Check if stream parameters have to change        uint32_t currentWidth, currentHeight;        res = device->getStreamInfo(mPreviewStreamId,                ¤tWidth, ¤tHeight, 0);        if (res != OK) {            ALOGE("%s: Camera %d: Error querying preview stream info: "                    "%s (%d)", __FUNCTION__, mId, strerror(-res), res);            return res;        }        if (currentWidth != (uint32_t)params.previewWidth ||                currentHeight != (uint32_t)params.previewHeight) {            ALOGV("%s: Camera %d: Preview size switch: %d x %d -> %d x %d",                    __FUNCTION__, mId, currentWidth, currentHeight,                    params.previewWidth, params.previewHeight);            res = device->waitUntilDrained();            if (res != OK) {                ALOGE("%s: Camera %d: Error waiting for preview to drain: "                        "%s (%d)", __FUNCTION__, mId, strerror(-res), res);                return res;            }            res = device->deleteStream(mPreviewStreamId);            if (res != OK) {                ALOGE("%s: Camera %d: Unable to delete old output stream "                        "for preview: %s (%d)", __FUNCTION__, mId,                        strerror(-res), res);                return res;            }            mPreviewStreamId = NO_STREAM;        }    }    if (mPreviewStreamId == NO_STREAM) {//首次create stream        res = device->createStream(mPreviewWindow,                params.previewWidth, params.previewHeight,                CAMERA2_HAL_PIXEL_FORMAT_OPAQUE, &mPreviewStreamId);//创建一个Camera3OutputStream        if (res != OK) {            ALOGE("%s: Camera %d: Unable to create preview stream: %s (%d)",                    __FUNCTION__, mId, strerror(-res), res);            return res;        }    }    res = device->setStreamTransform(mPreviewStreamId,            params.previewTransform);    if (res != OK) {        ALOGE("%s: Camera %d: Unable to set preview stream transform: "                "%s (%d)", __FUNCTION__, mId, strerror(-res), res);        return res;    }    return OK;}

该函数首先是查看当前StreamingProcessor模块下是否存在Stream，没有的话，则交由Camera3Device创建一个stream。显然，一个StreamingProcessor只能拥有一个PreviewStream，而一个Camera3Device显然控制着所有的Stream。

注意：在Camera2Client中，Stream大行其道，5大模块的数据交互均以stream作为基础。

下面我们来重点关注Camera3Device的接口createStream，他是5个模块创建stream的基础：

status_t Camera3Device::createStream(sp<ANativeWindow> consumer,        uint32_t width, uint32_t height, int format, int *id) {    ATRACE_CALL();    Mutex::Autolock il(mInterfaceLock);    Mutex::Autolock l(mLock);    ALOGV("Camera %d: Creating new stream %d: %d x %d, format %d",            mId, mNextStreamId, width, height, format);    status_t res;    bool wasActive = false;    switch (mStatus) {        case STATUS_ERROR:            CLOGE("Device has encountered a serious error");            return INVALID_OPERATION;        case STATUS_UNINITIALIZED:            CLOGE("Device not initialized");            return INVALID_OPERATION;        case STATUS_UNCONFIGURED:        case STATUS_CONFIGURED:            // OK            break;        case STATUS_ACTIVE:            ALOGV("%s: Stopping activity to reconfigure streams", __FUNCTION__);            res = internalPauseAndWaitLocked();            if (res != OK) {                SET_ERR_L("Can't pause captures to reconfigure streams!");                return res;            }            wasActive = true;            break;        default:            SET_ERR_L("Unexpected status: %d", mStatus);            return INVALID_OPERATION;    }    assert(mStatus != STATUS_ACTIVE);    sp<Camera3OutputStream> newStream;    if (format == HAL_PIXEL_FORMAT_BLOB) {//图片        ssize_t jpegBufferSize = getJpegBufferSize(width, height);        if (jpegBufferSize <= 0) {            SET_ERR_L("Invalid jpeg buffer size %zd", jpegBufferSize);            return BAD_VALUE;        }        newStream = new Camera3OutputStream(mNextStreamId, consumer,                width, height, jpegBufferSize, format);//jpeg 缓存的大小    } else {        newStream = new Camera3OutputStream(mNextStreamId, consumer,                width, height, format);//Camera3OutputStream    }    newStream->setStatusTracker(mStatusTracker);    res = mOutputStreams.add(mNextStreamId, newStream);//一个streamid与Camera3OutputStream绑定    if (res < 0) {        SET_ERR_L("Can't add new stream to set: %s (%d)", strerror(-res), res);        return res;    }    *id = mNextStreamId++;//至少一个previewstream 一般还有CallbackStream    mNeedConfig = true;    // Continue captures if active at start    if (wasActive) {        ALOGV("%s: Restarting activity to reconfigure streams", __FUNCTION__);        res = configureStreamsLocked();        if (res != OK) {            CLOGE("Can't reconfigure device for new stream %d: %s (%d)",                    mNextStreamId, strerror(-res), res);            return res;        }        internalResumeLocked();    }    ALOGV("Camera %d: Created new stream", mId);    return OK;}

该函数重点是关注一个new Camera3OutputStream，在Camera3Device主要存在Camera3OutputStream和Camera3InputStream
两种stream，前者主要作为HAL的输出，是请求HAL填充数据的OutPutStream，后者是由Framework将Stream进行填充。无论是Preview、record还是capture均是从HAL层获取数据，故都会以OutPutStream的形式存在，是我们关注的重点，后面在描述Preview的数据流时还会进一步的阐述。

每当创建一个OutPutStream后，相关的stream信息被push维护在一个mOutputStreams的KeyedVector<int, sp<camera3::Camera3OutputStreamInterface> >表中，分别是该stream在Camera3Device中创建时的ID以及Camera3OutputStream的sp值。同时对mNextStreamId记录下一个Stream的ID号。

上述过程完成StreamingProcessor模块中一个PreviewStream的创建，其中Camera3OutputStream创建时的ID值被返回记录作为mPreviewStreamId的值，此外每个Stream都会有一个对应的ANativeWindow，这里称之为Consumer。

（2）mCallbackProcessor->updateStream(params)

对比StreamingProcessor模块创建previewstream的过程，很容易定位到Callback模块是需要建立一个callback流，同样需要创建一个Camera3OutputStream来接收HAL返回的每一帧帧数据，是否需要callback可以通过callbackenable来控制。一般但预览阶段可能不需要回调每一帧的数据到APP，但涉及到相应的其他业务如视频处理时，就需要进行callback的enable。

status_t CallbackProcessor::updateStream(const Parameters ¶ms) {    ATRACE_CALL();    status_t res;    Mutex::Autolock l(mInputMutex);    sp<CameraDeviceBase> device = mDevice.promote();    if (device == 0) {        ALOGE("%s: Camera %d: Device does not exist", __FUNCTION__, mId);        return INVALID_OPERATION;    }    // If possible, use the flexible YUV format    int32_t callbackFormat = params.previewFormat;    if (mCallbackToApp) {        // TODO: etalvala: This should use the flexible YUV format as well, but        // need to reconcile HAL2/HAL3 requirements.        callbackFormat = HAL_PIXEL_FORMAT_YV12;    } else if(params.fastInfo.useFlexibleYuv &&            (params.previewFormat == HAL_PIXEL_FORMAT_YCrCb_420_SP ||             params.previewFormat == HAL_PIXEL_FORMAT_YV12) ) {        callbackFormat = HAL_PIXEL_FORMAT_YCbCr_420_888;    }    if (!mCallbackToApp && mCallbackConsumer == 0) {        // Create CPU buffer queue endpoint, since app hasn't given us one        // Make it async to avoid disconnect deadlocks        sp<IGraphicBufferProducer> producer;        sp<IGraphicBufferConsumer> consumer;        BufferQueue::createBufferQueue(&producer, &consumer);//BufferQueueProducer与BufferQueueConsumer        mCallbackConsumer = new CpuConsumer(consumer, kCallbackHeapCount);        mCallbackConsumer->setFrameAvailableListener(this);//当前CallbackProcessor继承于CpuConsumer::FrameAvailableListener        mCallbackConsumer->setName(String8("Camera2Client::CallbackConsumer"));        mCallbackWindow = new Surface(producer);//用于queue操作，这里直接进行本地的buffer操作    }    if (mCallbackStreamId != NO_STREAM) {        // Check if stream parameters have to change        uint32_t currentWidth, currentHeight, currentFormat;        res = device->getStreamInfo(mCallbackStreamId,                ¤tWidth, ¤tHeight, ¤tFormat);        if (res != OK) {            ALOGE("%s: Camera %d: Error querying callback output stream info: "                    "%s (%d)", __FUNCTION__, mId,                    strerror(-res), res);            return res;        }        if (currentWidth != (uint32_t)params.previewWidth ||                currentHeight != (uint32_t)params.previewHeight ||                currentFormat != (uint32_t)callbackFormat) {            // Since size should only change while preview is not running,            // assuming that all existing use of old callback stream is            // completed.            ALOGV("%s: Camera %d: Deleting stream %d since the buffer "                    "parameters changed", __FUNCTION__, mId, mCallbackStreamId);            res = device->deleteStream(mCallbackStreamId);            if (res != OK) {                ALOGE("%s: Camera %d: Unable to delete old output stream "                        "for callbacks: %s (%d)", __FUNCTION__,                        mId, strerror(-res), res);                return res;            }            mCallbackStreamId = NO_STREAM;        }    }    if (mCallbackStreamId == NO_STREAM) {        ALOGV("Creating callback stream: %d x %d, format 0x%x, API format 0x%x",                params.previewWidth, params.previewHeight,                callbackFormat, params.previewFormat);        res = device->createStream(mCallbackWindow,                params.previewWidth, params.previewHeight,                callbackFormat, &mCallbackStreamId);//Creating callback stream        if (res != OK) {            ALOGE("%s: Camera %d: Can't create output stream for callbacks: "                    "%s (%d)", __FUNCTION__, mId,                    strerror(-res), res);            return res;        }    }    return OK;}

对比updatePreviewStream可以发现，该函数自助创建了一套surface/BufferQueue/CpuConsumer的机制，这套类似SurfaceFlinger的buffer管理机制可参看一文Android5.1中surface生产者和消费者间的处理框架简述。此外通过createStream请求Camera3Device建立一个Stream，其中Stream的ID值保存在mCallBackStreamId当中，并将一个CallbackWindow和当前的Stream绑定。

通过这个对比，也需要重点关注到，对于每个Camera3OutPutStream来说，每一个stream都被一个Consumer，而在此处都是Surface(ANativeWindow)所拥有，这个Consumer和HAL相匹配来说是消费者，但对于真正的处理Buffer的Consumer来说如CPUConsumer，Surface却又是以一个Product的角色存在的。

（3）updateProcessorStream(mJpegProcessor, params)

status_t Camera2Client::updateProcessorStream(sp<ProcessorT> processor,                                              camera2::Parameters params) {    // No default template arguments until C++11, so we need this overload    return updateProcessorStream<ProcessorT, &ProcessorT::updateStream>(            processor, params);}template <typename ProcessorT,          status_t (ProcessorT::*updateStreamF)(const Parameters &)>status_t Camera2Client::updateProcessorStream(sp<ProcessorT> processor,                                              Parameters params) {    status_t res;    // Get raw pointer since sp<T> doesn't have operator->*    ProcessorT *processorPtr = processor.get();    res = (processorPtr->*updateStreamF)(params);.......}

该模板函数处理过程最终通过非显示实例到显示实例调用JpegProcessor::updateStream，该函数处理的逻辑基本和Callback模块处理一致，创建的一个OutPutStream和CaptureWindow相互绑定，同时Stream的ID保存在mCaptureStreamId中。

此外需要说明一点：

在preview模式下，就去创建一个jpeg处理的stream，目的在于启动takepicture时，可以更快的进行capture操作。是通过牺牲内存空间来提升效率。

（4）整合startPreviewL中所有的stream 到Vector<int32_t> outputStreams

outputStreams.push(getPreviewStreamId());//预览stream

outputStreams.push(getCallbackStreamId())//Callback stream

目前一次Preview构建的stream数目至少为两个。

（5）mStreamingProcessor->updatePreviewRequest()

在创建好多路stream后，由StreamingProcessor模块来将所有的stream信息交由Camera3Device去打包成Request请求。

注意：

Camera HAL2/3的特点是：将所有stream的请求都转化为几个典型的Request请求，而这些Request需要由HAL去解析，进而处理所需的业务。这也是Camera3数据处理复杂化的原因所在。

status_t StreamingProcessor::updatePreviewRequest(const Parameters ¶ms) {    ATRACE_CALL();    status_t res;    sp<CameraDeviceBase> device = mDevice.promote();    if (device == 0) {        ALOGE("%s: Camera %d: Device does not exist", __FUNCTION__, mId);        return INVALID_OPERATION;    }    Mutex::Autolock m(mMutex);    if (mPreviewRequest.entryCount() == 0) {        sp<Camera2Client> client = mClient.promote();        if (client == 0) {            ALOGE("%s: Camera %d: Client does not exist", __FUNCTION__, mId);            return INVALID_OPERATION;        }        // Use CAMERA3_TEMPLATE_ZERO_SHUTTER_LAG for ZSL streaming case.        if (client->getCameraDeviceVersion() >= CAMERA_DEVICE_API_VERSION_3_0) {            if (params.zslMode && !params.recordingHint) {                res = device->createDefaultRequest(CAMERA3_TEMPLATE_ZERO_SHUTTER_LAG,                        &mPreviewRequest);            } else {                res = device->createDefaultRequest(CAMERA3_TEMPLATE_PREVIEW,                        &mPreviewRequest);            }        } else {            res = device->createDefaultRequest(CAMERA2_TEMPLATE_PREVIEW,                    &mPreviewRequest);//创建一个Preview相关的request，由底层的hal来完成default创建        }        if (res != OK) {            ALOGE("%s: Camera %d: Unable to create default preview request: "                    "%s (%d)", __FUNCTION__, mId, strerror(-res), res);            return res;        }    }    res = params.updateRequest(&mPreviewRequest);//根据参数来更新CameraMetadata request    if (res != OK) {        ALOGE("%s: Camera %d: Unable to update common entries of preview "                "request: %s (%d)", __FUNCTION__, mId,                strerror(-res), res);        return res;    }    res = mPreviewRequest.update(ANDROID_REQUEST_ID,            &mPreviewRequestId, 1);//mPreviewRequest的ANDROID_REQUEST_ID    if (res != OK) {        ALOGE("%s: Camera %d: Unable to update request id for preview: %s (%d)",                __FUNCTION__, mId, strerror(-res), res);        return res;    }    return OK;}

该函数的处理过程是一个构建并初始化mPreviewRequest的过程，分以下几个流程来分析：

a mPreviewRequest是一个CameraMetadata类型数据，用于封装当前previewRequest。

b device->createDefaultRequest(CAMERA3_TEMPLATE_PREVIEW, &mPreviewRequest)

   const camera_metadata_t *rawRequest;    ATRACE_BEGIN("camera3->construct_default_request_settings");    rawRequest = mHal3Device->ops->construct_default_request_settings(        mHal3Device, templateId);    ATRACE_END();    if (rawRequest == NULL) {        SET_ERR_L("HAL is unable to construct default settings for template %d",                templateId);        return DEAD_OBJECT;    }    *request = rawRequest;    mRequestTemplateCache[templateId] = rawRequest;

最终是由hal来实现构建一个rawrequest,即对于Preview，而言是构建了一个CAMERA3_TEMPLATE_PREVIEW类型的Request。其实对HAL而言，rawrequest本质是用于操作一个camera_metadata_t类型的数据：

struct camera_metadata {    metadata_size_t          size;    uint32_t                 version;    uint32_t                 flags;    metadata_size_t          entry_count;    metadata_size_t          entry_capacity;    metadata_uptrdiff_t      entries_start; // Offset from camera_metadata    metadata_size_t          data_count;    metadata_size_t          data_capacity;    metadata_uptrdiff_t      data_start; // Offset from camera_metadata    uint8_t                  reserved[];};

该数据结构可以存储多种数据，且可以根据entry tag的不同类型来存储数据，同时数据量的大小也可以自动调整。

c mPreviewRequest.update(ANDROID_REQUEST_ID,&mPreviewRequestId, 1)

将当前的PreviewRequest相应的ID保存到camera metadata。

（6）mStreamingProcessor->startStream启动整个预览的stream流

该函数的处理过程较为复杂，可以说是整个Preview正常工作的核心控制

status_t StreamingProcessor::startStream(StreamType type,        const Vector<int32_t> &outputStreams) {.....CameraMetadata &request = (type == PREVIEW) ?            mPreviewRequest : mRecordingRequest;//取preview的CameraMetadata request....res = request.update(    ANDROID_REQUEST_OUTPUT_STREAMS,        outputStreams);//CameraMetadata中添加outputStreams res = device->setStreamingRequest(request);//向hal发送request.....}

该函数首先是根据当前工作模式来确定StreamingProcessor需要处理的Request，该模块负责Preview和Record两个Request。

以PreviewRequest就是之前createDefaultRequest构建的，这里先是将这个Request所需要操作的Outputstream打包到一个tag叫ANDROID_REQUEST_OUTPUT_STREAMS的entry当中。

a:setStreamingRequest

真正的请求Camera3Device去处理这个带有多路stream的PreviewRequest。

status_t Camera3Device::setStreamingRequest(const CameraMetadata &request,                                            int64_t* /*lastFrameNumber*/) {    ATRACE_CALL();    List<const CameraMetadata> requests;    requests.push_back(request);    return setStreamingRequestList(requests, /*lastFrameNumber*/NULL);}

该函数将mPreviewRequest push到一个list，调用setStreamingRequestList

status_t Camera3Device::setStreamingRequestList(const List<const CameraMetadata> &requests,                                                int64_t *lastFrameNumber) {    ATRACE_CALL();    return submitRequestsHelper(requests, /*repeating*/true, lastFrameNumber);}

status_t Camera3Device::submitRequestsHelper(        const List<const CameraMetadata> &requests, bool repeating,        /*out*/        int64_t *lastFrameNumber) {//repeating = 1;lastFrameNumber = NULL    ATRACE_CALL();    Mutex::Autolock il(mInterfaceLock);    Mutex::Autolock l(mLock);    status_t res = checkStatusOkToCaptureLocked();    if (res != OK) {        // error logged by previous call        return res;    }    RequestList requestList;    res = convertMetadataListToRequestListLocked(requests, /*out*/&requestList);//返回的是CaptureRequest RequestList    if (res != OK) {        // error logged by previous call        return res;    }    if (repeating) {        res = mRequestThread->setRepeatingRequests(requestList, lastFrameNumber);//重复的request存入到RequestThread    } else {        res = mRequestThread->queueRequestList(requestList, lastFrameNumber);//capture模式，拍照单词    }    if (res == OK) {        waitUntilStateThenRelock(/*active*/true, kActiveTimeout);        if (res != OK) {            SET_ERR_L("Can't transition to active in %f seconds!",                    kActiveTimeout/1e9);        }        ALOGV("Camera %d: Capture request %" PRId32 " enqueued", mId,              (*(requestList.begin()))->mResultExtras.requestId);    } else {        CLOGE("Cannot queue request. Impossible.");        return BAD_VALUE;    }    return res;}

b convertMetadataListToRequestListLocked

这个函数是需要将Requestlist中保存的CameraMetadata数据转换为List<sp<CaptureRequest> >

status_t Camera3Device::convertMetadataListToRequestListLocked(        const List<const CameraMetadata> &metadataList, RequestList *requestList) {    if (requestList == NULL) {        CLOGE("requestList cannot be NULL.");        return BAD_VALUE;    }    int32_t burstId = 0;    for (List<const CameraMetadata>::const_iterator it = metadataList.begin();//CameraMetadata, mPreviewRequest            it != metadataList.end(); ++it) {        sp<CaptureRequest> newRequest = setUpRequestLocked(*it);//新建CaptureRequest由CameraMetadata转化而来        if (newRequest == 0) {            CLOGE("Can't create capture request");            return BAD_VALUE;        }        // Setup burst Id and request Id        newRequest->mResultExtras.burstId = burstId++;        if (it->exists(ANDROID_REQUEST_ID)) {            if (it->find(ANDROID_REQUEST_ID).count == 0) {                CLOGE("RequestID entry exists; but must not be empty in metadata");                return BAD_VALUE;            }            newRequest->mResultExtras.requestId = it->find(ANDROID_REQUEST_ID).data.i32[0];//设置该request对应的id        } else {            CLOGE("RequestID does not exist in metadata");            return BAD_VALUE;        }        requestList->push_back(newRequest);        ALOGV("%s: requestId = %" PRId32, __FUNCTION__, newRequest->mResultExtras.requestId);    }    return OK;}

这里是对List<const CameraMetadata>进行迭代解析处理，如当前模式下仅存在PreviewRequest这一个CameraMetadata，通过setUpRequestLocked将其转换为一个CaptureRequest。

c 重点来关注setUpRequestLocked复杂的处理过程

sp<Camera3Device::CaptureRequest> Camera3Device::setUpRequestLocked(        const CameraMetadata &request) {//mPreviewRequest    status_t res;    if (mStatus == STATUS_UNCONFIGURED || mNeedConfig) {        res = configureStreamsLocked();      ......    sp<CaptureRequest> newRequest = createCaptureRequest(request);//CameraMetadata转为CaptureRequest，包含mOutputStreams    return newRequest;}

configureStreamsLocked函数主要是将Camera3Device侧建立的所有Stream包括Output与InPut格式的交由HAL3层的Device去实现处理的核心接口是configure_streams与register_stream_buffer。该部分内容会涉及到更多的数据流，详细的处理过程会放在下一博文中进行分析。

createCaptureRequest函数是将一个CameraMetadata格式的数据如PreviewRequest转换为一个CaptureRequest：

sp<Camera3Device::CaptureRequest> Camera3Device::createCaptureRequest(        const CameraMetadata &request) {//mPreviewRequest    ATRACE_CALL();    status_t res;    sp<CaptureRequest> newRequest = new CaptureRequest;    newRequest->mSettings = request;//CameraMetadata    camera_metadata_entry_t inputStreams =            newRequest->mSettings.find(ANDROID_REQUEST_INPUT_STREAMS);    if (inputStreams.count > 0) {        if (mInputStream == NULL ||                mInputStream->getId() != inputStreams.data.i32[0]) {            CLOGE("Request references unknown input stream %d",                    inputStreams.data.u8[0]);            return NULL;        }        // Lazy completion of stream configuration (allocation/registration)        // on first use        if (mInputStream->isConfiguring()) {            res = mInputStream->finishConfiguration(mHal3Device);            if (res != OK) {                SET_ERR_L("Unable to finish configuring input stream %d:"                        " %s (%d)",                        mInputStream->getId(), strerror(-res), res);                return NULL;            }        }        newRequest->mInputStream = mInputStream;        newRequest->mSettings.erase(ANDROID_REQUEST_INPUT_STREAMS);    }    camera_metadata_entry_t streams =            newRequest->mSettings.find(ANDROID_REQUEST_OUTPUT_STREAMS);//读取存储在CameraMetadata的stream id信息    if (streams.count == 0) {        CLOGE("Zero output streams specified!");        return NULL;    }    for (size_t i = 0; i < streams.count; i++) {        int idx = mOutputStreams.indexOfKey(streams.data.i32[i]);//Camera3OutputStream的id在mOutputStreams中        if (idx == NAME_NOT_FOUND) {            CLOGE("Request references unknown stream %d",                    streams.data.u8[i]);            return NULL;        }        sp<Camera3OutputStreamInterface> stream =                mOutputStreams.editValueAt(idx);//返回的是Camera3OutputStream，preview/callback等stream        // Lazy completion of stream configuration (allocation/registration)        // on first use        if (stream->isConfiguring()) {//STATE_IN_CONFIG或者STATE_IN_RECONFIG            res = stream->finishConfiguration(mHal3Device);//register_stream_buffer， STATE_CONFIGURED            if (res != OK) {                SET_ERR_L("Unable to finish configuring stream %d: %s (%d)",                        stream->getId(), strerror(-res), res);                return NULL;            }        }        newRequest->mOutputStreams.push(stream);//Camera3OutputStream添加到CaptureRequest的mOutputStreams    }    newRequest->mSettings.erase(ANDROID_REQUEST_OUTPUT_STREAMS);    return newRequest;}

该函数主要处理指定的这个CameraMetadata mPreviewRequest下对应所拥有的Output与Input Stream，对于Preview而言，至少存在OutPutStream包括一路StreamProcessor与一路可选的CallbackProcessor。

在构建这个PreviewRequest时，已经将ANDROID_REQUEST_OUTPUT_STREAMS这个Tag进行了初始化，相应的内容为Vector<int32_t> &outputStreams，包含着属于PreviewRequest这个Request所需要的输出stream的ID值，通过这个ID index值，可以遍历到Camera3Device下所createstream创造的Camera3OutputStream，即说明不同类型的Request在Camera3Device端存在多个Stream，而每次不同业务下所需要Request的对应的Stream又仅是其中的个别而已。

idx =  mOutputStreams.indexOfKey(streams.data.i32[i])是通过属于PreviewRequest中包含的一个stream的ID值来查找到mOutputStreams这个KeyedVector中对应的标定值index。注意：两个索引值不一定是一致的。

mOutputStreams.editValueAt(idx)是获取一个与该ID值(如Previewstream ID、Callback Stream ID等等)相对应的Camera3OutputStream。

在找到了当前Request中所有的Camera3OutputStream后，将其维护在CaptureRequest中

class CaptureRequest : public LightRefBase<CaptureRequest> {      public:        CameraMetadata                      mSettings;        sp<camera3::Camera3Stream>          mInputStream;        Vector<sp<camera3::Camera3OutputStreamInterface> >                                            mOutputStreams;        CaptureResultExtras                 mResultExtras;    };

mSettings是保存CameraMetadata PreviewRequest，vector mOutPutStreams保存着当前Request提取出来的Camera3OutputStream，至此构建了一个CaptureRequest。

返回到convertMetadataListToRequestListLocked中，现在已经完成了一个CameraMetadata Request的处理，生产的是一个CaptureRequest。我们将这个ANDROID_REQUEST_ID的ID值，保留在

newRequest->mResultExtras.requestId = it->find(ANDROID_REQUEST_ID).data.i32[0]。

这个值在整个Camera3的架构中，仅存在3大种Request类型，说明了整个和HAL层交互的Request类型是不多的：

预览Request mPreviewRequest：        mPreviewRequestId(Camera2Client::kPreviewRequestIdStart),

拍照Request mCaptureRequest：mCaptureId(Camera2Client::kCaptureRequestIdStart),

录像Request mRecordingRequest：        mRecordingRequestId(Camera2Client::kRecordingRequestIdStart),

    static const int32_t kPreviewRequestIdStart = 10000000;    static const int32_t kPreviewRequestIdEnd   = 20000000;    static const int32_t kRecordingRequestIdStart  = 20000000;    static const int32_t kRecordingRequestIdEnd    = 30000000;    static const int32_t kCaptureRequestIdStart = 30000000;    static const int32_t kCaptureRequestIdEnd   = 40000000;

至此执行requestList->push_back(newRequest)后生成了一个requestList，本质上可以先认为这次仅是含有PreviewRequest相关的内容。

d mRequestThread->setRepeatingRequests(requestList)

对于Preview来说，一次Preview后底层硬件就该可以连续的工作，而不需要进行过多的切换，故Framework每次向HAL发送的Request均是一种repeat的操作模式，故调用了一个重复的RequestQueue来循环处理每次的Request。

status_t Camera3Device::RequestThread::setRepeatingRequests(        const RequestList &requests,        /*out*/        int64_t *lastFrameNumber) {    Mutex::Autolock l(mRequestLock);    if (lastFrameNumber != NULL) {//第一次进来为null        *lastFrameNumber = mRepeatingLastFrameNumber;    }    mRepeatingRequests.clear();    mRepeatingRequests.insert(mRepeatingRequests.begin(),            requests.begin(), requests.end());    unpauseForNewRequests();//signal request_thread in waitfornextrequest    mRepeatingLastFrameNumber = NO_IN_FLIGHT_REPEATING_FRAMES;    return OK;}

将Preview线程提交的Request加入到mRepeatingRequests中后，唤醒RequestThread线程去处理当前新的Request。

(7) RequestThread 请求处理线程

RequestThread::threadLoop()函数主要用于响应并处理新加入到Request队列中的请求。

bool Camera3Device::RequestThread::threadLoop() {....    sp<CaptureRequest> nextRequest = waitForNextRequest();//返回的是mRepeatingRequests，mPreviewRequest    if (nextRequest == NULL) {        return true;    }    // Create request to HAL    camera3_capture_request_t request = camera3_capture_request_t();//CaptureRequest转为给HAL3.0的camera3_capture_request_t    request.frame_number = nextRequest->mResultExtras.frameNumber;//当前帧号    Vector<camera3_stream_buffer_t> outputBuffers;    // Get the request ID, if any    int requestId;    camera_metadata_entry_t requestIdEntry =            nextRequest->mSettings.find(ANDROID_REQUEST_ID);    if (requestIdEntry.count > 0) {        requestId = requestIdEntry.data.i32[0];//获取requestid,这里是mPreviewRequest的id    } else {        ALOGW("%s: Did not have android.request.id set in the request",                __FUNCTION__);        requestId = NAME_NOT_FOUND;    } .....    camera3_stream_buffer_t inputBuffer;    uint32_t totalNumBuffers = 0;.....    // Submit request and block until ready for next one    ATRACE_ASYNC_BEGIN("frame capture", request.frame_number);    ATRACE_BEGIN("camera3->process_capture_request");    res = mHal3Device->ops->process_capture_request(mHal3Device, &request);//调用底层的process_capture_request    ATRACE_END();   .......}

函数主体内容较为复杂，分以下几个部分来说明他的响应逻辑：

(7.1) waitForNextRequest()

        Camera3Device::RequestThread::waitForNextRequest() {    status_t res;    sp<CaptureRequest> nextRequest;    // Optimized a bit for the simple steady-state case (single repeating    // request), to avoid putting that request in the queue temporarily.    Mutex::Autolock l(mRequestLock);    while (mRequestQueue.empty()) {        if (!mRepeatingRequests.empty()) {            // Always atomically enqueue all requests in a repeating request            // list. Guarantees a complete in-sequence set of captures to            // application.            const RequestList &requests = mRepeatingRequests;            RequestList::const_iterator firstRequest =                    requests.begin();            nextRequest = *firstRequest;//取            mRequestQueue.insert(mRequestQueue.end(),                    ++firstRequest,                    requests.end());//把当前的mRepeatingRequests插入到mRequestQueue            // No need to wait any longer            mRepeatingLastFrameNumber = mFrameNumber + requests.size() - 1;            break;        }        res = mRequestSignal.waitRelative(mRequestLock, kRequestTimeout);//等待下一个request        if ((mRequestQueue.empty() && mRepeatingRequests.empty()) ||                exitPending()) {            Mutex::Autolock pl(mPauseLock);            if (mPaused == false) {                ALOGV("%s: RequestThread: Going idle", __FUNCTION__);                mPaused = true;                // Let the tracker know                sp<StatusTracker> statusTracker = mStatusTracker.promote();                if (statusTracker != 0) {                    statusTracker->markComponentIdle(mStatusId, Fence::NO_FENCE);                }            }            // Stop waiting for now and let thread management happen            return NULL;        }    }    if (nextRequest == NULL) {        // Don't have a repeating request already in hand, so queue        // must have an entry now.        RequestList::iterator firstRequest =                mRequestQueue.begin();        nextRequest = *firstRequest;        mRequestQueue.erase(firstRequest);//取一根mRequestQueue中的CaptureRequest，来自于mRepeatingRequests的next    }    // In case we've been unpaused by setPaused clearing mDoPause, need to    // update internal pause state (capture/setRepeatingRequest unpause    // directly).    Mutex::Autolock pl(mPauseLock);    if (mPaused) {        ALOGV("%s: RequestThread: Unpaused", __FUNCTION__);        sp<StatusTracker> statusTracker = mStatusTracker.promote();        if (statusTracker != 0) {            statusTracker->markComponentActive(mStatusId);        }    }    mPaused = false;    // Check if we've reconfigured since last time, and reset the preview    // request if so. Can't use 'NULL request == repeat' across configure calls.    if (mReconfigured) {        mPrevRequest.clear();        mReconfigured = false;    }    if (nextRequest != NULL) {        nextRequest->mResultExtras.frameNumber = mFrameNumber++;//对每一个非空的request需要帧号++        nextRequest->mResultExtras.afTriggerId = mCurrentAfTriggerId;        nextRequest->mResultExtras.precaptureTriggerId = mCurrentPreCaptureTriggerId;    }    return nextRequest;}

该函数是响应RequestList的核心，通过不断的轮训休眠等待一旦mRepeatingRequests有Request可处理时，就将他内部所有的CaptureRequest加入到mRequestQueue 中去，理论来说每一个CaptureRequest对应着一帧的请求处理，每次响应时可能会出现mRequestQueue包含了多个CaptureRequest。

通过nextRequest->mResultExtras.frameNumber = mFrameNumber++表示当前CaptureRequest在处理的一帧图像号。

对于mRepeatingRequests而言，只有其非空，在执行完一次queue操作后，在循环进入执行时，会自动对mRequestQueue进行erase操作，是的mRequestQueue变为empty后再次重新加载mRepeatingRequests中的内容，从而形成一个队repeatRequest的重复响应过程。

(7.2)   camera_metadata_entry_t requestIdEntry = nextRequest->mSettings.find(ANDROID_REQUEST_ID);提取该CaptureRequest对应的Request 类型值

(7.3) getBuffer操作

涉及到比较复杂的数据流操作过程的内容见下一博文

(7.4) mHal3Device->ops->process_capture_request(mHal3Device, &request)

这里的request是已经由一个CaptureRequest转换为和HAL3.0交互的camera3_capture_request_t结构。

8 小结

至此已经完成了一次向HAL3.0 Device发送一次完整的Request的请求。从最初Preview启动建立多个OutPutStream，再是将这些Stream打包成一个mPreviewRequest来启动stream，随后将这个Request又转变为一个CaptureRequest，直到转为Capture list后交由RequestThread来处理这些请求。每一次的Request简单可以说是Camera3Device向HAL3.0请求一帧数据，当然每一次Request也可以包含各种控制操作，如AutoFocus等内容，会在后续补充。

到这里从StartPreview的入口开始，直到相应的Request下发到HAL3.0，基本描述了一次完成的控制流的处理。对于较为复杂的数据流本质也是一并合并在这个控制操作中的，但作为Buffer视频缓存流的管理维护将在下一博文进行描述与总结。