mongodb源码分析(四)查询1之mongo的查询请求

           在之前的2篇文章中分别分析了mongod和mongo的启动流程，下面开始将分析mongodb的查询，由于查询部分流程比较长，将分成mongo端的请求，mongod端的数据库的加载，mongod query的选取，mongod文档的匹配与数据的响应几部分来分析。

         首先进入mongo的查询请求部分.mongo的查询请求部分归纳起来很简单就是将请求分装成一个Message结构,然后将其发送到服务端,等待服务端的相应数据,取得数据最后显示结果.下面来看具体流程分析吧.

   当我们点击db.coll.find({x:1})时按照上一篇文章的讲解，我们首先来到了mongo/shell/dbshell.cpp

            if ( ! wascmd ) {                try {                    if ( scope->exec( code.c_str() , "(shell)" , false , true , false ) )//执行相应的javascript代码                        scope->exec( "shellPrintHelper( __lastres__ );" , "(shell2)" , true , true , false );                }                catch ( std::exception& e ) {                    cout << "error:" << e.what() << endl;                }            }

下面进入javascript代码,其在mongo/shell/collection.js.

//这里因为我们只设置了query,所以其它选项都是空的，this.getQueryOptions()目前只有一个SlaveOK的option，在replset模式下是不能查询secondary服务器的，需要调用rs.SlaveOK()之后才能对secondary进行查询，其执行SlaveOK后每次查询时都会添加一个QueryOption.DBCollection.prototype.find = function (query, fields, limit, skip, batchSize, options) {    return new DBQuery( this._mongo , this._db , this ,                        this._fullName , this._massageObject( query ) , fields , limit , skip , batchSize , options || this.getQueryOptions() );}

继续前进看看DBQuery,上一篇文章提到这里的new DBQuery对象的创建发生在:

    JSBool dbquery_constructor( JSContext *cx, JSObject *obj, uintN argc, jsval *argv, jsval *rval ) {        try {            smuassert( cx ,  "DDQuery needs at least 4 args" , argc >= 4 );    //整个代码都是创建一个DBQuery对象，并未进行任何的查询请求动作            Convertor c(cx);            c.setProperty( obj , "_mongo" , argv[0] );            c.setProperty( obj , "_db" , argv[1] );            c.setProperty( obj , "_collection" , argv[2] );            c.setProperty( obj , "_ns" , argv[3] );            if ( argc > 4 && JSVAL_IS_OBJECT( argv[4] ) )                c.setProperty( obj , "_query" , argv[4] );            else {                JSObject * temp = JS_NewObject( cx , 0 , 0 , 0 );                CHECKNEWOBJECT( temp, cx, "dbquery_constructor" );                c.setProperty( obj , "_query" , OBJECT_TO_JSVAL( temp ) );            }            if ( argc > 5 && JSVAL_IS_OBJECT( argv[5] ) )                c.setProperty( obj , "_fields" , argv[5] );            else                c.setProperty( obj , "_fields" , JSVAL_NULL );            if ( argc > 6 && JSVAL_IS_NUMBER( argv[6] ) )                c.setProperty( obj , "_limit" , argv[6] );            else                c.setProperty( obj , "_limit" , JSVAL_ZERO );            if ( argc > 7 && JSVAL_IS_NUMBER( argv[7] ) )                c.setProperty( obj , "_skip" , argv[7] );            else                c.setProperty( obj , "_skip" , JSVAL_ZERO );            if ( argc > 8 && JSVAL_IS_NUMBER( argv[8] ) )                c.setProperty( obj , "_batchSize" , argv[8] );            else                c.setProperty( obj , "_batchSize" , JSVAL_ZERO );            if ( argc > 9 && JSVAL_IS_NUMBER( argv[9] ) )                c.setProperty( obj , "_options" , argv[9] );            else                c.setProperty( obj , "_options" , JSVAL_ZERO );            c.setProperty( obj , "_cursor" , JSVAL_NULL );            c.setProperty( obj , "_numReturned" , JSVAL_ZERO );            c.setProperty( obj , "_special" , JSVAL_FALSE );        }        catch ( const AssertionException& e ) {            if ( ! JS_IsExceptionPending( cx ) ) {                JS_ReportError( cx, e.what() );            }            return JS_FALSE;        }        catch ( const std::exception& e ) {            log() << "unhandled exception: " << e.what() << ", throwing Fatal Assertion" << endl;            fassertFailed( 16323 );        }        return JS_TRUE;    }

可以看到上面只有DBQuery对象的构建动作,并没有真正的查询请求，那么查询请求去哪里了呢?回到

                try {                    if ( scope->exec( code.c_str() , "(shell)" , false , true , false ) )//执行相应的javascript代码                        scope->exec( "shellPrintHelper( __lastres__ );" , "(shell2)" , true , true , false );                }

继续分析shellPrintHelper函数，这里__lastres__是什么，搜索整个source insight工程发现mongo/scripting/engine_spidermonkey.cpp中:

        bool exec( const StringData& code,const string& name = "(anon)",bool printResult = false,bool reportError = true, bool assertOnError = true,int timeoutMs = 0 ) {            JSBool worked = JS_EvaluateScript( _context,                                               _global,                                               code.data(),                                               code.size(),                                               name.c_str(),                                               1,                                               &ret );            if ( worked )                _convertor->setProperty( _global , "__lastres__" , ret );        }

原来__lastres__就是上一条执行语句的结果也就是这里的DBQuery对象.继续分析shellPrintHelper函数(mongo/util/util.js)

shellPrintHelper = function (x) {    if (typeof (x) == "undefined") {        // Make sure that we have a db var before we use it        // TODO: This implicit calling of GLE can cause subtle, hard to track issues - remove?        if (__callLastError && typeof( db ) != "undefined" && db.getMongo ) {            __callLastError = false;            // explicit w:1 so that replset getLastErrorDefaults aren't used here which would be bad.            var err = db.getLastError(1);            if (err != null) {                print(err);            }        }        return;    }    if (x == __magicNoPrint)        return;    if (x == null) {        print("null");        return;    }    if (typeof x != "object")        return print(x);    var p = x.shellPrint;//我们这里是DBQuery对象，所以执行到这里，来到了DBQuery.shellPrint函数    if (typeof p == "function")        return x.shellPrint();    var p = x.tojson;    if (typeof p == "function")        print(x.tojson());    else        print(tojson(x));}

DBQuery.prototype.shellPrint = function(){//(mongo/util/query.js)    try {        var start = new Date().getTime();        var n = 0;//还有查询结果并且输出数目小于shellBatchSize,循环打印结果        while ( this.hasNext() && n < DBQuery.shellBatchSize ){//这里shellBatchSize定义为20            var s = this._prettyShell ? tojson( this.next() ) : tojson( this.next() , "" , true );            print( s );//调用native函数native_print打印结果            n++;        }        if (typeof _verboseShell !== 'undefined' && _verboseShell) {            var time = new Date().getTime() - start;            print("Fetched " + n + " record(s) in " + time + "ms");        }         if ( this.hasNext() ){            print( "Type \"it\" for more" );            ___it___  = this;        }        else {            ___it___  = null;        }   }    catch ( e ){        print( e );    }    }

继续看看hasNext函数和next函数:

DBQuery.prototype.hasNext = function(){    this._exec();    if ( this._limit > 0 && this._cursorSeen >= this._limit )//超过了限制返回false，将不会再输出结果        return false;    var o = this._cursor.hasNext();    return o;}DBQuery.prototype.next = function(){    this._exec();        var o = this._cursor.hasNext();    if ( o )        this._cursorSeen++;    else        throw "error hasNext: " + o;        var ret = this._cursor.next();    if ( ret.$err && this._numReturned == 0 && ! this.hasNext() )        throw "error: " + tojson( ret );    this._numReturned++;    return ret;}

继续前进到_exec函数:

DBQuery.prototype._exec = function(){//到这里终于到了this._mongo.find    if ( ! this._cursor ){        assert.eq( 0 , this._numReturned );        this._cursor = this._mongo.find( this._ns , this._query , this._fields , this._limit , this._skip , this._batchSize , this._options );        this._cursorSeen = 0;    }    return this._cursor;}

到这里我们来到了this._mongo.find,这里_mongo是一个Mongo对象，在上一篇文章中我们了解到find函数是本地函数mongo_find.继续分析

mongo_find(mongo/scripting/sm_db.cpp).这里删除了部分错误处理代码.

    JSBool mongo_find(JSContext *cx, JSObject *obj, uintN argc, jsval *argv, jsval *rval) {            shared_ptr< DBClientWithCommands > * connHolder = (shared_ptr< DBClientWithCommands >*)JS_GetPrivate( cx , obj );            smuassert( cx ,  "no connection!" , connHolder && connHolder->get() );            DBClientWithCommands *conn = connHolder->get();            Convertor c( cx );            string ns = c.toString( argv[0] );            BSONObj q = c.toObject( argv[1] );            BSONObj f = c.toObject( argv[2] );            int nToReturn = (int) c.toNumber( argv[3] );            int nToSkip = (int) c.toNumber( argv[4] );            int batchSize = (int) c.toNumber( argv[5] );            int options = (int)c.toNumber( argv[6] );//上面一篇文章我们分析到这里的conn其实是由ConnectionString::connect函数返回的，其返回的对象指针可能是:DBClientConnection对应Master,也就是只设置了一个地址，DBClientReplicaSet对应pair或者set模式,SyncClusterConnection对应sync模式，继续分析流程我们选择最简单的Master模式,只有一个地址的服务端            auto_ptr<DBClientCursor> cursor = conn->query( ns , q , nToReturn , nToSkip , f.nFields() ? &f : 0  , options , batchSize );            if ( ! cursor.get() ) {                log() << "query failed : " << ns << " " << q << " to: " << conn->toString() << endl;                JS_ReportError( cx , "error doing query: failed" );                return JS_FALSE;            }            JSObject * mycursor = JS_NewObject( cx , &internal_cursor_class , 0 , 0 );            CHECKNEWOBJECT( mycursor, cx, "internal_cursor_class" );            verify( JS_SetPrivate( cx , mycursor , new CursorHolder( cursor, *connHolder ) ) );            *rval = OBJECT_TO_JSVAL( mycursor );        return JS_TRUE;    }

那么我们继续前进来到DBClientConnection::query函数，该函数只是简单调用了DBClientBase::query函数.

    auto_ptr<DBClientCursor> DBClientBase::query(const string &ns, Query query, int nToReturn,            int nToSkip, const BSONObj *fieldsToReturn, int queryOptions , int batchSize ) {        auto_ptr<DBClientCursor> c( new DBClientCursor( this,//根据传入的参数创建一个DBClientCursor对象                                    ns, query.obj, nToReturn, nToSkip,                                    fieldsToReturn, queryOptions , batchSize ) );        if ( c->init() )//创建Message并向服务端发送查询请求            return c;        return auto_ptr< DBClientCursor >( 0 );    }

    bool DBClientCursor::init() {        Message toSend;        _assembleInit( toSend );//构建将要发送的查询请求这是一个Message,具体来说Message负责发送数据,具体的数据是在MsgData中        verify( _client );        if ( !_client->call( toSend, *batch.m, false, &_originalHost ) ) {//实际的发送数据,同时这里发送了数据后会调用recv接收数据            // log msg temp?                                              //接收的数据同样是MsgData，同样由Message来管理            log() << "DBClientCursor::init call() failed" << endl;            return false;        }        if ( batch.m->empty() ) {            // log msg temp?            log() << "DBClientCursor::init message from call() was empty" << endl;            return false;        }        dataReceived();//根据上面的batch.m收到的数据得出查询是否成功成功则设置cursorId,下一次请求时operation就变动为dbGetmore了.        return true;   //查询错误则抛出异常    }

_assembleInit是创建一个message结构，若是第一次请求那么请求操作为dbQuery,若不是则请求操作为dbGetmore.来看看MsgData的具体结构吧.

回到mongo_find函数.

            auto_ptr<DBClientCursor> cursor = conn->query( ns , q , nToReturn , nToSkip , f.nFields() ? &f : 0  , options , batchSize );            if ( ! cursor.get() ) {//这里得到了cursor                log() << "query failed : " << ns << " " << q << " to: " << conn->toString() << endl;                JS_ReportError( cx , "error doing query: failed" );                return JS_FALSE;            }            JSObject * mycursor = JS_NewObject( cx , &internal_cursor_class , 0 , 0 );//将cursor封装成一个javascript对象,javascript就能            CHECKNEWOBJECT( mycursor, cx, "internal_cursor_class" );//使用游标了            verify( JS_SetPrivate( cx , mycursor , new CursorHolder( cursor, *connHolder ) ) );

我们回到javascript部分的打印请求.

DBQuery.prototype.hasNext = function(){    this._exec();    if ( this._limit > 0 && this._cursorSeen >= this._limit )        return false;    var o = this._cursor.hasNext();//这里的cursor对应于上面的C++对象cursor,其hasNext函数对应的native函数为internal_cursor_hasNext,    return o;                      //这是在mongo启动时初始化javascript环境时绑定的}

继续看到了internal_cursor_hasNext:

    JSBool internal_cursor_hasNext(JSContext *cx, JSObject *obj, uintN argc, jsval *argv, jsval *rval) {        try {            DBClientCursor *cursor = getCursor( cx, obj );            *rval = cursor->more() ? JSVAL_TRUE : JSVAL_FALSE;//这里返回的就是是否还有数据,如果本地没有查询数据了,那么其会再构建一个        }                                                     //dbGetmore的请求向服务器请求更多数据,还是没有则返回false,表示没有数据了        catch ( const AssertionException& e ) {            if ( ! JS_IsExceptionPending( cx ) ) {                JS_ReportError( cx, e.what() );            }            return JS_FALSE;        }        catch ( const std::exception& e ) {            log() << "unhandled exception: " << e.what() << ", throwing Fatal Assertion" << endl;            fassertFailed( 16290 );        }        return JS_TRUE;    }

最后javascript部分打印数据:

DBQuery.prototype.shellPrint = function(){    try {        var start = new Date().getTime();        var n = 0;        while ( this.hasNext() && n < DBQuery.shellBatchSize ){//前面分析这里hasNext对应C++函数internal_cursor_hasNext,next对应            var s = this._prettyShell ? tojson( this.next() ) : tojson( this.next() , "" , true );//internal_cursor_next,怎么得到的数据不再分析            print( s );//调用native_print打印结果            n++;        }

下面来看看类似这种形式的命令:db.coll.find().count(), db.coll.find().showDiskLoc().

DBQuery.prototype.sort = function( sortBy ){//可以看到,这里只是在查询时增加了相应的查询请求而已    return this._addSpecial( "orderby" , sortBy );}DBQuery.prototype.hint = function( hint ){    return this._addSpecial( "$hint" , hint );}DBQuery.prototype.min = function( min ) {    return this._addSpecial( "$min" , min );}DBQuery.prototype.max = function( max ) {    return this._addSpecial( "$max" , max );}DBQuery.prototype.showDiskLoc = function() {    return this._addSpecial( "$showDiskLoc" , true);}

DBQuery.prototype.count = function( applySkipLimit ){//而这里count这种是变更了查询的,这里是向服务器发送了一个count命令,执行的并不是    var cmd = { count: this._collection.getName() };//查询请求    if ( this._query ){        if ( this._special )            cmd.query = this._query.query;        else             cmd.query = this._query;    }    cmd.fields = this._fields || {};    if ( applySkipLimit ){        if ( this._limit )            cmd.limit = this._limit;        if ( this._skip )            cmd.skip = this._skip;    }        var res = this._db.runCommand( cmd );    if( res && res.n != null ) return res.n;    throw "count failed: " + tojson( res );}DBQuery.prototype.size = function(){    return this.count( true );}

        总结,本文分析了mongodb自带的mongo客户端发起的查询请求以及结果的打印过程，

总体来说流程很简单,唯一令人感到麻烦的就是代码的执行流程在javascript和C++中来回的切换,

需要注意的就是对于mongo的查询请求的格式MsgData.

原文链接:http://blog.csdn.net/yhjj0108/article/details/8253349

作者:yhjj0108,杨浩