mongod 启动流程

    根据工作需要，开始研究起了mongo。以前用过一些开源，也看过一些开源代码，也看过别人的源码分析，都是大神，甚是羡慕。上一篇中提到的几篇文章写的很好，突然就感觉到其实读代码写记录是最好的。今天从mongo开始，先从简单的来，中间有错误的话，欢迎大家指出，不胜感激。如果有幸被转载，请注明出处，熬夜写博客真心很痛苦。

    先从mongo编译后生成的程序开始吧，我的是2.5.4的源码。刚开始编完后在bin目录中有很多可执行文件，比如mongo/mongos/mongod/mongostat/mongotop等，既然是可执行文件，一定有main函数，搜索之后就可以发现在哪些目录中。研究源码流程也就从main函数开始。

    mongod是mongo的数据库存储程序，位于src/mongo/db/db.cpp。

    主程序代码很简单：

int main(int argc, char* argv[], char** envp) {    int exitCode = mongoDbMain(argc, argv, envp);    ::_exit(exitCode);}

    三个参数的main函数，第三个参数是环境变量，可以循环将其打印出来，是a=b型的键值对，需要继续解析才能使用。

    mongoDbMain函数：

static int mongoDbMain(int argc, char* argv[], char **envp) {    static StaticObserver staticObserver;    getcurns = ourgetns;    setupSignalHandlers();    dbExecCommand = argv[0];    srand(curTimeMicros());    {        unsigned x = 0x12345678;        unsigned char& b = (unsigned char&) x;        if ( b != 0x78 ) {            out() << "big endian cpus not yet supported" << endl;            return 33;        }    }    if( argc == 1 )        cout << dbExecCommand << " --help for help and startup options" << endl;    mongo::runGlobalInitializersOrDie(argc, argv, envp);    startupConfigActions(std::vector<std::string>(argv, argv + argc));    cmdline_utils::censorArgvArray(argc, argv);    if (!initializeServerGlobalState())        ::_exit(EXIT_FAILURE);    // Per SERVER-7434, startSignalProcessingThread() must run after any forks    // (initializeServerGlobalState()) and before creation of any other threads.    startSignalProcessingThread();    dataFileSync.go();#if defined(_WIN32)    if (ntservice::shouldStartService()) {        ntservice::startService();        // exits directly and so never reaches here either.    }#endif    StartupTest::runTests();    initAndListen(serverGlobalParams.port);    dbexit(EXIT_CLEAN);    return 0;}

    staticObserver虽然名字是静态观察，但实现很简单，具体作用还不清楚，大概是头文件中说的那样。

    ourgentns是一个函数，获取当前的ns名，即db.table，会在log中调用。

    setupSignalHandlers是个函数，设置信号的处理，reportEvent和UnhandledException的设置。

    argv[0]是程序名字。    中间一段是测试是否是小端机器，目前不支持大端。

    mongo::runGlobalInitializersOrDie(argc,argv,envp)主要解析命令行，即启动参数，其中会读取配置文件，解析参数并赋值给变量，比如“cpu”，“noauth”，“quota”，“repair path”，“nohints”等。 全局性质的初始化，上面的参数中有需要执行的命令也会在这里执行。在mongo/src/mongo/base/initializer.cpp。

for (size_t i = 0; i < sortedNodes.size(); ++i) {            InitializerFunction fn = _graph.getInitializerFunction(sortedNodes[i]);            if (!fn) {                return Status(ErrorCodes::InternalError,                              "topSort returned a node that has no associated function: \"" +                              sortedNodes[i] + '"');            }            status = fn(&context);            if (Status::OK() != status)                return status;        }

    其中fn就是要执行的函数。

    censor，文档说是一种算法：Look for an equal sign in arg。

    initializeServerGlobalState，一些认证设置。

    startSignalProcessingThread监听异步信号SIGUSR1，默认会创建boost::thread it( signalProcessingThread );

    dataFileSync.go，定时同步文件。    

    runTests运行一些测试程序。

    initAndListen初始化各个模块，并开始监听事件，MessageServer开始运行。

    dbexit数据库退出。

     其中dataFileSync的go函数是class BackgroundJob的函数。这个类是后台线程分发，并执行子类的run方法。boost::bind比较复杂，简单说就是绑定一些参数到函数上。

BackgroundJob& BackgroundJob::go() {        boost::thread t( boost::bind( &BackgroundJob::jobBody , this, _status ) );        return *this;    } 

    然后jobBody会执行run。更多BackgroundJob的信息参考头文件说明。

    今天先到这里，明天继续分析完善。

    现在重点看下initAndListen，这个函数开启了各种监听，包括socket，port，还有检测等。

void initAndListen(int listenPort) {        try {            _initAndListen(listenPort);        }        catch ( DBException &e ) {            log() << "exception in initAndListen: " << e.toString() << ", terminating" << endl;            dbexit( EXIT_UNCAUGHT );        }        catch ( std::exception &e ) {            log() << "exception in initAndListen std::exception: " << e.what() << ", terminating" << endl;            dbexit( EXIT_UNCAUGHT );        }        catch ( int& n ) {            log() << "exception in initAndListen int: " << n << ", terminating" << endl;            dbexit( EXIT_UNCAUGHT );        }        catch(...) {            log() << "exception in initAndListen, terminating" << endl;            dbexit( EXIT_UNCAUGHT );        }    }

    参数是要监听的端口，在这传入的是serverGlobalParams.port,这里port的值是枚举类型的DefaultDBPort=27017.

void _initAndListen(int listenPort ) {        Client::initThread("initandlisten");        bool is32bit = sizeof(int*) == 4;        {            ProcessId pid = ProcessId::getCurrent();            LogstreamBuilder l = log();            l << "MongoDB starting : pid=" << pid              << " port=" << serverGlobalParams.port              << " dbpath=" << storageGlobalParams.dbpath;            if( replSettings.master ) l << " master=" << replSettings.master;            if( replSettings.slave )  l << " slave=" << (int) replSettings.slave;            l << ( is32bit ? " 32" : " 64" ) << "-bit host=" << getHostNameCached() << endl;        }        DEV log() << "_DEBUG build (which is slower)" << endl;        logStartupWarnings();#if defined(_WIN32)        printTargetMinOS();#endif        logProcessDetails();        {            stringstream ss;            ss << endl;            ss << "*********************************************************************" << endl;            ss << " ERROR: dbpath (" << storageGlobalParams.dbpath << ") does not exist." << endl;            ss << " Create this directory or give existing directory in --dbpath." << endl;            ss << " See http://dochub.mongodb.org/core/startingandstoppingmongo" << endl;            ss << "*********************************************************************" << endl;            uassert(10296,  ss.str().c_str(), boost::filesystem::exists(storageGlobalParams.dbpath));        }        {            stringstream ss;            ss << "repairpath (" << storageGlobalParams.repairpath << ") does not exist";            uassert(12590,  ss.str().c_str(),                    boost::filesystem::exists(storageGlobalParams.repairpath));        }        // TODO check non-journal subdirs if using directory-per-db        checkReadAhead(storageGlobalParams.dbpath);        acquirePathLock(mongodGlobalParams.repair);        boost::filesystem::remove_all(storageGlobalParams.dbpath + "/_tmp/");        FileAllocator::get()->start();        // TODO:  This should go into a MONGO_INITIALIZER once we have figured out the correct        // dependencies.        if (snmpInit) {            snmpInit();        }        MONGO_ASSERT_ON_EXCEPTION_WITH_MSG( clearTmpFiles(), "clear tmp files" );        dur::startup();        if (storageGlobalParams.durOptions & StorageGlobalParams::DurRecoverOnly)            return;        unsigned long long missingRepl = checkIfReplMissingFromCommandLine();        if (missingRepl) {            log() << startupWarningsLog;            log() << "** WARNING: mongod started without --replSet yet " << missingRepl                  << " documents are present in local.system.replset" << startupWarningsLog;            log() << "**          Restart with --replSet unless you are doing maintenance and no"                  << " other clients are connected." << startupWarningsLog;            log() << "**          The TTL collection monitor will not start because of this." << startupWarningsLog;            log() << "**          For more info see http://dochub.mongodb.org/core/ttlcollections" << startupWarningsLog;            log() << startupWarningsLog;        }        if (mongodGlobalParams.scriptingEnabled) {            ScriptEngine::setup();            globalScriptEngine->setCheckInterruptCallback( jsInterruptCallback );            globalScriptEngine->setGetCurrentOpIdCallback( jsGetCurrentOpIdCallback );        }        // On replica set members we only clear temp collections on DBs other than "local" during        // promotion to primary. On pure slaves, they are only cleared when the oplog tells them to.        // The local DB is special because it is not replicated.  See SERVER-10927 for more details.        const bool shouldClearNonLocalTmpCollections = !(missingRepl                                                         || replSettings.usingReplSets()                                                         || replSettings.slave == SimpleSlave);        repairDatabasesAndCheckVersion(shouldClearNonLocalTmpCollections);        if (mongodGlobalParams.upgrade)            return;        uassertStatusOK(getGlobalAuthorizationManager()->initialize());        /* this is for security on certain platforms (nonce generation) */        srand((unsigned) (curTimeMicros() ^ startupSrandTimer.micros()));        snapshotThread.go();        d.clientCursorMonitor.go();        PeriodicTask::startRunningPeriodicTasks();        if (missingRepl) {            // a warning was logged earlier        }        else {            startTTLBackgroundJob();        }#ifndef _WIN32        mongo::signalForkSuccess();#endif        if(getGlobalAuthorizationManager()->isAuthEnabled()) {            // open admin db in case we need to use it later. TODO this is not the right way to            // resolve this.            Client::WriteContext c("admin", storageGlobalParams.dbpath);        }        getDeleter()->startWorkers();        // Starts a background thread that rebuilds all incomplete indices.         indexRebuilder.go();         listen(listenPort);        // listen() will return when exit code closes its socket.        exitCleanly(EXIT_NET_ERROR);    }

    Client::initThread用于创建一个数据库操作的cleint。

    ProcessId::getCurrent()获取pid并检测是否32位机器。logStartupWarnings() logProcessDetails()记录一些log信息。

    checkReadAhead()检查dbpath。

    acquirePathLock，如果使用每一个db一个目录，则检查非journal的子目录。

    remove_all删除db目录中的临时目录。

    FileAllocator::get()->start()位于mongo/src/mongo/util/file_allocator.cpp.FileAllocator::get()是单例模式，start会执行run方法。

FileAllocator* FileAllocator::get(){        if ( ! _instance )            _instance = new FileAllocator();        return _instance;    }void FileAllocator::start() {        boost::thread t( boost::bind( &FileAllocator::run , this ) );    }void FileAllocator::run( FileAllocator * fa ) {        setThreadName( "FileAllocator" );        {            // initialize unique temporary file name counter            // TODO: SERVER-6055 -- Unify temporary file name selection            SimpleMutex::scoped_lock lk(_uniqueNumberMutex);            _uniqueNumber = curTimeMicros64();        }        while( 1 ) {            {                scoped_lock lk( fa->_pendingMutex );                if ( fa->_pending.size() == 0 )                    fa->_pendingUpdated.wait( lk.boost() );            }            while( 1 ) {                string name;                long size;                {                    scoped_lock lk( fa->_pendingMutex );                    if ( fa->_pending.size() == 0 )                        break;                    name = fa->_pending.front();                    size = fa->_pendingSize[ name ];                }                string tmp;                long fd = 0;                try {                    log() << "allocating new datafile " << name << ", filling with zeroes..." << endl;                                        boost::filesystem::path parent = ensureParentDirCreated(name);                    tmp = fa->makeTempFileName( parent );                    ensureParentDirCreated(tmp);#if defined(_WIN32)                    fd = _open( tmp.c_str(), _O_RDWR | _O_CREAT | O_NOATIME, _S_IREAD | _S_IWRITE );#else                    fd = open(tmp.c_str(), O_CREAT | O_RDWR | O_NOATIME, S_IRUSR | S_IWUSR);#endif                    if ( fd < 0 ) {                        log() << "FileAllocator: couldn't create " << name << " (" << tmp << ") " << errnoWithDescription() << endl;                        uasserted(10439, "");                    }#if defined(POSIX_FADV_DONTNEED)                    if( posix_fadvise(fd, 0, size, POSIX_FADV_DONTNEED) ) {                        log() << "warning: posix_fadvise fails " << name << " (" << tmp << ") " << errnoWithDescription() << endl;                    }#endif                    Timer t;                    /* make sure the file is the full desired length */                    ensureLength( fd , size );                    close( fd );                    fd = 0;                    if( rename(tmp.c_str(), name.c_str()) ) {                        const string& errStr = errnoWithDescription();                        const string& errMessage = str::stream()                                << "error: couldn't rename " << tmp                                << " to " << name << ' ' << errStr;                        msgasserted(13653, errMessage);                    }                    flushMyDirectory(name);                    log() << "done allocating datafile " << name << ", "                          << "size: " << size/1024/1024 << "MB, "                          << " took " << ((double)t.millis())/1000.0 << " secs"                          << endl;                    // no longer in a failed state. allow new writers.                    fa->_failed = false;                }                catch ( const std::exception& e ) {                    log() << "error: failed to allocate new file: " << name                          << " size: " << size << ' ' << e.what()                          << ".  will try again in 10 seconds" << endl;                    if ( fd > 0 )                        close( fd );                    try {                        if ( ! tmp.empty() )                            boost::filesystem::remove( tmp );                        boost::filesystem::remove( name );                    } catch ( const std::exception& e ) {                        log() << "error removing files: " << e.what() << endl;                    }                    scoped_lock lk( fa->_pendingMutex );                    fa->_failed = true;                    // not erasing from pending                    fa->_pendingUpdated.notify_all();                                                            sleepsecs(10);                    continue;                }                {                    scoped_lock lk( fa->_pendingMutex );                    fa->_pendingSize.erase( name );                    fa->_pending.pop_front();                    fa->_pendingUpdated.notify_all();                }            }        }    }

    在run中，_pending存放的可能是文件的名字，并且创建临时目录。

dur::startup();

    会开启journal线程，创建持久化目录和文件。

if (mongodGlobalParams.scriptingEnabled) {            ScriptEngine::setup();            globalScriptEngine->setCheckInterruptCallback( jsInterruptCallback );            globalScriptEngine->setGetCurrentOpIdCallback( jsGetCurrentOpIdCallback );        }

    ScriptEngine::setup()会创建v8脚本引擎，在mongo/src/mongo/scripting/engine_v8.cpp

snapshotThread.go();

    会开启快照线程，定时拍照，默认4秒。在/mongo/src/mongo/db/stats/snapshots.cpp

d.clientCursorMonitor::go();

    d是全局的DGlobals,获得clientCursorMonitor()时会创建ClientCursorMonitor对象，在mongo/src/mongo/db/clientcursor.cpp，主要检测内存状态，默认一分钟一次。

PeriodicTask::startRunningPerioodicTasks();

    启动定期执行任务的线程，创建PeriodicTaskRunner,检测shutdown信号，没有就等待，默认60秒。

if (missingRepl) {            // a warning was logged earlier        }        else {            startTTLBackgroundJob();        }

    如果没有设置"--replSet"则之前会有警告，如果设置了这里会直行startTTLBackgroundJob();创建ttl监视，淘汰过期数据。

getDeleter()->startWorkers();

    启动deleter线程，绑定RangeDeleter::doWork()，删除给定namespace的文档，可以immediate也可以lazy删除。

listen(listenPort);

    这是监听socket的开始地方。

void listen(int port) {        //testTheDb();        MessageServer::Options options;        options.port = port;        options.ipList = serverGlobalParams.bind_ip;        MessageServer * server = createServer( options , new MyMessageHandler() );        server->setAsTimeTracker();        // we must setupSockets prior to logStartup() to avoid getting too high        // a file descriptor for our calls to select()        server->setupSockets();        logStartup();        startReplication();        if (serverGlobalParams.isHttpInterfaceEnabled)            boost::thread web( boost::bind(&webServerThread, new RestAdminAccess() /* takes ownership */));#if(TESTEXHAUST)        boost::thread thr(testExhaust);#endif        server->run();    }

    这个监听结构也是消息模式的，类似于android的handler机制。createServer()是关键一句。options包含要监听的ip和port，MyMessageHandler是收到消息后处理的类，继承MessageHandler,返回的是新创建的ProtMessageServer的父类，初始化时就给变量赋值。其中Listener是一个抽象出来的监听类，setupSockets()中会创建，设置，添加socket。

    还会启动log和replication。最后server的run开始真正进入了等待循环。用的select监听并处理数据。

run里面的accepted实际执行的是mongo/src/mongo/util/net/message_server_port.cpp中的acceptedMP().

try {#ifndef __linux__  // TODO: consider making this ifdef _WIN32                {                    HandleIncomingMsgParam* himParam = new HandleIncomingMsgParam(p, _handler);                    boost::thread thr(boost::bind(&handleIncomingMsg, himParam));                }.....

    其中创建了HandleIncomingMsgParam，并绑定处理函数handleIncomingMsg和参数himParam。当有消息过来时就可以处理了。

    先到这里吧。