Mongoose源码分析

Mongoose是一个简易的web服务器，所谓web服务器，简单的说就是把服务断的数据返回给客户端。

的源码很简单，主要就是Mongoose.c文件，里面包含了大部分的处理。

Mongoose里面有几个比较重要点的数据结构：

1、mg_context详解mg_context结构体——表示Mongoose的上下文，也称为一个实例句柄。它的成员如下：struct mg_context {intstop_flag;/* Should we stop event loop*/SSL_CTX*ssl_ctx;/* SSL context*/FILE*access_log;/* Opened access log*/FILE*error_log;/* Opened error log*/struct socketlisteners[MAX_LISTENING_SOCKETS];intnum_listeners;struct callbackcallbacks[MAX_CALLBACKS];intnum_callbacks;char*options[NUM_OPTIONS];/* Configured opions*/pthread_mutex_topt_mutex[NUM_OPTIONS];/* Option protector*/intmax_threads;/* Maximum number of threads*/intnum_threads;/* Number of threads*/intnum_idle;/* Number of idle threads*/pthread_mutex_tthr_mutex;/* Protects (max|num)_threads*/pthread_cond_tthr_cond;pthread_mutex_tbind_mutex;/* Protects bind operations*/struct socketqueue[20];/* Accepted sockets*/intsq_head;/* Head of the socket queue*/intsq_tail;/* Tail of the socket queue*/pthread_cond_tempty_cond;/* Socket queue empty condvar*/pthread_cond_tfull_cond;/* Socket queue full condvar*/mg_spcb_tssl_password_callback;mg_callback_tlog_callback;};这个结构体在mg_start()中创建和初始化，其它函数大部分都会用它。因此mg_start()应该首先被调用。它非常重要，几乎所有的函数都要用到它。1)、stop_flag表示是否应该停止的标记，它有三个可能的值0、1、2。 stop_flag=0表示 不应该停止，这是初始值；stop_flag=1表示停止，在mg_stop()函数中的一开始设置stop_flag=1，这会触发mg_fini()，且在mg_stop()中会一直等待mg_fini执行完成；stop_flag=2用于通知mg_stop()，mg_fini已经执行完成，stop_flag=2在mg_fini函数中的末尾设置。2)、ssl_ctx是结构体ssl_ctx_st的实例，它来自OpenSSL开源项目，作者把它放到这里的原因是使其独立于OpenSSL的源码安装，这样只有系统上面安装有SSL库，mongoose+SSL就能编译通过。3)、access_log、error_log很明显是指向访问日志文件、错误日志文件。4)、listeners数组存储mongoose建立的多个web server，每个web server都是listeners数组中的一个元素。例如，一个服务器可以分别在端口8080、8888建立web server，这样8080端口的那个server是listerns数组中的一个元素，8888端口的那个server也是listeners数组中的一个元素。换句话说，listeners数组表示web server的socket地址。num_listeners表示listeners数组的元素个数。5)、callbacks是结构体callback的数组，而callback本身是一个结构体，包含几个回调句柄。num_callbacks是callbacks数组元素的个数。6)、options数组，是用于存储配置选项的，例如端口号、工作目录等等。opt_mutext对配置进行操作的互斥变量。7)、max_threads表示允许的最大线程数量、num_threads表示当前的线程数量、num_idle表示空闲的线程数量。之所以会有空闲进程，是因为当创建一个线程处理连接请求之后，它会保持一段时间空闲而不是直接销毁。如果这里再用新的连接到来或等待队列中有需要处理的连接，空闲进程会被分配去处理。8)、thr_mutex、thr_cond、bind_mutex是用于互斥信号量和条件变量。9)、queue[20]队列数组存储client的连接请求，每个元素都是client的socket。sq_head、sq_tail分别是队列头、尾用于操作队列queue。empty_cond、full_cond分别表示队列是否为空、满的条件变量。10)、ssl_password_callback和log_callback都是函数指针，分别指向SSL密码处理函数、log处理函数。他们原型是：/* * Register SSL password handler. * This is needed only if SSL certificate asks for a password. Instead of * prompting for a password on a console a specified function will be called. */typedef int (*mg_spcb_t)(char *buf, int num, int w, void *key);/* * User-defined callback function prototype for URI handling, error handling, * or logging server messages. */typedef void (*mg_callback_t)(struct mg_connection *,const struct mg_request_info *info, void *user_data

2、mg_connection详解故名思意，这个结构体用户保存client的连接信息。它的成员如下：/* * Client connection. */struct mg_connection {struct mg_request_inforequest_info;struct mg_context *ctx;/* Mongoose context we belong to*/SSL*ssl;/* SSL descriptor*/struct socketclient;/* Connected client*/time_tbirth_time;/* Time connection was accepted*/bool_tfree_post_data;/* post_data was malloc-ed*/bool_tembedded_auth;/* Used for authorization*/uint64_tnum_bytes_sent;/* Total bytes sent to client*/};上面的字段意思都很明显这里就不一一阐述了。可以看出， 每个连接都保存了一个Mongoose上下文（mg_context * ctx），这个很重要，对连接请求进行处理时都会用到。这里也可以看出mg_context相当于一个实例句柄。结构体mg_request_info用于保存每个请求的信息，例如，当我打开博客主页http://www.cnblogs.com/skynet/的时候，会发出一个请求信息，包括请求的方法是POST还是GET等、uri即http://www.cnblogs.com/skynet/、http版本、还有一些http头信息等等。关于结构体mg_request_info的详细信息参见下一小节。

主要就是上面的两个数据结构了

下面看下如何使用它。

首先我们需要启动http服务，这是通过调用mg_start来实现的

struct mg_context *mg_start(mg_callback_t user_callback, void *user_data,const char **options) {

这个函数包括一个mg_callback_t类型的回调，user_data可以为NULL,option是一引起需要设置的选项，没有的话会使用里面默认的，如端口默认是8080等。

  

struct mg_context *mg_start(mg_callback_t user_callback, void *user_data,const char **options) {struct mg_context *ctx;const char *name, *value, *default_value;int i;__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "hp mg_start 1111");#if defined(_WIN32) && !defined(__SYMBIAN32__)WSADATA data;WSAStartup(MAKEWORD(2,2), &data);#endif // _WIN32// Allocate context and initialize reasonable general case defaults.// TODO(lsm): do proper error handling here.ctx = (struct mg_context *) calloc(1, sizeof(*ctx));ctx->user_callback = user_callback;  //保存回调ctx->user_data = user_data;__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "hp mg_start 2222");while (options && (name = *options++) != NULL) {  //解析optionif ((i = get_option_index(name)) == -1) {__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "Invalid option: %s", name);free_context(ctx);return NULL;} else if ((value = *options++) == NULL) {__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "%s: option value cannot be NULL", name);free_context(ctx);return NULL;}ctx->config[i] = mg_strdup(value);__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "config:[%s] -> [%s]", name, value);}__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "hp mg_start 3333");// Set default value if neededfor (i = 0; config_options[i * ENTRIES_PER_CONFIG_OPTION] != NULL; i++) {default_value = config_options[i * ENTRIES_PER_CONFIG_OPTION + 2];if (ctx->config[i] == NULL && default_value != NULL) {ctx->config[i] = mg_strdup(default_value);DEBUG_TRACE(("Setting default: [%s] -> [%s]",config_options[i * ENTRIES_PER_CONFIG_OPTION + 1],default_value));}}__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "hp mg_start 4444");// NOTE(lsm): order is important here. SSL certificates must// be initialized before listening ports. UID must be set last.int a = !set_gpass_option(ctx);int b = !set_ssl_option(ctx);int c = !set_ports_option(ctx);//设置监听端口int d = !set_uid_option(ctx);int e = !set_acl_option(ctx);__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "hp mg_start set_gpass_option(ctx):%d",a);__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "hp mg_start set_ports_option(ctx):%d",b);__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "hp mg_start set_acl_option(ctx):%d",c);__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "hp mg_start set_uid_option(ctx):%d",d);__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "hp mg_start set_ssl_option(ctx):%d",e);if (a || b || c || d || e) {__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "hp mg_start set.......");free_context(ctx);return NULL;}__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "hp mg_start 5555");#if !defined(_WIN32) && !defined(__SYMBIAN32__)// Ignore SIGPIPE signal, so if browser cancels the request, it// won't kill the whole process.(void) signal(SIGPIPE, SIG_IGN);#endif // !_WIN32(void) pthread_mutex_init(&ctx->mutex, NULL);(void) pthread_cond_init(&ctx->cond, NULL);(void) pthread_cond_init(&ctx->sq_empty, NULL);(void) pthread_cond_init(&ctx->sq_full, NULL);__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "hp mg_start 6666");// Start master (listening) threadstart_thread(ctx, (mg_thread_func_t) master_thread, ctx);//启动主线程,处理到来的连接__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "hp mg_start 7777");__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "atoi(ctx->config[NUM_THREADS]) = %d.", atoi(ctx->config[NUM_THREADS]));// Start worker threadsfor (i = 0; i < atoi(ctx->config[NUM_THREADS]); i++) {if (start_thread(ctx, (mg_thread_func_t) worker_thread, ctx) != 0) {//启动工作线程，具体连接处理cry(fc(ctx), "Cannot start worker thread: %d", ERRNO);} else {ctx->num_threads++;}}__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "hp mg_start 8888");return ctx;}

mg_start主要是进行一些初始化操作，然后等待启动一个线程等待客户端连接的到来。再启动了一定数量的工作线程进行具体的处理

首先来看一下主线程：

static void master_thread(struct mg_context *ctx) {fd_set read_set;struct timeval tv;struct socket *sp;int max_fd;__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "master_thread");while (ctx->stop_flag == 0) {FD_ZERO(&read_set);max_fd = -1;// Add listening sockets to the read setfor (sp = ctx->listening_sockets; sp != NULL; sp = sp->next) {add_to_set(sp->sock, &read_set, &max_fd);}tv.tv_sec = 0;tv.tv_usec = 200 * 1000;__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "before  select");if (select(max_fd + 1, &read_set, NULL, NULL, &tv) < 0) { //是否有连接到来#ifdef _WIN32// On windows, if read_set and write_set are empty,// select() returns "Invalid parameter" error// (at least on my Windows XP Pro). So in this case, we sleep here.sleep(1);#endif // _WIN32} else {for (sp = ctx->listening_sockets; sp != NULL; sp = sp->next) {if (FD_ISSET(sp->sock, &read_set)) {accept_new_connection(sp, ctx);//处理到来的连接}}}}DEBUG_TRACE(("stopping workers"));// Stop signal received: somebody called mg_stop. Quit.close_all_listening_sockets(ctx);// Wakeup workers that are waiting for connections to handle.pthread_cond_broadcast(&ctx->sq_full);// Wait until all threads finish(void) pthread_mutex_lock(&ctx->mutex);while (ctx->num_threads > 0) {(void) pthread_cond_wait(&ctx->cond, &ctx->mutex);}(void) pthread_mutex_unlock(&ctx->mutex);// All threads exited, no sync is needed. Destroy mutex and condvars(void) pthread_mutex_destroy(&ctx->mutex);(void) pthread_cond_destroy(&ctx->cond);(void) pthread_cond_destroy(&ctx->sq_empty);(void) pthread_cond_destroy(&ctx->sq_full);// Signal mg_stop() that we're donectx->stop_flag = 2;DEBUG_TRACE(("exiting"));}

这里主要是有连接到来的时候调用accept_new_connection把它加入到一个队列里面去。后面的代码就不跟踪了

再来年下worker_thread

[

static void worker_thread(struct mg_context *ctx) {struct mg_connection *conn;int buf_size = atoi(ctx->config[MAX_REQUEST_SIZE]);conn = (struct mg_connection *) calloc(1, sizeof(*conn) + buf_size);conn->buf_size = buf_size;conn->buf = (char *) (conn + 1);assert(conn != NULL);__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "worker_thread");while (ctx->stop_flag == 0 && consume_socket(ctx, &conn->client)) {//从队列里面去取一个client socketconn->birth_time = time(NULL);conn->ctx = ctx;__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "into while");// Fill in IP, port info early so even if SSL setup below fails,// error handler would have the corresponding info.// Thanks to Johannes Winkelmann for the patch.conn->request_info.remote_port = ntohs(conn->client.rsa.u.sin.sin_port);memcpy(&conn->request_info.remote_ip,&conn->client.rsa.u.sin.sin_addr.s_addr, 4);conn->request_info.remote_ip = ntohl(conn->request_info.remote_ip);conn->request_info.is_ssl = conn->client.is_ssl;if (!conn->client.is_ssl || (conn->client.is_ssl && sslize(conn,SSL_accept))) {process_new_connection(conn);//进行具体的处理}close_connection(conn);}free(conn);// Signal master that we're done with connection and exiting(void) pthread_mutex_lock(&ctx->mutex);ctx->num_threads--;(void) pthread_cond_signal(&ctx->cond);assert(ctx->num_threads >= 0);(void) pthread_mutex_unlock(&ctx->mutex);DEBUG_TRACE(("exiting"));}

具体的处理是通过process_new_connection进行的

static void process_new_connection(struct mg_connection *conn) {struct mg_request_info *ri = &conn->request_info;int keep_alive_enabled;const char *cl;__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "process_new_connection");keep_alive_enabled = !strcmp(conn->ctx->config[ENABLE_KEEP_ALIVE], "yes");do {reset_per_request_attributes(conn);// If next request is not pipelined, read it inif ((conn->request_len = get_request_len(conn->buf, conn->data_len))== 0) {__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "conn->request_len = %d.", conn->request_len);conn->request_len = read_request(NULL, conn->client.sock,//这里进行client数据的读取conn->ssl, conn->buf, conn->buf_size, &conn->data_len);}assert(conn->data_len >= conn->request_len);if (conn->request_len == 0 && conn->data_len == conn->buf_size) {send_http_error(conn, 413, "Request Too Large", "");return;}if (conn->request_len <= 0) {return; // Remote end closed the connection}// Nul-terminate the request cause parse_http_request() uses sscanfconn->buf[conn->request_len - 1] = '\0';if (!parse_http_request(conn->buf, ri) || (!conn->client.is_proxy//请求解析&& !is_valid_uri(ri->uri))) {// Do not put garbage in the access log, just send it back to the clientsend_http_error(conn, 400, "Bad Request","Cannot parse HTTP request: [%.*s]", conn->data_len,conn->buf);} else if (strcmp(ri->http_version, "1.0") && strcmp(ri->http_version,"1.1")) {// Request seems valid, but HTTP version is strangesend_http_error(conn, 505, "HTTP version not supported", "");log_access(conn);} else {// Request is valid, handle itcl = get_header(ri, "Content-Length");conn->content_len = cl == NULL ? -1 : strtoll(cl, NULL, 10);conn->birth_time = time(NULL);if (conn->client.is_proxy) {handle_proxy_request(conn);} else {handle_request(conn);//处理请求}log_access(conn);discard_current_request_from_buffer(conn);}// conn->peer is not NULL only for SSL-ed proxy connections} while (conn->peer || (keep_alive_enabled && should_keep_alive(conn)));}

这里面的读取请求是通过read_request，里面再调用pull进行数据的读取

static int pull(FILE *fp, SOCKET sock, SSL *ssl, char *buf, int len) {int nread;if (ssl != NULL) {nread = SSL_read(ssl, buf, len);} else if (fp != NULL) {// Use read() instead of fread(), because if we're reading from the CGI// pipe, fread() may block until IO buffer is filled up. We cannot afford// to block and must pass all read bytes immediately to the client.nread = read(fileno(fp), buf, (size_t) len);if (ferror(fp))nread = -1;} else {__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "sock = %d", sock);nread = recv(sock, buf, (size_t) len, 0);__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "buf = %s", buf);}return nread;}

// This is the heart of the Mongoose's logic.// This function is called when the request is read, parsed and validated,// and Mongoose must decide what action to take: serve a file, or// a directory, or call embedded function, etcetera.static void handle_request(struct mg_connection *conn) {struct mg_request_info *ri = &conn->request_info;char path[PATH_MAX];int uri_len;struct mgstat st;__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "handle_request");if ((conn->request_info.query_string = strchr(ri->uri, '?')) != NULL) {*conn->request_info.query_string++ = '\0';}__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "ri->uri = %s.", ri->uri);uri_len = strlen(ri->uri);(void) url_decode(ri->uri, (size_t) uri_len, ri->uri,(size_t) (uri_len + 1), 0);remove_double_dots_and_double_slashes(ri->uri);convert_uri_to_file_name(conn, ri->uri, path, sizeof(path));__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "ri->request_method = %s.", ri->request_method);DEBUG_TRACE(("%s", ri->uri));if (!check_authorization(conn, path)) {send_authorization_request(conn);} else if (call_user(conn, MG_NEW_REQUEST) != NULL) {//这里调用用户注册的函数__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "call_user  .MG_NEW_REQUEST");// Do nothing, callback has served the request} else if (strstr(path, PASSWORDS_FILE_NAME)) {// Do not allow to view passwords filessend_http_error(conn, 403, "Forbidden", "Access Forbidden");} else if (conn->ctx->config[DOCUMENT_ROOT] == NULL) {send_http_error(conn, 404, "Not Found", "Not Found");} else if ((!strcmp(ri->request_method, "PUT") || !strcmp(ri->request_method, "DELETE"))&& (conn->ctx->config[PUT_DELETE_PASSWORDS_FILE] == NULL|| !is_authorized_for_put(conn))) {send_authorization_request(conn);} else if (!strcmp(ri->request_method, "PUT")) {put_file(conn, path);} else if (!strcmp(ri->request_method, "DELETE")) {if (mg_remove(path) == 0) {send_http_error(conn, 200, "OK", "");} else {send_http_error(conn, 500, http_500_error, "remove(%s): %s", path,strerror(ERRNO));}} else if (mg_stat(path, &st) != 0) {send_http_error(conn, 404, "Not Found", "%s", "File not found");} else if (st.is_directory && ri->uri[uri_len - 1] != '/') {(void) mg_printf(conn, "HTTP/1.1 301 Moved Permanently\r\n""Location: %s/\r\n\r\n", ri->uri);} else if (st.is_directory && !substitute_index_file(conn, path,sizeof(path), &st)) {if (!mg_strcasecmp(conn->ctx->config[ENABLE_DIRECTORY_LISTING], "yes")) {handle_directory_request(conn, path);} else {send_http_error(conn, 403, "Directory Listing Denied","Directory listing denied");}} else if (match_extension(path, conn->ctx->config[CGI_EXTENSIONS])) {if (strcmp(ri->request_method, "POST") && strcmp(ri->request_method,"GET")) {send_http_error(conn, 501, "Not Implemented","Method %s is not implemented", ri->request_method);} else {handle_cgi_request(conn, path);}} else if (match_extension(path, conn->ctx->config[SSI_EXTENSIONS])) {handle_ssi_file_request(conn, path);} else if (is_not_modified(conn, &st)) {send_http_error(conn, 304, "Not Modified", "");} else {__android_log_print(ANDROID_LOG_DEBUG, "mongoose", "handle_file_request");handle_file_request(conn, path, &st);}}

再来看一下请求解析的函数：

 

static int parse_http_request(char *buf, struct mg_request_info *ri) {int status = 0;// RFC says that all initial whitespaces should be ingoredwhile (*buf != '\0' && isspace(* (unsigned char *) buf)) {buf++;}ri->request_method = skip(&buf, " ");ri->uri = skip(&buf, " ");ri->http_version = skip(&buf, "\r\n");if (is_valid_http_method(ri->request_method) && strncmp(ri->http_version,"HTTP/", 5) == 0) {ri->http_version += 5; /* Skip "HTTP/" */parse_http_headers(&buf, ri);status = 1;}return status;}

它的主要工作就是从buf中提取出信息放到ri（一个mg_request_info结构）中去，因为buf是一个无结构的字符串数组。要将它存储到ri中去，需要找到对应的子串。
这里主要用到了skip()、parse_http_headers()方法，其中skip()很关键

当我们要发送数据给client端时，可以通过mg_write函数来实现，这个函数可以在回调函数里面去调用。

int mg_write(struct mg_connection *conn, const void *buf, size_t len) {return (int) push(NULL, conn->client.sock, conn->ssl, (const char *) buf,(int64_t) len);}

// Write data to the IO channel - opened file descriptor, socket or SSL// descriptor. Return number of bytes written.static int64_t push(FILE *fp, SOCKET sock, SSL *ssl, const char *buf,int64_t len) {int64_t sent;int n, k;sent = 0;while (sent < len) {/* How many bytes we send in this iteration */k = len - sent > INT_MAX ? INT_MAX : (int) (len - sent);if (ssl != NULL) {n = SSL_write(ssl, buf + sent, k);} else if (fp != NULL) {n = fwrite(buf + sent, 1, (size_t) k, fp);if (ferror(fp))n = -1;} else {n = send(sock, buf + sent, (size_t) k, 0);}if (n < 0)break;if(n == 0){usleep(1000);}sent += n;}return sent;}

主要就是send发送数据过去

mongoose的基本流程大概也就这样，其它的就以后需要时再去具体分析吧。

转自：http://blog.csdn.net/new_abc/article/details/7679661