深度理解unix/linux系列中select模型

前一段时间使用IOCP开发一个服务器后，在公网测试效果非常不理想，考虑到后面得项目采用Linux完成，IOCP暂告一段落了。后来在看一些流行得服务器架构中，多数采用select模型，深感不解，经过深入找资料，略有所唔。其效率也非常高。已经夜半了，无所事事，四处瞎逛，翻起一篇老帖，心中泛起无数波澜，得来全不费工夫。
代码下载：

下载文件 (已下载 694 次)

点击这里下载文件: examples.tar.gz

THE WORLD OF SELECT()
So just why am I so hyped on select()?

One traditional way to write network servers is to have the main server block on accept(), waiting for a connection. Once a connection comes in, the server fork()s, the child process handles the connection and the main server is able to service new incoming requests.

With select(), instead of having a process for each request, there is usually only one process that "multi-plexes" all requests, servicing each request as much as it can.

So one main advantage of using select() is that your server will only require a single process to handle all requests. Thus, your server will not need shared memory or synchronization primitives for different 'tasks' to communicate.

One major disadvantage of using select(), is that your server cannot act like there's only one client, like with a fork()'ing solution. For example, with a fork()'ing solution, after the server fork()s, the child process works with the client as if there was only one client in the universe -- the child does not have to worry about new incoming connections or the existence of other sockets. With select(), the programming isn't as transparent.

Okay, so how do you use select()?

select() works by blocking until something happens on a file descriptor (aka a socket). What's 'something'? Data coming in or being able to write to a file descriptor -- you tell select() what you want to be woken up by. How do you tell it? You fill up a fd_set structure with some macros.

Most select()-based servers look pretty much the same:

引用

Fill up a fd_set structure with the file descriptors you want to know when data comes in on.
Fill up a fd_set structure with the file descriptors you want to know when you can write on.
Call select() and block until something happens.
Once select() returns, check to see if any of your file descriptors was the reason you woke up. If so, 'service' that file descriptor in whatever particular way your server needs to (i.e. read in a request for a Web page).
Repeat this process forever.

Quit with the pseudo-code, show me some real code!

Okay, let's take a look at a sample server included with Vic Metcalfe's Socket Programming FAQ (my comments are in red):

view plaincopy to clipboardprint?

1. /*   
2.    PLEASE READ THE FILE NB-APOLOGY!!!!  There are some things you should  
3.    know about this source before you read it.  Thanks.  
4.  
5.      
6.    Quang Ngo alerted me to a bug where the variable listnum in deal_with_data()  
7.    wasn't being passed in by parameter, thus it was always garbage. I have  
8.    quick-fixed this in the code below. - Spencer (October 12, 1999)  
9.      
10.  
11.    Non blocking server demo   
12.    By Vic Metcalfe ([e-mail])  
13.    For the unix-socket-faq  
14. */  
15.   
16. #include "sockhelp.h"   
17. #include <ctype.h>   
18. #include <sys/time.h>   
19. #include <fcntl.h>   
20.   
21. int sock;            /* The socket file descriptor for our "listening"  
22.                      socket */  
23. int connectlist[5];  /* Array of connected sockets so we know who  
24.        we are talking to */  
25. fd_set socks;        /* Socket file descriptors we want to wake  
26.       up for, using select() */  
27. int highsock;       /* Highest #'d file descriptor, needed for select() */  
28.   
29. void setnonblocking(sock)   
30. int sock;   
31. {   
32.   int opts;   
33.   
34.   opts = fcntl(sock,F_GETFL);   
35.   if (opts < 0) {   
36.     perror("fcntl(F_GETFL)");   
37.     exit(EXIT_FAILURE);   
38.   }   
39.   opts = (opts | O_NONBLOCK);   
40.   if (fcntl(sock,F_SETFL,opts) < 0) {   
41.     perror("fcntl(F_SETFL)");   
42.     exit(EXIT_FAILURE);   
43.   }   
44.   return;   
45. }   

/*    PLEASE READ THE FILE NB-APOLOGY!!!!  There are some things you should   know about this source before you read it.  Thanks.      Quang Ngo alerted me to a bug where the variable listnum in deal_with_data()   wasn't being passed in by parameter, thus it was always garbage. I have   quick-fixed this in the code below. - Spencer (October 12, 1999)      Non blocking server demo    By Vic Metcalfe ([e-mail])   For the unix-socket-faq*/#include "sockhelp.h"#include <ctype.h>#include <sys/time.h>#include <fcntl.h>int sock;            /* The socket file descriptor for our "listening"                     socket */int connectlist[5];  /* Array of connected sockets so we know who       we are talking to */fd_set socks;        /* Socket file descriptors we want to wake      up for, using select() */int highsock;       /* Highest #'d file descriptor, needed for select() */void setnonblocking(sock)int sock;{  int opts;  opts = fcntl(sock,F_GETFL);  if (opts < 0) {    perror("fcntl(F_GETFL)");    exit(EXIT_FAILURE);  }  opts = (opts | O_NONBLOCK);  if (fcntl(sock,F_SETFL,opts) < 0) {    perror("fcntl(F_SETFL)");    exit(EXIT_FAILURE);  }  return;}

view plaincopy to clipboardprint?

1. void build_select_list() {   
2.   int listnum;       /* Current item in connectlist for for loops */  
3.   
4.   /* First put together fd_set for select(), which will  
5.      consist of the sock veriable in case a new connection  
6.      is coming in, plus all the sockets we have already  
7.      accepted. */  
8.      
9.      
10.   /* FD_ZERO() clears out the fd_set called socks, so that  
11.     it doesn't contain any file descriptors. */  
12.      
13.   FD_ZERO(&socks);   
14.      
15.   /* FD_SET() adds the file descriptor "sock" to the fd_set,  
16.     so that select() will return if a connection comes in  
17.     on that socket (which means you have to do accept(), etc. */  
18.      
19.   FD_SET(sock,&socks);   
20.      
21.   /* Loops through all the possible connections and adds  
22.     those sockets to the fd_set */  
23.      
24.   for (listnum = 0; listnum < 5; listnum++) {   
25.     if (connectlist[listnum] != 0) {   
26.       FD_SET(connectlist[listnum],&socks);   
27.       if (connectlist[listnum] > highsock)   
28.         highsock = connectlist[listnum];   
29.     }   
30.   }   
31. }   

void build_select_list() {  int listnum;       /* Current item in connectlist for for loops */  /* First put together fd_set for select(), which will     consist of the sock veriable in case a new connection     is coming in, plus all the sockets we have already     accepted. */      /* FD_ZERO() clears out the fd_set called socks, so that    it doesn't contain any file descriptors. */    FD_ZERO(&socks);    /* FD_SET() adds the file descriptor "sock" to the fd_set,    so that select() will return if a connection comes in    on that socket (which means you have to do accept(), etc. */    FD_SET(sock,&socks);    /* Loops through all the possible connections and adds    those sockets to the fd_set */    for (listnum = 0; listnum < 5; listnum++) {    if (connectlist[listnum] != 0) {      FD_SET(connectlist[listnum],&socks);      if (connectlist[listnum] > highsock)        highsock = connectlist[listnum];    }  }}

view plaincopy to clipboardprint?

1. void handle_new_connection() {   
2.   int listnum;       /* Current item in connectlist for for loops */  
3.   int connection; /* Socket file descriptor for incoming connections */  
4.   
5.   /* We have a new connection coming in!  We'll  
6.   try to find a spot for it in connectlist. */  
7.   connection = accept(sock, NULL, NULL);   
8.   if (connection < 0) {   
9.     perror("accept");   
10.     exit(EXIT_FAILURE);   
11.   }   
12.   setnonblocking(connection);   
13.   for (listnum = 0; (listnum < 5) && (connection != -1); listnum ++)   
14.     if (connectlist[listnum] == 0) {   
15.       printf("/nConnection accepted:   FD=%d; Slot=%d/n",   
16.         connection,listnum);   
17.       connectlist[listnum] = connection;   
18.       connection = -1;   
19.     }   
20.   if (connection != -1) {   
21.     /* No room left in the queue! */  
22.     printf("/nNo room left for new client./n");   
23.     sock_puts(connection,"Sorry, this server is too busy.  "  
24.           Try again later!/r/n");   
25.     close(connection);   
26.   }   
27. }   

void handle_new_connection() {  int listnum;       /* Current item in connectlist for for loops */  int connection; /* Socket file descriptor for incoming connections */  /* We have a new connection coming in!  We'll  try to find a spot for it in connectlist. */  connection = accept(sock, NULL, NULL);  if (connection < 0) {    perror("accept");    exit(EXIT_FAILURE);  }  setnonblocking(connection);  for (listnum = 0; (listnum < 5) && (connection != -1); listnum ++)    if (connectlist[listnum] == 0) {      printf("/nConnection accepted:   FD=%d; Slot=%d/n",        connection,listnum);      connectlist[listnum] = connection;      connection = -1;    }  if (connection != -1) {    /* No room left in the queue! */    printf("/nNo room left for new client./n");    sock_puts(connection,"Sorry, this server is too busy.  "          Try again later!/r/n");    close(connection);  }}

view plaincopy to clipboardprint?

1. void deal_with_data(   
2.   int listnum      /* Current item in connectlist for for loops */  
3.   ) {   
4.   char buffer[80];     /* Buffer for socket reads */  
5.   char *cur_char;      /* Used in processing buffer */  
6.   
7.   if (sock_gets(connectlist[listnum],buffer,80) < 0) {   
8.     /* Connection closed, close this end  
9.        and free up entry in connectlist */  
10.     printf("/nConnection lost: FD=%d;  Slot=%d/n",   
11.       connectlist[listnum],listnum);   
12.     close(connectlist[listnum]);   
13.     connectlist[listnum] = 0;   
14.   } else {   
15.     /* We got some data, so upper case it  
16.        and send it back. */  
17.     printf("/nReceived: %s; ",buffer);   
18.     cur_char = buffer;   
19.     while (cur_char[0] != 0) {   
20.       cur_char[0] = toupper(cur_char[0]);   
21.       cur_char++;   
22.     }   
23.     sock_puts(connectlist[listnum],buffer);   
24.     sock_puts(connectlist[listnum],"/n");   
25.     printf("responded: %s/n",buffer);   
26.   }   
27. }   

void deal_with_data(  int listnum      /* Current item in connectlist for for loops */  ) {  char buffer[80];     /* Buffer for socket reads */  char *cur_char;      /* Used in processing buffer */  if (sock_gets(connectlist[listnum],buffer,80) < 0) {    /* Connection closed, close this end       and free up entry in connectlist */    printf("/nConnection lost: FD=%d;  Slot=%d/n",      connectlist[listnum],listnum);    close(connectlist[listnum]);    connectlist[listnum] = 0;  } else {    /* We got some data, so upper case it       and send it back. */    printf("/nReceived: %s; ",buffer);    cur_char = buffer;    while (cur_char[0] != 0) {      cur_char[0] = toupper(cur_char[0]);      cur_char++;    }    sock_puts(connectlist[listnum],buffer);    sock_puts(connectlist[listnum],"/n");    printf("responded: %s/n",buffer);  }}

view plaincopy to clipboardprint?

1. void read_socks() {   
2.   int listnum;       /* Current item in connectlist for for loops */  
3.   
4.   /* OK, now socks will be set with whatever socket(s)  
5.      are ready for reading.  Lets first check our  
6.      "listening" socket, and then check the sockets  
7.      in connectlist. */  
8.      
9.   /* If a client is trying to connect() to our listening  
10.     socket, select() will consider that as the socket  
11.     being 'readable'. Thus, if the listening socket is  
12.     part of the fd_set, we need to accept a new connection. */  
13.      
14.   if (FD_ISSET(sock,&socks))   
15.     handle_new_connection();   
16.   /* Now check connectlist for available data */  
17.      
18.   /* Run through our sockets and check to see if anything  
19.     happened with them, if so 'service' them. */  
20.      
21.   for (listnum = 0; listnum < 5; listnum++) {   
22.     if (FD_ISSET(connectlist[listnum],&socks))   
23.       deal_with_data(listnum);   
24.   } /* for (all entries in queue) */  
25. }   

void read_socks() {  int listnum;       /* Current item in connectlist for for loops */  /* OK, now socks will be set with whatever socket(s)     are ready for reading.  Lets first check our     "listening" socket, and then check the sockets     in connectlist. */    /* If a client is trying to connect() to our listening    socket, select() will consider that as the socket    being 'readable'. Thus, if the listening socket is    part of the fd_set, we need to accept a new connection. */    if (FD_ISSET(sock,&socks))    handle_new_connection();  /* Now check connectlist for available data */    /* Run through our sockets and check to see if anything    happened with them, if so 'service' them. */    for (listnum = 0; listnum < 5; listnum++) {    if (FD_ISSET(connectlist[listnum],&socks))      deal_with_data(listnum);  } /* for (all entries in queue) */}

view plaincopy to clipboardprint?

1. int main (argc, argv)    
2. int argc;   
3. char *argv[];   
4. {   
5.   char *ascport;  /* ASCII version of the server port */  
6.   int port;       /* The port number after conversion from ascport */  
7.   struct sockaddr_in server_address; /* bind info structure */  
8.   int reuse_addr = 1;  /* Used so we can re-bind to our port  
9.         while a previous connection is still  
10.         in TIME_WAIT state. */  
11.   struct timeval timeout;  /* Timeout for select */  
12.   int readsocks;       /* Number of sockets ready for reading */  
13.   
14.   /* Make sure we got a port number as a parameter */  
15.   if (argc < 2) {   
16.     printf("Usage: %s PORT/r/n",argv[0]);   
17.     exit(EXIT_FAILURE);   
18.   }   
19.   
20.   /* Obtain a file descriptor for our "listening" socket */  
21.   sock = socket(AF_INET, SOCK_STREAM, 0);   
22.   if (sock < 0) {   
23.     perror("socket");   
24.     exit(EXIT_FAILURE);   
25.   }   
26.   /* So that we can re-bind to it without TIME_WAIT problems */  
27.   setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &reuse_addr,   
28.     sizeof(reuse_addr));   
29.   
30.   /* Set socket to non-blocking with our setnonblocking routine */  
31.   setnonblocking(sock);   
32.   
33.   /* Get the address information, and bind it to the socket */  
34.   ascport = argv[1]; /* Read what the user gave us */  
35.   port = atoport(ascport); /* Use function from sockhelp to  
36.                                     convert to an int */  
37.   memset((char *) &server_address, 0, sizeof(server_address));   
38.   server_address.sin_family = AF_INET;   
39.   server_address.sin_addr.s_addr = htonl(INADDR_ANY);   
40.   server_address.sin_port = port;   
41.   if (bind(sock, (struct sockaddr *) &server_address,   
42.     sizeof(server_address)) < 0 ) {   
43.     perror("bind");   
44.     close(sock);   
45.     exit(EXIT_FAILURE);   
46.   }   
47.   
48.   /* Set up queue for incoming connections. */  
49.   listen(sock,5);   
50.   
51.   /* Since we start with only one socket, the listening socket,  
52.      it is the highest socket so far. */  
53.   highsock = sock;   
54.   memset((char *) &connectlist, 0, sizeof(connectlist));   
55.   
56.   while (1) { /* Main server loop - forever */  
57.     build_select_list();   
58.     timeout.tv_sec = 1;   
59.     timeout.tv_usec = 0;   
60.        
61.     /* The first argument to select is the highest file  
62.       descriptor value plus 1. In most cases, you can  
63.       just pass FD_SETSIZE and you'll be fine. */  
64.          
65.     /* The second argument to select() is the address of  
66.       the fd_set that contains sockets we're waiting  
67.       to be readable (including the listening socket). */  
68.          
69.     /* The third parameter is an fd_set that you want to  
70.       know if you can write on -- this example doesn't  
71.       use it, so it passes 0, or NULL. The fourth parameter  
72.       is sockets you're waiting for out-of-band data for,  
73.       which usually, you're not. */  
74.        
75.     /* The last parameter to select() is a time-out of how  
76.       long select() should block. If you want to wait forever  
77.       until something happens on a socket, you'll probably  
78.       want to pass NULL. */  
79.        
80.     readsocks = select(highsock+1, &socks, (fd_set *) 0,    
81.       (fd_set *) 0, &timeout);   
82.        
83.     /* select() returns the number of sockets that had  
84.       things going on with them -- i.e. they're readable. */  
85.          
86.     /* Once select() returns, the original fd_set has been  
87.       modified so it now reflects the state of why select()  
88.       woke up. i.e. If file descriptor 4 was originally in  
89.       the fd_set, and then it became readable, the fd_set  
90.       contains file descriptor 4 in it. */  
91.        
92.     if (readsocks < 0) {   
93.       perror("select");   
94.       exit(EXIT_FAILURE);   
95.     }   
96.     if (readsocks == 0) {   
97.       /* Nothing ready to read, just show that  
98.          we're alive */  
99.       printf(".");   
100.       fflush(stdout);   
101.     } else  
102.       read_socks();   
103.   } /* while(1) */  
104. } /* main */  

int main (argc, argv) int argc;char *argv[];{  char *ascport;  /* ASCII version of the server port */  int port;       /* The port number after conversion from ascport */  struct sockaddr_in server_address; /* bind info structure */  int reuse_addr = 1;  /* Used so we can re-bind to our port        while a previous connection is still        in TIME_WAIT state. */  struct timeval timeout;  /* Timeout for select */  int readsocks;       /* Number of sockets ready for reading */  /* Make sure we got a port number as a parameter */  if (argc < 2) {    printf("Usage: %s PORT/r/n",argv[0]);    exit(EXIT_FAILURE);  }  /* Obtain a file descriptor for our "listening" socket */  sock = socket(AF_INET, SOCK_STREAM, 0);  if (sock < 0) {    perror("socket");    exit(EXIT_FAILURE);  }  /* So that we can re-bind to it without TIME_WAIT problems */  setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &reuse_addr,    sizeof(reuse_addr));  /* Set socket to non-blocking with our setnonblocking routine */  setnonblocking(sock);  /* Get the address information, and bind it to the socket */  ascport = argv[1]; /* Read what the user gave us */  port = atoport(ascport); /* Use function from sockhelp to                                    convert to an int */  memset((char *) &server_address, 0, sizeof(server_address));  server_address.sin_family = AF_INET;  server_address.sin_addr.s_addr = htonl(INADDR_ANY);  server_address.sin_port = port;  if (bind(sock, (struct sockaddr *) &server_address,    sizeof(server_address)) < 0 ) {    perror("bind");    close(sock);    exit(EXIT_FAILURE);  }  /* Set up queue for incoming connections. */  listen(sock,5);  /* Since we start with only one socket, the listening socket,     it is the highest socket so far. */  highsock = sock;  memset((char *) &connectlist, 0, sizeof(connectlist));  while (1) { /* Main server loop - forever */    build_select_list();    timeout.tv_sec = 1;    timeout.tv_usec = 0;        /* The first argument to select is the highest file      descriptor value plus 1. In most cases, you can      just pass FD_SETSIZE and you'll be fine. */          /* The second argument to select() is the address of      the fd_set that contains sockets we're waiting      to be readable (including the listening socket). */          /* The third parameter is an fd_set that you want to      know if you can write on -- this example doesn't      use it, so it passes 0, or NULL. The fourth parameter      is sockets you're waiting for out-of-band data for,      which usually, you're not. */        /* The last parameter to select() is a time-out of how      long select() should block. If you want to wait forever      until something happens on a socket, you'll probably      want to pass NULL. */        readsocks = select(highsock+1, &socks, (fd_set *) 0,       (fd_set *) 0, &timeout);        /* select() returns the number of sockets that had      things going on with them -- i.e. they're readable. */          /* Once select() returns, the original fd_set has been      modified so it now reflects the state of why select()      woke up. i.e. If file descriptor 4 was originally in      the fd_set, and then it became readable, the fd_set      contains file descriptor 4 in it. */        if (readsocks < 0) {      perror("select");      exit(EXIT_FAILURE);    }    if (readsocks == 0) {      /* Nothing ready to read, just show that         we're alive */      printf(".");      fflush(stdout);    } else      read_socks();  } /* while(1) */} /* main */
  
Okay, so maybe that wasn't the best example...

Got some suggests? Corrections? Please let me know.

In the mean time, here's some references to other sources of information about select():

Socket FAQ Question about select()
Straight from the excellent socket FAQ.

Unix Programming FAQ Question about select()
Straight from the other excellent FAQ.

Unix Socket Programming FAQ Examples
The above nbserver.c sample code and more socket stuff.

thttpd - tiny/turbo/throttling HTTP server
Nifty little single-threaded, non-fork()'ing, select() based Web server.

BOA
Another single-threaded, select() based Web server.