Multitasking server architect

    
Next: 4.4 The Request-Response Loop Up: 4. Inside Apache Previous: 4.2 Structure of the   Contents   Index

Subsections

• 4.3.1 Inetd: A common multitasking architecture
• 4.3.2 Overview -- Apache Multitasking Architectures
• 4.3.3 The Preforking Multiprocessing Architecture
• 4.3.4 Apache Multitasking Architectures and MPMs
• 4.3.5 Win32/WinNT MPM
• 4.3.6 Worker MPM
• 4.3.7 Others MPMs

---

4.3 Multitasking server architectures (G)

The most influencing factor when creating multitasking architecturesis the operating system in use. With every operating system the environmentand their performance attributes change. Therefore certain multitaskingarchitectures are incompatible or not performant enough on certainoperating systems. The second major influencing factor are the use-scenarios.Depending on how much processing is involved with a single request,how many requests a server will have to handle and/or whether requestslogically depend on each other, certain architectures might be moreadvantageous than others.

Section 4.3.1 explains how a commonmultitasking network server architecture works and discusses its shortcomingsif used as an HTTP server. The Apache server architecture will beshown in section 4.3.2.

 

4.3.1 Inetd: A common multitasking architecture

 

4.3.1.1 Master server as a gatekeeper

Network servers handling concurrent requests usually show a multitaskingarchitecture. Although Apache doesn't use it, we will now describea common pattern using processes which works as follows:

A master server process (G)iswaiting for incoming requests. Whenever a request from a client comesin, the master server establishes the connection and then createsa child server process by forking itself. The child serverprocess handles the request(s) of the client while the master serverprocess returns to its waiting state.

 

Figure 4.3:Multiprocessing Architecture of an inetd server (View PDF)

In figure 4.3 you see the structure ofthis kind of multiprocessing server. At the top there are one or manyclients sending requests (R) to the server. The requests are receivedby the TCP/IP Communication Service of the operating system. The MasterServer has registered itself as responsible for any request that comesin. Therefore the communication service wakes it up. The Master Serveraccepts the connection request so the communication service can establishthe TCP/IP connection and create a new socket data structure.

The master server creates a child server process by doing a fork() system call. In figure 4.3this is symbolized by the``write'' arrow from the master serverto the storage area enclosed by a dashed line. The child server knowsabout the connection because it knows the connection socket. It cannow communicate with the client until one of them closes the connection.Meanwhile the master server waits for new requests to arrive.

With TCP/IP, a connection end point has to be identified by the IPaddress of the machine and a port number (for example,the port for HTTP requests has number 80). The master server processregisters itself as a listener for the port (which in turn becomesa server port). Note that a connection request arrives at the serverport while the connection is established using a connection port.The major difference between a server and a connection port is thata server port is solely used to accept connections from any client.A connection port uses the same TCP/IP portnumber but is associatedwith one specific connection and therefore with one communicationpartner. Connection ports are used to transmit data and thereforethe server port can remain open for further connection requests.

 

Figure 4.4:Behavior of a multiprocessing server (View PDF)

The behavior of the server is shown in figure 4.4.The system calls accept() or select() block^4.1 the master server process until a request comes in. accept()waitsfor requests on one server port while select() is a meansto observe multiple ports. In this case, after a request has beenreceived by the TCP/IP Communication Service, the master server canestablish the connection with the system call accept(). Afterthat it creates a new process with fork(). If the requesthas to be handled by a different program, it has to be loaded andexecuted with exec().

 

4.3.1.2 The INETD

The INETD is a typical server using the multitasking architecturedescribed above. It waits for requests on a set of ports defined inthe configuration file /etc/inetd.conf. Whenever a requestcomes in, inetd starts the (child) server program defined in the configurationfile. That program then handles the request.

Apache also provides a mode of operation to work with the inetd. Inthis case, the inetd is the gatekeeper for the HTTP port (80) andstarts Apache whenever an HTTP request arrives. Apache answers therequest and exits.

 

4.3.1.3 Drawbacks

This multiprocessing architecture is useful if the handling of theclient request takes some time or a session state has to be kept bythe (child) server because the communication between client and serverdoes not end after the response to the first request.

HTTP, however, is a stateless protocol.No session information needs to be kept by the server -- it onlyneeds to respond to one request and can ``forget'' about it afterwards.An HTTP server based on the inetd architecture would be inefficient.The master server would have to create a process for each HTTP connection,which would handle this one connection only and then die. While themaster server creates a process it cannot accept incoming requests.Although process creation does not take a long time on modern operatingsystems, this gatekeeper function of the master server forms a bottleneckfor the entire server.

 

4.3.2 Overview -- Apache Multitasking Architectures

All Apache Multitasking Architectures are based on a task pool architecture.At start-up, Apache creates a number of tasks (processes and/or threads),most of them are idle. A request will be processed by an idle task,therefore there's no need to create a task for request processinglike the inetd described in section 4.3.1.

Another common component is the master server, a task controllingthe task pool -- either control the number of idle tasks or justrestart the server something goes wrong. The master server also offersthe control interface for the administrator.

In the following, the preforking architecture will be presented asthe first and most important architecture for unix systems. Then wepresent a selection of other Apache multitasking architectures andemphasize the differences concerning the preforking architecture.

 

4.3.3 The Preforking Multiprocessing Architecture

 

4.3.3.1 The leader-followers pattern

The preforking architecture is based on a pool of tasks (processesor threads) which are playing 3 different roles:

 

• wait for requests (listener)
• process a request (worker)
• queue in and wait to become the listener (idle worker)

Figure 4.5:The leader-followers pattern used in the preforking server architecture (View PDF)

A description of the pattern can be found in [5]. Figure4.5 shows the structure ofthe system: The listener is the leader. Only one task can be grantedthe right to wait for connection requests. If the listener gets arequest, it hands over the right to listen and switches his role toworker, that means he processes the request using the connection heestablished as listener. If he's finished processing the request,he closes the connection and becomes an idle worker. That means hequeues in waiting to become the listener. Usually an idle worker taskwill be suspended.

What are the differences between the server strategy described insection 4.3.1 and the leader-followerstrategy? Firstly, an incoming request will be treated immediatelyby the listener task -- no new task has to be created. On the otherhand there should always be a certain number of idle worker tasksto make sure there is always a listener. Secondly, there is no needto pass information about a request to another task because the listenerjust switches his role and keeps the information.

The task pool must be created at server start. The number of tasksin the pool should be big enough to ensure quick server response,but a machine has resource restriction. The solution is to controlthe number of tasks in the pool by another agent: the master server.

 

4.3.3.2 Preforking Architecture

The Preforking architecture was the first multitasking architectureof Apache. In Apache 2.0 it is still the default MPM for Unix. TheNetware MPM very closely resembles the Preforking functionality withthe exception that it uses Netware threads instead of unix processes.Summarizingly the Preforking architecture of Apache takes a conventionalApproach as each child server is a process by itself. That makes Preforkinga stable architecture but also reduces performance.

 

Figure 4.6:The Apache 2.0 Preforking MPM (View PDF)

The structure diagram in figure 4.6shows the structure of the Preforking architecture of Apache 2.0 andis important for the description of the behavior of Apache. You cansee which component is able to communicate with which other componentand which storage a component can read or modify. The block diagramfor the Apache 2.0 MPM version of the Preforking architecture verymuch resembles the version that was used on Apache 1.3, however thereis one difference: The Master Server uses a ``pipe of death''instead of signals to shut down the child servers for a (graceful)restart.

The Preforking (G) architecture shownin figure 4.6 seems to be verysimilar to the inetd architecture in figure 4.3at first sight. There is one master server and multiple child servers.One big difference is the fact that the child server processes existbefore a request comes in. As the master server uses the fork()system call to create processes and does this before the first requestcomes in, it is called a preforking server. The master serverdoesn't wait for incoming requests at all -- the existing child serverswait and then handle the request directly.

The master server creates a set of idle childserver processes, which register with the TCP/IPcommunication service to get the next request. The first child servergetting a connection handles the request, sends the response and waitsfor the next request. The master server adjusts the number of idlechild server processes within given bounds.

 

4.3.3.3 General Behavior

Figure 4.7 shows theoverall behavior of the server, including the master server and thechild servers.

 

Figure 4.7:Overview: The behavior of Apache (View PDF)

Independently from the underlying multitasking architecture, Apache'sbehaviour consists of a sequence of the following parts which willbe discussed individually for each of the architectures:

 

• First-time initialization:
  Allocate resources, read and check configuration, become a daemon.
• The restart loop:
  (Re-)read configuration, create task pool by starting child serverprocesses and enter the master server loop.
• The master server loop:
  Control the number of idle child server processes in the task pool.
• The request-response loop (Child serveronly):
  Wait to become leader, wait for a connection request, become workerand enter the keep-alive-loop.
• The keep-alive-loop (Child server only):
  Process HTTP requests
• Clean-up before deactivation (Master server and Child servers)

Figure 4.8 shows the behaviorof Apache using the preforking architecture in greater detail.

As each multitasking architecture distinguishes itself from othersby using different means to create and organize child servers, thebehaviour of different multitasking architectures mainly differs whenchild servers, also called workers are created during the restartloop and within the master server loop when the workers are monitoredand replaced.

 

4.3.3.4 Initialization & Restart Loop

 

Figure 4.8:Details of the behavior of Apache (View PDF)

 

4.3.3.4.1 Initialization

The server structure shown in figure 4.6has to be set up at start-up (start processes, create sockets andpipes, allocate memory) and destroyed at shutdown. This is calledactivation and deactivation.

There are three types of initializations:

 

1. at the first activation of Apache
2. every time the restart loop is run
3. every time a child server is created

Apache 2.0 starts with main().After entering the Restart Loop, it calls the configured MPM usingap_mpm_run() .(Apache 1.3 using Preforking starts with the procedure REALMAIN().) The following comments explain the operationsshown in figure 4.8

:

 

• create static pools: Apache initializes memory areas in its ownmemory management (pool management, see section 4.6)
• register information about prelinked modules: The biggest partof the HTTP server functionality is located in the modules (see section3.3 for further details). Modules can eitherbe included in the apache binary or loaded dynamically. Even if theyare included in the binary (prelinked), they have to be registered.
• read command line and set parameters: The administrator can overridedefaults or config file configuration data with command line parameters.The command line parameter -X enforces the 'register one processmode' and can be used for debugging purposes.It prevents the creation of child server processes. If no child serverprocesses exist, there is no need for a master server -- the oneexisting process enters the request-response loop and behaves likea single child server.
• read per-server configuration: The master server (nothing elseexists at this time) reads the configuration files and merges theinformation with its configuration data. Configuration data also includesinformation about the modules to be loaded. Note that configurationdata has to be read a second time in the restart loop!
  ``per-server configuration'' means all static configuration datain contrast to the configuration data in .htaccessfiles called ``per-request configuration''.
• graceful_mode := false: At this time only the master server processexists, so there is no sense in using graceful mode. (In gracefulmode -- see section 4.3.3 -- Apacheperforms a restart keeping active child servers alive.)
• detach process: Each process is usually a child processof the process that created it. The parent process can be the shellprocess, for example. If the shell terminates, all child processesof the shell process are terminated, too. Furthermore, all input andoutput streams of the child process (STDIN, STDOUT, STDERR) are connectedwith the shell.
  Apache performs the detach after it has read the configuration dataand tried to initialize the modules. After the detach no error messagewill be printed in the shell, because the master server has disconnectedfrom the shell and now runs as a background task.
  The detach process consists of the following actions:

   

  1. create a clone of the current process using fork()

      

  2. immediately stop the parent process
  3. the clone detaches the input and output streams from the shell
  4. the clone sets its process group ID to the group ID of process number1 (init). It 'denies' every relationship with its 'real' parent processand from now on only depends on the system process init.

The 'one_process_mode' is useful for debuggingpurposes. Apache skips the detach operation and is still availablefor the debugger.

 

4.3.3.4.2 Restart Loop

Every time the administrator forces a restart of the Apache server,it processes the restart loop which can be found in main().(In Apache 1.3, the restart loop is located in the procedure standalone_main().)After reading the configuration, it calls ap_mpm_run() of the Preforking MPM.

The loop has the following parts:

 

1. initialize and prepare resources for new child servers, read and processconfiguration files
2. create child servers
3. Master server: observe child servers (Master Server Loop, see section4.3.3).
   Child servers: Handle HTTP requests (Request-Response Loop, see section4.4).
4. kill child servers (graceful restart: kill idle child servers only)

The configuration files are read by the masterserver only. The child servers get their configuration data when theyare created by the master server. Whenever the administrator wantsto apply configuration changes to the server, he has to enforce arestart. A graceful restartallows child server processes to complete their processing of currentrequests before being replaced by servers of the new generation. Eachchild server updates its status in the scoreboard(a shared memory area) and compares its own generation ID with theglobal generation ID, whenever it completes request handling.

 

• read per-server configuration: The master server reads and processesthe configuration files. At this time only the master server (andmaybe some non-idle child servers of the old generation) exist.
• set up server sockets for listening: Apache can listen on many ports.It is important not to close the server sockets during restart.
• init scoreboard: In case a graceful restart is processed, the scoreboardentries for the remaining child servers must be kept. Otherwise thereare no child servers and the scoreboard can be initialized.
• one_process_mode: This mode is used fordebugging purposes (see also detach). The master server becomes childserver and enters the request-response loop.
• startup children & register them in the scoreboard: The masterserver creates child server processes with the procedure startup_children().It uses the fork() system call. As a consequence, all childserver processes get a copy of the memory imprint of the master serverand its system resources. Therefore they ``know'' the configurationdata and have access to the TCP/IP sockets and the log file sockets.
  If Apache is started by the super user (root), the master server processis the only process using the root User ID. The child server processesinitialize, set their user ID to a non-privileged account like ``nobody''or ``wwwrun'' and enter the request-response loop.
  The master server creates an entry in the scoreboard for every childserver including its process ID and generation ID.
• restart/shutdown pending := false: The master server entersthe master server loop. It exits the loop only if one of those globalvariables is set to ``true'' by the signal handler.
• Master server loop: (see section 4.3.3and figure 4.9) At the beginningof the loop the master server waits a certain time or receives thenotification that a child server has died. Then it counts the numberof idle child servers and regulates their number by creating or killingone.
• proclaim new generation: Each time the master server processes therestart loop, it increments the generation ID. All child servers itcreates have this generation ID in their scoreboard entry. Whenevera child server completes the handling of a request, it checks itsgeneration ID against the global generation ID. If they don't match,it exits the request-response loop and terminates.
  This behavior is important for the graceful restart.
• finish all/idle children: Both shutdown and restart result in thedeath of all child servers. When the administrator requests a gracefulrestart, only the idle child servers are killed.
• free resources: Apache returns the occupied resources to the system:Memory, TCP/IP ports and file handles.

 

4.3.3.5 Inter-Process Communication (Signals and Pipe of Death)

Apache is controlled by signals. Signals are a kind of software interrupts.They can occur at any time during program execution.The processorstops normal program execution and processes the signal handler procedure.If none is defined in the current program, the default handler isused which usually terminates the current program. After the executionof the signal handler the processor returns to normal execution unlessthe program was terminated by the signal.

 

4.3.3.5.1 Administrator controls the master server

The administrator can send signals directly (using kill atthe shell command prompt) or with the help of a script. The masterserver reacts to three signals:

 

• SIGTERM: shut down server (set shutdown_pending:= true)
• SIGHUP: restart server (set restart_pending:= true and graceful_mode := false)
• SIGUSR1: restart server gracefully (set restart_pending:= true and graceful_mode := true)

The signal handlers for the master server are registered in the procedureset_signals().It registers the signal handler procedures sig_term()and restart().

In the upper right corner of figure 4.8you see a small petri net describing the behavior of the signal handlerof the master server. Apache activates and deactivates signal handlingat certain points in initialization and in the restart loop. Thisis not shown in figure 4.8.

 

4.3.3.5.2 Master Server controls the child servers

While the administrator controls the master server by signals only,the master server uses either signals or a pipe to control the numberof child servers, the Pipe of Death. (Apache 1.3 used signals only).

 

Table 4.1:Normal versus graceful restart

normal restartgraceful restart
send SIGHUP to the process group

 

 

For a shutdown or non-graceful restart, the master server sends aSIGHUP signal to the process group. The operating system ``distributes''the signals to all child processes belonging to the group (all childprocesses created by the master server process). The master serverthen ``reclaims'' the notification about the termination of allchild servers. If not all child processes have terminated yet, ituses increasingly stronger means to terminate the processes.

A graceful restart should affect primarily the idle child server processes.While Apache 1.3 just sent a SIGUSR1 signal^4.2 to the process group, Apache 2 puts ``Char of Death'' items intothe Pipe of Death (pod). The busy child servers will check the pipeafter processing a request, even before comparing their generation.In both cases they set the die_now flag and terminateupon beginning a new iteration in the request-response loop

Table 4.1 liststhe differences between a normal and a Graceful Restart (G).

 

4.3.3.5.3 Pipe of Death (PoD)

The Master server of Apache 2.0 uses the Pipe of Death for inter-processcommunication with the child servers to terminate supernumerary onesand during graceful restart. All child servers of one generation sharea pipe.

If the master server puts a Char of Death in the queue using ap_mpm_pod_signal()or sends CoD to all child servers with ap_mpm_pod_killpg(),these procedures also create a connection request to the listeningport using the procedure dummy_connection()and terminate the connection immediately. The child server waitingfor new incoming connection requests (the listener) will accept therequest and skip processing the request as the connection is alreadyterminated by the client. After that it checks the PoD which causeshim to terminate. Busy child servers can continue serving their currentconnection without being interrupted.

 

4.3.3.6 Master Server Loop

 

4.3.3.6.1 Overview

In this loop the master server on the one hand controls the numberof idle child servers, on the other hand replaces the child serversit just killed while performing a graceful restart.

 

Figure 4.9:The Master Server Loop (View PDF)

While the restart loop can be found within the server core in main(),the master server loop it is located within the corresponding MPM,which in this case is the Preforking: ap_mpm_run() .(In Apache 1.3 it can be found in the procedure standalone_main() in the file http_main.c.)

In figure 4.9 you see thedetails of the loop. The upper part deals with the reaction to thedeath of a child process and special issues of a graceful restart.The lower part is labeled ``perform idle server maintenance''.It shows a loop in which the master server counts the number of idleservers and gets a list of free entries in the scoreboard. It comparesthe number of idle children (idle_count) with the limitsgiven in the configuration (ap_daemons_max_free and ap_daemons_min_free).If there are too many idle servers, it kills exactly one of them (thelast idle server in the scoreboard). If the number of idle child serversis too low, the master server creates as many child server processesas needed (see exponential mode below).

 

4.3.3.6.2 Graceful Restart -- Reaction to the death of a child process

The following remarks mainly deal with the graceful restart and thereaction to the death of a child process:

 

• set rem.child2start: The variable remaining_children_to_startis used only in the case of a graceful restart. It holds the numberof child servers that should be started after the server is up. Noticethat the master server does not use the startup_childrenprocedure. While performing the wait() system call, the masterserver gets a notification for every idle child server it has killed.If the number of initial child servers was changed in the configuration,it would be wrong to just replace dead children, so the master serveruses remaining_children_to_start to control the number.
• pid := wait or timeout: The wait() system call is used to wait for the termination of a child processcreated with fork(). After waiting for a given period oftime, the master server continues execution of the master server loopeven if it has not received a termination notification.
• wait() was successful (pid is set):

   

  • process_child_status: Get the reason for the death of thechild process
  • find_child_by_pid: Look for the scoreboard entry
  • entry (slot) found: set child status to SERVER DEAD. If remaining_children_to_startis not zero, create a new child server to replace the dead child server.
  • entry not found: Check if this child process has been an ``otherchild''<sup>4.3</sup>(reap_other_child(), see below). If it is neither an ``otherchild'' nor a scoreboard entry matches, and if graceful mode is set,then the following situation must have happened:
    The administrator has reduced the number of allowed child serversand forced a graceful restart. A child server process that has beenbusy had a slot greater than the allowed number. Now it terminates,but its entry can not be found in the scoreboard.
• time-out: If remaining_children_to_start is still notzero after all terminated child servers have been replaced, thereare more servers to be created. This is done by the procedure startup_children().

 

4.3.3.6.3 Performing Idle Server Maintenance

The lower part of figure 4.9shows the behavior of the procedure perform_idle_server_maintenance()which is called whenever a time-out occurred and the graceful restarthas been finished.

The master server counts the number of idle servers and the numberof remaining slots (entries) in the scoreboard and compares it withthree limits given in the configuration:

 

• [ap_daemons_limit]maximum number of child servers. Thesum of busy and idle child servers and free slots or just the numberof slots of the scoreboard.
• [ap_daemons_max_free]maximum number of idle child servers.If the number of idle servers is too high, one child server processis killed in each loop.
• [ap_daemons_min_free]minimum number of idle child servers.If the number of idle servers is too low, the master server has tocreate new child servers. If no free slots are left (because ap_daemons_limithas been reached), a warning is written to the log.

Exponential mode:Some operating systems may slow down if too many child processes arecreated within a short period. Therefore the master server does notimmediately create the needed number of child servers with make_child().It creates one in the first loop, two in the second, four in the thirdand so on. It holds the number of child servers to be created in thenext loop in the variable idle_spawn_rate and increments it with every turn until the number of idle child serversis within the limit.

Example: ap_daemons_min_free is set to 5 but suddenly there isonly 1 idle server left. The master server creates one child serverand waits again. 2 idle servers are still not enough, so the mastercreates 2 more child servers and waits again. In the meantime, a newrequest occupies one of the new child servers. The master server nowcounts 3 idle child servers and creates 4 new ones. After the time-outit counts 7 idle child servers and resets the idle_spawn_rateto 1.

 

4.3.3.7 Child Servers (Worker)

The Child Servers sometimes referred to as a workers form the heartof the HTTP Server as they are responsible for handling requests.While the multitasking architecture is not responsible for handlingrequests it is still responsible for creating child servers, initializingthem, maintaining them and relaying incoming connections to them.

 

4.3.3.7.1 Initialization, Configuration and Server restarts

The master server creates child server processes using the fork()system call. Processes have separate memory areas and are not allowedto read or write into another processes' memory. It is a good ideato process the configuration once by the master server than by eachchild server. The configuration could be stored in a shared memoryarea which could be read by every child server process. As not everyplatform offers shared memory, the master server processes the configurationfiles before it creates the child server processes. The child serverprocesses are clones of the master server process and therefore havethe same configuration information which they never change.

Whenever the administrator wants to apply changes to the server configuration,he has to advice the master server to read and process the new configurationdata. The existing child server processes have the old configurationand must be replaced by new processes. To avoid interrupting the processingof HTTP requests, Apache offers the ``graceful restart''mode (see section 4.3.3), whichallows child servers to use the old configuration until they havefinished processing their request.

The initialization of a child server can be found in the correspondingMPM (Preforking: child_main(),Apache 1.3: child_main()).It consists of the following steps (see also figure 4.13):

 

• establish access to resources: The child server process has justbeen created by the master server using fork().At this time the child server process has the same privileges as themaster. This is important if Apache has been started by the superuser (root). Before the child server sets its user ID to a non-privilegeduser, it must get access to common resources:

   

  • Private memory (ap_make_sub_pool())
  • The scoreboard (may be implemented as shared memoryor as a file)
  • The accept mutex (depends on the implementation)
• Re-initialize modules (ap_init_child_modules()): Everymodule has been initialized by the master server before. Re-initializationis necessary if the module allocates system resources or depends onthe process ID (e.g. for a database connection).
• Set up time-out handling: To avoid infinite blocking of the childserver, Apache uses a time-out for the request handling. It usesalarms, a concept similar to signals. It is like setting an alarmclock to a given time and leaving the processing of the request whenthe ``alarm bell rings''. This is done using the concept of ``longjump''.
• Within the loop, there are two initialization steps left:

   

  • clear time-out: reset alarm timer
  • clear transient pool: Every memory allocation within the request-responseloop concerns the transient pool. At the beginning of the loop, thepool is cleared.
• set status := ready in the scoreboard except after a newgeneration has been announced.

 

4.3.3.7.2 Accepting Connections

The Preforking architecture uses an accept mutex(G)to distribute incomming connections among multiple child servers.The accept mutex makes sure that onlyone child server process exclusively waits for a TCP request (usingthe system call accept()) -- this is what a listenerdoes. The Accept Mutex^4.4 is a means of controlling access to the TCP/IP service. It is usedto guarantee that, at any time, exactly one process waits for TCP/IPconnection requests.

There are various implementations of the Accept Mutex for differentOS configurations. Some need a special initialization phase for everychild server. It works this way:

 

• call procedure accept_mutex_on(): acquire the mutex orblock until available
• call procedure accept_mutex_off(): release the mutex

After having received a connection, the child server releases theaccept mutex and processes the request -- it becomes a worker andlets the next process wait for a request. This is usually called theLeader-Follower pattern: The listener isthe leader, the idle workers are the followers (see figure 4.5).As Apache uses operating system dependend techniques for the mutex,it is possible depending on the operating system that all currentlyblocked child servers are woken when one child servers returns themutex after receiving a connection. If so, excessive scheduling causedunnecessarily as only one of the woken child servers will get themutex, the others will be blocked and therefore return to sleep. Thatis a problem which is addressed by the Leader MPM where followersare organized in a way such that only one of them is woken when theaccept mutex is returned.

Once a connection is received by a child server, the scope of responsibilityof the multitasking architecture ends. The child server calls therequest handling routine which is equally used by any multitaskingarchitecture.

 

4.3.3.7.2.1 Accept Mutex vs. select()

In an inetd server (see section 4.3.1),there is only one process waiting for a TCP/IP connection request.Within the Apache HTTP server, there are possibly hundreds of idlechild servers concurrently waiting for a connection request on morethan one server port. This can cause severe problems on some operatingsystems.

Example: Apache has been configured to listen to the ports 80, 1080and 8080. 10 Idle child server processes wait for incoming TCP/IPconnection requests using the blocking system call select()(they are inactive until the status of one of the ports changes).Now a connection request for port 1080 comes in. 10 child server processeswake up, check the port that caused them to wake up and try to establishthe connection with accept(). The first is successful andprocesses the request, while 9 child servers keep waiting for a connectionat port 1080 and none at port 80 and 8080! (This worst-case scenariois only true for blocking^4.5 accept())

Therefore in a scenario where there are multiple child servers waitingto service multiple ports the select() accept() pair is not sufficientto achieve mutual exclusion between the multiple workers. ThereforePreforking has to use the accept mutex.

In general it is a bad idea to waste resources of the operating systemto handle concurrency. As some operating systems can't queue the childserver processes waiting for a connection request, Apache has to doit.

 

4.3.4 Apache Multitasking Architectures and MPMs

A multiprocessing architecture's main task is to provide a fast respondingserver which uses the underlying operating system efficiently. Usuallyeach architecture has to accept a trade-off between stability andperformance.

In case of a HTTP server, the multitasking architecture describesthe strategy how to create and control the worker tasks and how theyget a request to process.

The first choice concerns the tasks: Depending on the platform, theserver can use processes or threads or both to implement the tasks.Processes have a larger context (for example the process' memory)that affects the time needed to switch between processes. Threadsare more lightweight because they share most of the context, unfortunatelybad code can corrupt other thread's memory or more worse crash allthreads of the process.

The next aspect affects the way how tasks communicate (Inter TaskCommunication). In general, this can be done by shared memory, signalsor events, semaphores or mutex and pipes and sockets.

As all MPMs use a Task Pool strategy (idle worker tasks remain suspendeduntil a request comes in which can immediately be processed by anidle worker task), there must be a means to suspend all idle workertasks and wake up one whenever a request occurs. For this, an operatingsystem mechanism like a conditional variable or a semaphore must beused to implement a mutex. The tasks are suspended when calling ablocking procedure to get the mutex.

The server sockets are a limited resource, therefore there can onlybe one listener per socket or one listener at all. Either there arededicated listener tasks that have to use a job queue to hand overrequest data to the worker tasks, or all server tasks play both roles:One idle worker becomes the listener, receives a request and becomesa worker processing the request.

Finally a task can control the task pool by adjusting the number ofidle worker tasks within a given limit.

Apache includes a variety of multitasking architectures. OriginallyApache supported different architectures only to support differentoperating systems. Apache 1.3 had two major architectures which hadto be defined at compile time using environment variables that theprecompiler used to execute macros which in turn selected the correspondigcode for the operating system used:

 

• The Preforking Architecture for Unix
• The Job Queue Architecture for Windows(G)

Multiprocessing implementation was completely changed in Apache 2.0by introducing Multi Processing Modules (MPM).These Modules are exchangeable and completely encapsulate the multiprocessingarchitecture. As any other module, an MPM's module structure is alikethe standard module structure and includes a command table. MPMs cannotbe dynamically included and therefore have to be chosen when compilingthe server from the sources. Due to the nature of a multitasking architecturemodel, only one MPM can be included in one server at one time.

An MPM's responsibility is located within the main loops of Apache.The main server will do all initialization and configuration processingbefore calling the method ap_mpm_run() in main()to hand over to the MPM.

 

Figure 4.10:Responsibility of an Apache 2.0 MPM (View PDF)

It is the MPM's responsibility to take care of starting threads and/orprocesses as needed. The MPM will also be responsible for listeningon the sockets for incoming requests. When requests arrive, the MPMwill distribute them among the created threads and/or processes. Thesewill then run the standard Apache request handling procedures. Whenrestarting or shutting down, the MPM will hand back to the main server.Therefore all server functionality is still the same for any MPM,but the multiprocessing model is exchangeable. Figure 4.10shows the responsibility of an Apache MPM in the overall behaviorof Apache (see also figure 4.7).The dotted line marks the actions for which the MPM takes responsibility.

Version 2.0 currently includes the following MPMs:

 

• Preforking and Netware -- MPMs that resemble the functionality ofthe Preforking architecture of Apache 1.3
• WinNT -- Apache 1.3's Win32 version was similar to this, howeverthe WinNT MPM is enhanced by the IOCP operating system concept
• Worker -- A new MPM that makes use of both processes and threadsand performs better than Preforking
• Leader and PerChild -- Two MPM's still in an experimental state analternative to Preforking and Worker on Linux based Systems

 

4.3.5 Win32/WinNT MPM

The Windows multitasking architecture has some significant differencescompared to the preforking architecture. It uses threads(G)instead of processes and the number of child servers is not variablebut fixed. Threads are used because threads are a lot more performantthan windows processes. There are two Windows processes in this multitaskingarchitecture: The worker process (childserver process) contains the threads which handle the HTTP requests,while the supervisor process (masterprocess) makes sure that the worker process works correctly. For connectiondistribution to the workers a job queue is used. Additionallythe Apache 2.0 MPM version of this multitasking architecture usesa windows NT operating concept called I/O Completion Port insteadof a job queue when used on Windows NT platforms. The version 1.3of this architecture as well as the Windows32 version of the Apache2.0 MPM use a single listener thread to accept connections. The WindowsNTvariant of the MPM uses one listener per port that Apache has to listenon.

 

Figure 4.11:The Apache 2.0 WinNT MPM (View PDF)

Figure 4.11 shows the system structureof the server using the WinNT MPM: The Master Server Process createsthe Child Server Process and then monitors the Child Server in casethe process dies.

 

4.3.5.1 Initialization & Restart Loop

The initialization procedure of the Win32 multitasking architectureclosely resembles the one described for the Preforking architecture.All initialization is similar to the point where the Preforking MPMis called or the Apache 1.3 architecture starts to create child serverprocesses.

Both, the 1.3 and the 2.0 version use the master server process asthe supervisor process. That in turn creates the second process thatcontains all the worker threads. When started the worker process onlycontains a single master thread which then spawns the fixed numberof worker threads. These correspond to the child server processeswithin the Preforking architecture. The Windows multitasking versionuses a fixed number of threads since idle threads impose almost noperformance issue. Therefore as many threads as are desirable foroptimum performance are started right away and the server can be usedto its maximum capability, without the overhead and delay of spawningnew processes dynamically.

Both Windows multitasking architectures only support graceful restartor shutdown.

 

4.3.5.2 Inter-Process/Thread Communication (events and Exit Flag)

A Variety of inter-process communication mechanisms is used in thismultitasking architecture. As Windows does not support signals, thenative Windows concept of events is used for communicationbetween the supervisor or master server process and the worker process.Here events are used for signaling:

 

• The master server process can signal the worker process that a shutdownor graceful restart is in progress.
• On the one hand the worker process can signal the master server processthat it needs to be restarted or that a serious error occurred thatrequires to shutdown the server.

The worker process itself uses various means for communication withthe listener(s) and the worker threads. When a shutdown event occursthe master thread puts ``die''- jobs into the job queue or theIOCP used. Thus idle and sleeping worker threads are woken and exit,while worker threads that are busy handling a request can completethe request and quit later. Additionally it sets various exit flagsthat can be read by the listener(s) as well as the worker threads.

However job queue or IOCP respectively are also used by the listenersto communicate arriving connections to the worker threads for requesthandling.

 

4.3.5.3 Master Server (Supervisor) Process

The master server process is called the supervisor process when enteringthe restart loop. It contains only one thread and is used to monitorthe second process called the worker process to be able to restartit in case it crashes. The user can communicate with this specificprocess using the control manager which can be found on any windowsplatform. The control manager then sends an event to the server whichsignals a restart or shutdown. Additionally the apache server suppliescommand line options that can be used for signaling.

 

4.3.5.4 Child Process: Listeners and the Job Queue

The worker process contains three kinds of threads: One master thread,a fixed number of worker threads and one ormultiple listeners. The master starts one or multiple listeners whichaccept the connection requests and put the connection data into ajob queue (like a gatekeeper). The worker threadsfetch the connection data from the queue and then read and handlethe request by calling the core's request handling routine which isused by all multitasking architectures. The communication betweenthe master and the worker threads is also accomplished via the jobqueue. However the only communication necessary between master andworker thread is to signal a worker to exit. If the master threadwants to decrease the number of worker threads due to a pending shutdownor restart, it puts "die"-jobs into the queue.

Instead of a selfmade job queue, the MPM of Version 2.0 uses the IOCPon Windows NT plattforms. The advantage of the I/O Completion Portis that it enables the server to specify an upper limit of activethreads. All worker threads registering with the IOCP are put to sleepas if registering with a job queue. When any of the events the IOCPis responsible for occurs one worker thread is woken to handle theevent (in Apache that can only be a new incomming connection request).If however the limit of active threads is exceeded, no threads arewoken to handle new requests until another thread blocks on a synchronouscall or reregisters with the IOCP. That technique is used to preventexcessive context switching and paging due to large numbers of activethreads.

 

4.3.5.5 Child Process: Worker Threads

Worker Threads are kept pretty simple in this architecture model.As they can share any memory with their parent process (the workerprocess) they do not need to initialize a lot of memory. All theymaintain is a counter of requests that the thread processed. The MPMversion also keeps a variable containing the current operating systemversion so that either the job queue or the IOCP is choosen when tryingto get a connection. Therefore the initialization is very short.

After intialization the worker registers with the IOCP or the jobqueue to retrieve a connection which it can handle. After receivingthe connection it calls the core's request processing routine. Itcontinues to do that until it is given a ``die''-job from thequeue, which would cause it to exit.

 

4.3.6 Worker MPM

The Worker MPM is a Multiprocessing Model for the Linux/Unix OperatingSystem Platform. In contrast to the Preforking and WinNT Model, thisMPM uses a combination of a multiprocessing and a multithreading model:It uses a variable number of processes, which include a fixed numberof threads (see figure 4.12). Thepreforking model on process level is extended by a job queue modelon thread level.

 

Figure 4.12:The Apache 2.0 Worker MPM (View PDF)

Still the master server process adjusts the number of idle processeswithin a given range based on server load and the max_child,max_idle and min_idle configurationdirective. Each child process incorporates a listener thread, whichlistens on all ports in use by the server. Multiple processes andtherefore multiple listener threads are mutually excluded using theaccept mutex like in the Preforking Model.

 

4.3.6.1 Child Server / Worker

Initialization of a child server is a more complex task in this case,as a child server is a more complex structure. First the master servercreates the child process, which in turn starts a so called starterthread that has to set up all worker threads and a single listenerthread. This behavior is reflected in figure 4.12.

Within each child process, the communication between the listenerand all worker threads is organized with two queues, the jobqueue and the idle queue.A listener thread will only apply for the accept mutex if it findsa token in the idle queue indicating that at least one idle workerthread is waiting to process a request. If the listener gets the mutex,it waits for a new request and puts a job item into the queue afterreleasing the accept mutex. Thus it is ensured that a incoming requestcan be served by a worker immedialtely.

After completing a request or a connection with multiple requests(see section 2.3.4for details) the worker thread registers as idle by putting a tokeninto the idle queue and returns to wait for a new item in the workerqueue.

Advantages of this approach are that it combines the stable conceptof multiprocessing with the increased performance of a multithreadingconcept. In case of a crash, only the process that crashed is affected.In multithreading a crashing thread can affect all threads belongingto the same parent process. Still threads are a lot more lightweightand therefore cause less performance overhead during start-up andconsume less memory while running.

 

4.3.7 Others MPMs

The MPMs mentioned so far are the MPMs used most often. Additionallythere are other MPMs available. However most of these mainly servean experimental purpose and are seldom used in productive environments.

 

4.3.7.1 Leader MPM

This MPM uses the preforking (Leader-Follower,see also figure 4.5 and thepattern description in [5](G))model on both process and thread level using a sophisticated mechanismfor the followers queue:

Each Child Process has a fixed number of threads like in Worker MPM.However, threads are not distinguished into worker and listener threads.Idle workers are put onto a stack. The topmost worker is made listenerand will upon receiving a connection immediately become a worker tohandle the connection itself. The worker on the stack below him willbecome the new listener and handle the next request. Upon finishinga request the worker will return to the top of the stack.

This approach addresses two performance issues. First there is nodelay due to handing a connection to a different task using a jobqueue, since each thread simply handles the connectionit accepts. Secondly since follower threads are organized in a stack,only one thread is woken when the listener position becomes available.The overhead that is caused when all threads are woken to competefor the mutex is avoided.

A thread returning to the stack is put on top. Therefore it is mostlikely that a thread on top will handle more requests than a threadat the bottom. Considering the paging techniques for virtual memorythat most operating systems use, paging is reduced as more often usedthreads do more work and thus are less likely to be swapped to thehard disk.

 

4.3.7.2 Per-Child MPM

Based on the Worker MPM, this experimental MPM uses a fixed numberof processes, which in turn have a variable number of threads. ThisMPM uses also uses the preforking model on both process and threadlevel. The advantage of this approach is that no new processes haveto be started or killed for load balancing.

An advantage of this MPM: Each process can have a separate UserID,which in turn is associated with different file access and programexecution rights. This is used to implement virtual hosts with differentrights for different users. Here each virtual host can have its ownprocess, which is equipped with the rights for the corresponding owner,and still the server is able to react to a changing server load bycreating or destroying worker threads.

 

---

Footnotes

... block^4.1

this means the server process remains inactive until a request comesin

... signal^4.2

The Apache 1.3 child server's reaction to a SIGUSR1 signal: Terminateif idle else set deferred_die and terminate later. (Seesignal handler usr1_handler()registered at child server initialization.)

... child''^4.3

In some cases the master server has to create child processes thatare not child server processes. They are registered as``other child''in a separate list. An example: Instead of writing log data to files,Apache can stream the data to a given program. The master server hasto create a process for the program and connect its STDIN stream withits logging stream. This process is an``other child''. Wheneverthe server is restarted, the logging process gets a SIGHUP or SIGUSR1.Either it terminates and has to be re-created by the according module(the logging module in the example) or it stays alive. The modulemust check the``other child'' list to find out if it has to createor re-use a process.

... Mutex^4.4

A mutex is a semaphore (G)used to enforce mutualexclusion for the access to a resource. A semaphore isa means for inter-process communication (IPC): A process can incrementor decrement the value of a semaphore. The process is suspended (blocked)if it tries to decrement and the value is zero, until another processincrements the value of the semaphore. To implement a mutex with asemaphore, you have to set the maximum value to 1.

... blocking^4.5

``blocking'' means: Any process calling accept() is suspended.If a connection request arrives, the first resumes its operation.

---

原文 http://www.fmc-modeling.org/category/projects/apache/amp/4_3Multitasking_server.html