Long Polling Requests, Servlet 3.0, and DeferredResult

In our practices, most of requests are long polling requests, sadly, we still have not utilized the servlet 3.0 asynchronous features.

1. From reference [1]:  Motivation for Asynchronicity In Web Applications

The most basic motivation for asynchronicity in web applications is to handle requests that take longer to complete. Maybe a slow database query, a call to an external REST APIs, or some other I/O-bound operations. Such longer requests can exhaust the Servlet container thread pool quickly and affect scalability.

In some cases you can return to the client immediately while a background job completes processing. For example sending an email, kicking off a database job, and others represent fire-and-forget scenarios that can be handled with Spring's @Async support or by posting an event to a Spring Integration channel and then returning a confirmation id the client can use to query for the results.

In other cases, where the result is required, we need to decouple processing from the Servlet container thread or else we'll exhaust its thread pool. Servlet 3 provides just such support where a Servlet (or a Spring MVC controller) can indicate the response should be left open after the Servlet container thread is exited.

To achieve this, a Servlet 3 web application can call request.startAsync() and use the returned AsyncContext to continue to write to the response from some other separate thread.At the same time from a client's perspective the request still looks like any other HTTP request-response interaction. It just takes longer to complete. The following is the sequence of events:

   (1) Client sends a request
   (2) Servlet container allocates a thread and invokes a servlet in it
   (3) The servlet calls request.startAsync(), saves the AsyncContext, and returns
   (4) The container thread is exited all the way but the response remains open
   (5) Some other thread uses the saved AsyncContext to complete the response
   (6)  Client receives the response

We do feel the pain. servlet 3.0 is the cure?

2. Spring's support for servlet 3.0: Callable & DeferredResult.

Part of DeferredResult's documentation:

/** * {@code DeferredResult} provides an alternative to using a {@link Callable} * for asynchronous request processing. While a {@code Callable} is executed * concurrently on behalf of the application, with a {@code DeferredResult} the * application can produce the result from a thread of its choice.

both callable and DeferredResult is asynchronous(but might not be non-blocking), difference is: callable is for concurrent execution, DeferredResult is more flexible, don't have to be executed concurrently, you can calculate the result in any thread(as long as it's in the same JVM? need confirmation and verification) at any time(within timeout limit). In our case, I prefer DeferredResult.

from Reference [2]: danny.lesnik:

In order to understand what DefferedResult is doing you need to understand Servlet 3.0 Async concept.

Using Servlet 3.0 you can take async context fro request, store it in kind of Collection.

AsyncContext aCtx = request.startAsync(request, response); 

as the result your Application Container Thread will be resleased. Make some operation on separate thread and write result back to the Servlet response:

aCtx.getResponse().getWriter().print(result);

From that point of your DefferedResult works absolutely the same.

and from the DefferedResult's author:

Roughly speaking. A DeferredResult is associated with an open request. When the request completes, the DeferredResult is removed from the map, and then, the client issues a new long polling request, which adds a new DeferredResult instance.

To me, the most mysterious part is how to write back the result to HTTP response, the core concept you need to understand is AsyncContext or DeferredResult.

phase 1: servlet container receive request(Point Start) -> allocate a servlet thread 001 -> create AsyncContext/DeferredResult -> servlet thread 001 quit (Point A).

phase 2: an arbitrary thread 002 figured out the result value -> get the particular AsyncContext/DeferredResult instance -> set the result (Point B)

phase 3: (servlet container?) allocate a thread 003-> the thread 003 handle the result and write response -> thread 003 quit(Point End).

Noteworthy:

(1) 3 threads involved.

(2) user's waiting time is: from point start to point end

(3) servlet thread work time roughly is: phase1 ?

(4) servlet 3.0 asynchronous servlet's advantage is: between point A and point End, no servlet thread is occupied for this long running request.

Reference:

[1] https://spring.io/blog/2012/05/07/spring-mvc-3-2-preview-introducing-servlet-3-async-support

[2] http://stackoverflow.com/questions/15357990/understanding-the-spring-mvcs-deferredresult-class-in-the-context-of-the-spring