Pramati Nex Gen LB

              Pramati NexGen LB is a software web load balancer which comes with greatest features compared to any typical LB.   Pramati NexGen LB can be put in front of any Web Container which can support standard Proxy headers.

Some of the salient features of Pramati NexGen LB are:

Protocols Support   

Supports HTTP, HTTPS and AJP protocol.

Better Configuration Support

• Allows sourcing entire LB configuration from Custom source. For Ex, the LB configuration can be provided form Database using DBConfiguration.   
• Allows dynamically configuring and reloading LB Configuration without requiring any downtime. 

Effective Handling of Slow Clients

Sometimes clients with slower I/O can hold the backend server resources for long time making the server less available. This is effectively handled in NexGen LB using the byte sink. Before dispatching the request to the actual backend node, the request body is first read to local file sink. Once the entire request body is read to sink, the request is dispatched to backend node by serving the request body from the file sink. Similarly, the response from the backend node is first written to file sink and later written to the actual client.

Advanced SSL

Supports the following advanced SSL features such as:

• Configuring separate certificate for  each virtual host.
• Dynamically reloading certificate without restarting the LB.

Advanced Node Selection mechanisms

Comes with wide range of pre-defined Node selection algorithms. Some of them are:

• Round Robin : Uses round robin strategy for selecting the backend node
• Weighted Round Robin: Same as round robin but gives weightage to nodes based on the weights assigned to nodes. For ex, the weights can be assigned based on the number of CPU's each node has.
• Fastest: Selects the node which has the fastest request processing time.
• Min-Requests: Selects the node which has processed the minimum number of requests.
• Max-Requests: Selects the node which has processed the maximum number of requests.

Supports wide range of Stickiness Mechanisms

Provides stickiness while node selection using the following stickiness mechanisms:

• Servlet Session: Node which has previously processed the same JSESSION id.
• User Name: Node which has previously processed the same user. This is supported for BASIC authentication only which send the user name encoded in Authentication header.
• Cookie : Node which has previously processed the same cookie and value.
• Client Address : Node which has processed the same client address previously.
• HTTP Header: Node which has previously processed the same header and value.
• HTTP Parameter: Node which has previously processed the same http parameter and value.
• Query Parameter: Node which has previously processed the same query parameter and value.
• Cookie Affinity: The node name which processes the request is sent as Cookie to the client. All the further requests are dispatched to the node based on the Cookie value.

Supports Custom Node Selection

Allows writing custom node selection mechanism in Java or JavaScript. The JavaScript node selector's can be modified at runtime which will be applied automatically.

Serving Content from Other Sources

 Content Handlers helps us serving the content from other locations such as:

• Static content from the LB local file system
• Content from any remote location.

Rewriting headers

Allows rewriting upstream headers while forwarding the requests to backend. Similarly does rewriting downstream headers. The following operations can be performed on the upstream/downstream header:

• Add
• Modify
• Delete

Comparing Servlet 3.0 Asynchronous Servlets with Jetty Continuations

Why do we need Asynchronous Servlets?
Asynchronous Servlets are useful when a Servlet needs to wait for resources or events. For example, a Servlet may need to wait for:

• Response from Remote Web Resource
• A JDBC connection
• A JMS message
• An application event

Because, waiting is an inefficient operation which consumes resources such threads, database connections, buffers etc.

Containers implementing Asynchronous Servlets:

• Jetty 6 supporting Continuations
• Containers implementing Servlet 3.0
• Jetty 7( onwards supports both Jetty Continuations and Servlet 3.0 Asynchronous Servlets

How does Asynchronous Servlets Work:

• Servlet/Filter can begin asynchronous processing
• Thread will return to the container after the asynchronous processing of the request has started. The Servlet response will not be committed like default Servlet Life cycle.
• Another thread may either 
• generate the response and call complete 
• or dispatch the request to run in container context.

Servlet 3.0 introducing new features compared to Jetty 6  

The Servlet 3.0 asynchronous API introduced some new asynchronous features not supported by jetty 6 continuations:

• Ability to complete an asynchronous request without re-dispatching
• Listeners for asynchronous events
• Dispatching asynchronous requests to specific contexts and/or resource
• Support for wrapping requests and responses.
• Applying Filters during dispatch mode.

Later, Jetty started adopting above features  except "Applying Filters during dispatch mode" since Jetty 7.

Comparing API: Jetty Continuation Vs Servlet 3.0 Async Servlet

Feature	Jetty Continuation	Servlet 3.0
Suspend	Continuation.suspend(HttpServletResponse)	HttpServletRequest.startAsync()
Resume	Continuation.resume()	AsyncContext.dispatch()
Complete	Continuation.complete()	AsyncContext.complete()

Understanding Asynchronous Servlets(Code):

Lets take an example where a Servlet blocks while waiting for Remote Web Service call to return and try to understand how the Servlets were written without the invent of Asynchronous Servlets , Jetty 6 Continuations and Servlet 3.0 Asynchronous Servlets.

Synchronous Servlets

public class RemoteWebServiceServlet extends HttpServlet {
      public void doGet(HttpServletRequest req, HttpServletResponse res) {
             Object result  = makeRemoteWebService(...);
                       writeResponse(result);
    }
}

 Asynchronous Servlet using Servlet 3.0

@WebServlet(async=“true”)
public class AsyncServlet extends HttpServlet {
     public void doGet(HttpServletRequest req, HttpServletResponse res) {
          // Check if request it is not async dispatch
          if (!req.isAsyncStarted()) {

               AsyncContext ctx = req.startAsync(req, res); //Start Async Processing
               ctx.start(new  RemoteServiceHandler() implements Runnable {
                    public void run() {
                        Object result  = makeRemoteWebService(...);
                         request.setAttribute("result", result);   
                         ctx.dispatch(); // Re-dispatch/ resume to generate response
                    }
               });
          } else {
               Object result = request.getAttribute("result");
               writeResponse(result);
          }
     }
}

  Asynchronous Servlet using Jetty Continuations:

 public class AsyncServlet extends HttpServlet {
     public void doGet(HttpServletRequest req, HttpServletResponse res) {
          / // Check if request it is not async dispatch
          Object result = request.getAttribute("result");
          if (result == null) {
               Continuation continuation = ContinuationSupport.getContinuation(request);
       continuation.suspend(response); //Start Async Processing
              threadpool.submit(new  RemoteServiceHandler() implements Runnable {
                    public void run() {
                        Object result  = makeRemoteWebService(...);
                         request.setAttribute("result", result);  
                        continuation.resume(); // Re-dispatch/ resume to generate response
                    }
               });
          } else {
               Object result = request.getAttribute("result");
               writeResponse(result);
          }
     }

}

 Internals of Servlet 3.0 Async and Jetty 6 Continuations:

Now, we will try to understand how Servlet 3.0 Async and Jetty 6 Continuations work internally. Infact, both work in the same way.   If you see below, there is a single implementation class for both Servlet 3.0 Async and Jetty 6 Contination:

public class AsyncContinuation implements AsyncContext, Continuation { 

               ...........

}

 And the below code shows that Jetty HttpServletRequest implementation internally uses Continuation API:

public class Request implements HttpServletRequest {

..............

public AsyncContext startAsync() throws IllegalStateException
    {
        if (!_asyncSupported)
            throw new IllegalStateException("!asyncSupported");
        _async.suspend();
        return _async;
    }
}

  

Why do you need a Custom Classloader?

Java Runtime loads the classes dynamically at runtime using class loaders. By default a JVM will consists of these three class loaders:  BootStrap Class loader This is the classloader which is loads all the core classes required by the JVM. They are basically the Java classes from the packages like java.lang.*, java.io.*, java.util.* etc. One can append additional class locations to this class loader using -Xbootclasspath/p: and -Xbootclasspath/a: JVM options. Extension Class loader It is the classloader which is an alternative to the BootStrap class loader. One can drop the additional classes packaged in JAR file  in /lib/ext folder to get it picked by JVM. System Class loader It is the classloader  which loads the classes specified in the CLASSPATH. It is also called as application classloader. JVM allows you  to create custom class loader which can address several problems. They are mostly useful in larger architectures consisting of several module/applications. Here are the advantages of the custom class loader: Modular architecture They allow you to define multiple class loader allowing modular architecture. Each module can have their own classloader. Isolation Clearly defines the scope of the class to within the class loader. It avoid conflicts if a class is present in two modules. Support Versioning Supports different versions of class within same VM for different modules. Memory Management Classes can be loaded and unloaded whenever required. Load classes from anywhere Classes can be loaded from anywhere, for ex, Database, Networks, or even define it on the fly. Add resources or classes dynamically All the above features allows you add classes or resources dynamically. Runtime Reloading Modified Classes Allows you to reload a class or classes at runtime by creating a child class loader to the actual class loader, which contains the modified classes. A good example of Custom class loader is JavaEE Application Server. Where each Web or Enterprise application will be class loader by itself.

FileUpload using Servlet 3.0

Servlet 3.0 container's has standard support for multipart data. If a request Content-Type is multipart/form-data and the MultiPartConfig is configured on the servlet,  the HttpServletRequest makes the parts available using Servlet 3.0 newly added methods in HttpServletRequest.

First, lets see the new API available in Servlet 3.0 :

javax.servlet.http.HttpServletRequest (API Documentation)

• Collection<Part> getParts()
• Part getPart(String partName)

javax.servelt.Part (API Documentation)

•  void  delete()                                               
•  String  getContentType()                 
•  String  getHeader(String name)                                                              
•  Collection<String>  getHeaderNames() 
•  Collection<String>  getHeaders(String name) 
•  InputStream  getInputStream()                       
•  String  getName()                                        
•  long  getSize()       
•  void  write(java.lang.String fileName)  

Lets us try to understand the API by writing an example.

The  first thing in this example is the upload page. This is how it looks:

<%@ page contentType="text/html;charset=UTF-8" language="java" %>
<html>
<head><title>Upload </title></head>
<body>
<form action="uploadservlet" method="post" enctype="multipart/form-data">
    Name :<input type="text" name="name"/><br>
    Age :<input type="text" name="age"/><br>
    Upload Photo:<input type="file" name="photo"/><br>
    <input type="submit"/>
</form>
</body>
</html>

Next is implementing the Upload Servlet. Here is how it looks:

@MultipartConfig(location = "D:\\images", maxFileSize = 102400, maxRequestSize = 204800,

                          fileSizeThreshold = 1024)
@WebServlet(name = "UploadServlet",urlPatterns = "/uploadservlet")public class UploadServlet extends HttpServlet
{
    protected void service(HttpServletRequest request,
                           HttpServletResponse response) throws ServletException, IOException
    {
        Collection<Part> parts = request.getParts();
        for (Part part : parts) {
            System.out.println("" + part.getName());
        }
        System.out.println(parts.size());
        if (parts.size() != 3) {
            response.sendError(500, "All Fields are mandatory. ");
        }

        Part filePart = request.getPart("photo");

        //read imageInputStream if required
        InputStream imageStream = filePart.getInputStream(); 

        

        //can also write the photo to local storage
        filePart.write("users/photo.jpg");
        filePart.delete(); //deletes the underlying storage if used

        //Read name, String Type
        Part namePart = request.getPart("name");

        //Part.getSize() returns the length of the individual part
        if (namePart.getSize() > 20) {
            //write name cannot exceed 20 chars
            response.sendError(500, "Name cannot exceed 20 chars");
        }
        //name , String type can also obtained using Request parameter
        String nameParameter = request.getParameter("name");

        int ageParameter = Integer.parseInt(request.getParameter("age"));

        response.getWriter().print("Welcome, " + nameParameter + "!. Your age is " + ageParameter);

    }

}

We will now discuss the above example in detail.

The HttpServletRequest will make the parts available using the getPart(..) and getParts() methods only if the MultiPartConfig is defined for that Servlet. It can be done in two ways:

• Using @MultiPartConfig annotation 
• In web.xml, by adding <multipart-config> entry inside <servlet> definition.

        <servlet>

           <servlet-name>UploadServlet</servlet-name>

           <servlet-class>com.fileupload.UploadServlet</servlet-class>

           <multipart-config>

               <file-size-threshold>1024</file-size-threshold>

               <location>D:\\images</location>

               <max-file-size>102400</max-file-size>

               <max-request-size>204800</max-request-size>

           </multipart-config>

        </servlet>

Lets discuss the properties MultiPartConfig has in details. These properties can be defined both for Annotation and in web.ml.

• file-size-threshold/fileSizeThreshold : The size beyond which the file is written to temporary file in disk.
• location : The location where the parts written using Part.write() will be stored.
• max-file-size/maxFileSize: The maximum size allowed for uploaded file.
• max-request-size / maxRequestSize :  The maxim size allowed for multipart/form-data requests.