Request Queuing on Engine Yard explained (SSL Edition)

Post featured image

We believe that performance is a critical measurement for any web application. Users tend to use fast applications more and are in general more satisfied when it feels snappy. The statement "performance is a feature" says it all (and for example page speed is a factor in Google's ranking, for a reason). That is why we measure many different metrics to know how LingoHub "feels" like. In our constant battle to make LingoHub faster, stronger, better, we always came across "Request Queuing" time:

lingohub-new-relic

Requesting Queuing explained in simple terms

"The time the request spends before actually being processed." In our case, the time the request sits on NGINX and waits to get processed by Rails. There are many good articles that explain it in much more detail and better and more accurately than I did:

Request Queuing, performance and SSL encryption

As you can see in the above diagram, request queuing takes almost as much time as the actual request processing. Seems like a low hanging fruit, right? So something can be done there. Queuing means that there are not enough Rails instances to handle our requests. Hence throwing in more servers to handle the load will fix the problem. Sadly this wasn't the case, because in our investigation with the help of EngineYard Support (who are awesome by the way), we realized that the reason behind this constant request queuing is usage of HTTPS for all requests. In times like these (Hi there PRISM) encryption is a reliable way to keep out prying eyes. LingoHub was 100% SSL secured from the get go. A secured connection requires several round-trips because of the required handshake. Such a handshake takes around 50-300ms depending on the location, ping time, etc. (worse for mobile). So the request is waiting at NGINX to finish the handshake before it is getting delivered to Rails.

So what can we do? Certainly not sacrificing security over performance, instead we have to look at other places to get out more performance. We will keep you posted on our battle for performance. Please share your ideas in the comments as well.