Protecting your Application from other Applications

Scenario: While servicing a HTTP request, you call an external Application via a plain old HTTP Post of an XML String.

During testing etc. everything goes swimmingly. However in production, this other service gets super slow -- and worse yet, hangs -- so -- in effect the Thread servicing your HTTP request is hung. Your end users' browsers are just spinning away. They happen to be a persistent bunch, they close their browser and retry again and again -- until your server simply ups and dies :(

You'll probably agree that is not a good scenario. Particularly in today's world with its emphasis on availability.

(Question: What is the timeout in IE before it times out? Answer at the end of the post.)

You can/should take precaution against this scenario. First, investigate the possibility of setting a timeout on your HTTP Post. If you are using java.net.HttpURLConnection you can set timeouts in Java 1.5 and on.

Second -- consider limiting the number of threads engaged at any one time in this activity. Here is the outline of a simple technique to do this --Assuming you are using the same multi-threaded instance to do your service call. Again, this is not meant to be working code:

public class ServiceImpl {
ConcurrentHashMap map = new ConcurrentHashMap();

public String send(....) {
if (map.size() > 25) throw new RuntimeException("Max Threads exceeded");
try {
map.put(Thread.currentThread(),start);
... call service..
return response;
}
finally {
map.remove(Thread.currentThread());
}
}

In practice you could also keep counts on the number of times the service was called, threw an exception etc. and use a restricted jsp to examine these values etc. If you 'practice' AOP, this would be a a great application of it.

If you are wondering how to arrive at the max number of threads to allow for a specific service -- ask this question -- what is the maximum percentage of total threads am I willing to dedicate to this part of my application? If the max number of threads in your container is configured at say 500 and your answer is 5%, then this number would be 25. In practice you would make this value configurable.

I actually implemented some counters in the above algorithm, including a 'max concurrent request' counter -- so I did get an idea of what the number could be for a specific back-end service call.

(Answer to the question posed earlier -- Five Minutes)