Answer
Hoo boy, that's a tough one.
First off, you should always do your own exception handling. An uncaught exception can silently kill your servlet, and if you don't know where to look in the log files, or if your server has a bug in it whereby it silently swallows certain exceptions, you'll have no idea where the trouble is.
The following code sets up a catch block that will trap any exception, and print its value to standard error output and to the ServletOutputStream so that the exception shows up on the browser (rather than being swallowed by the log file). Chances are that any error is in your code; the exception shows you what line the problem happened at. (If you see "Compiled Code" instead of line numbers in the exception stack trace, then turn off the JIT in your server.)
Code
Lately, I've started catching all Throwables, just in case. I know this is bad form but what are you gonna do?
Next, you should make liberal use of the log() method, and you should keep your own log files. Again, don't trust the server to do the right thing. Also, printing the log after every request can help debugging because you can immediately see the output of your servlet without going into the server-side log files. (Another problem this avoids is that some servlet engines forget to flush their logs after each request, so even if you go to the server, you won't see the most recent log messages.)
Here's some source code you can add to any HttpServlet that keeps an in-memory log:
Code
And here's some code that you can add to the end of your doGet or doPost method that prints out the entire log after each request:
Code
Both of these should be disabled once you actually ship your servlet, but they can be very useful during development and debugging.
You should remember to use servletrunner (renamed "JSDK WebServer" with the JSDK version 2 -- run with the script startserver) to debug your servlet. It's a tool that ships with the JSDK that basically starts up a miniature web server that runs your servlet inside itself. It means you don't have to stop and restart your real web server every time you recompile your servlet. It also affords a more pure environment, so you can make sure your servlet truly conforms to the spec before you try to run it inside a (possibly buggy or nonstandard) servlet engine. (Note that Tomcat does not have a replacement for servletrunner :-(.)
A few IDEs support servlet debugging. Symantec Cafe claims to have a fairly robust system for doing visual source-level debugging of servlets (as well as RMI, CORBA, and EJB objects). [If anyone has any experience with Cafe or other IDEs, please email &feedback;.] May Wone (abc408@hotmail.com) writes:
I am debugging servlets running servletRunner inside of IBM's VisualAge for Java.
On my first servlet project, I used a ton of log messages for debugging.
This is my second servlet project, and this debugging technique has enhanced my productivity many folds. I am able to set code break points, step through each line of code, inspect all the objects visible to this class as well as view the objects in the current stack. I can also view, suspend, resume threads.
The setup instructions are at: (http://www.ibm.com/java/education/server-side/server-side.html)
jRun also claims to have excellent log file support as well as some debugging facilities.
Eric Gilbertson (eric@bitsource.com) adds:
Another way to debug servlets is to invoke JWS with java_g and then attach to the process using any debugger that is capable of attaching to a running Java process given a debug password. To do this invoke JWS as follows:
This will start the Web server process and echo a password. The password may be then used with jdb to attach to the process. Note that this will start only the Web server, not the admin server.
Sun does not advertise this mechanism which in my mind is the only way to debug non-trivial servlets.