Performance tuning specialist Kirk Pepperdine has shared one of his own approaches for starting to find the true cause of performance problems: "the box" - a little diagram that is an abstraction of a complete system. The box is a reminder of the true causes of performance bottlenecks.
Read The Box: A Shortcut to finding Performance Bottlenecks.
The has 4 layers: People, Application, Java Virtual Machine, and Hardware. Each of these layers has a role in systems performance, and Kirk talks about when The purpose of the box is to eliminate guessing by showing us how to sequence an investigation.
Kirk advocates a bottom to top investigative process, the goal of which is to eliminate as many potential sources for the bottlenecks as can possibly eliminated before we start looking at the code.