Web server torturing

Hi, Due to the web server hit faced with FC6 release, some actions are being taken to minimize the chance of facing such issues again. One of the steps is to stress test our web-server infrastructure to measure our current and future capabilities. I'd like to run some tests on fp.o web-server, please let me know your thoughts/comments. Here are some details. Test Targets: 1- Measure our bare (no caching) maximum serving rate 2- Measure our cached serving rate, to assess the implemented caching efficiency 3- Gather numbers like (When do we get CPU saturated, RAM requirements ..). Possibly draw graphs (everyone thinks graphs are cool), the numbers should help us base future calculations on a solid basis 4- Future: Possibly implement a mechanism to cap the maximum connected clients to a specific server, to the maximum it can handle gracefully, to avoid killing a server Test Plan: 1- A script was written which uses apache's ab tool to stress the server. Script will run on the web-server host. 2- The script fires a total number of connections equal to ten times the maximum concurrency rate (to get good average, and avoid transients) 3- The concurrency rate is sweeped between 10 and 400 (my 1G-RAM machine swaps at about 100 connections)? any suggestions? 4- All ab output is recorded for future analysis 5- A monitoring thread is fired before ab is launched. The monitoring uses "top" to record load/cpu/ram/process information in log files as well 6- Tests are repeated with "ab -k" for enabling the HTTP keep-alive option. Not sure if this is needed, or if it will make much difference! comments? 7- Tests are done once with caching enabled and one more time without caching Please let me know your thoughts about the testing setup, should we be recording more data? should we be stressing the server in a different way, should we be testing some apache config options ... etc ? Thanks