[mod_python] mod_* performance

Guys,

I had been blaming bad performance with concurrent connections on
mod_python, when in fact an Apache config change can make all the
difference. The key is to limit the number of Apache processes based on
the RAM and average concurrency load that you have.

I found the mod_perl suffers from the same performance characteristics
becuase it's Apache behavior that impacts the performance on
simultinious connections, not mod_python or mod_perl.

I changed the MaxClients parameter in apache2.conf from 50 to 8:

<IfModule prefork.c>
StartServers         1
MinSpareServers      1
MaxSpareServers     10
MaxClients           8
MaxRequestsPerChild  0
</IfModule>

7k mod_python page with SQL caching:

Reqests  Concurrency   Req/Sec  Connect Time
400      1             136.54   7.324
400      5             131.29   38.085
400      15            126.28   118.780
400      50            96.73    516.883 
400      100           71.92    1390.431

To quote Stas Bekman <stas at stason.org>
"""
MinSpareServers, MaxSpareServers and StartServers are only important for
user response times. Sometimes users will have to wait a bit.

The important parameters are MaxClients and MaxRequestsPerChild.
MaxClients should be not too big, so it will not abuse your machine's
memory resources, and not too small, for if it is your users will be
forced to wait for the children to become free to serve them.
MaxRequestsPerChild should be as large as possible, to get the full
benefit of mod_perl, but watch your server at the beginning to make sure
your scripts are not leaking memory, thereby causing your server (and
your service) to die very fast.
"""

7k PHP page with standard connection pooling:

Reqests  Concurrency   Req/Sec  Connect Time
400      1             29.60    33.785
400      5             15.32    65.291
400      15            23.13    43.242
400      50            21.45    2656.869
400      100           23.21    4308.698

What we see here is that connect time (which translates into perceived
latency) really suffers in PHP when you hit it with a large number of
concurrent connections.

This one little tweak makes mod_perl and mod_python immune to http DOS
attacks. With the poper coding an increase of 300%-350% is possible over
PHP.

Eric Radman