[mod_python] High traffic recommendations

Marc D. Murray wrote:
> Thank you all for Your tips and responses. 
> 
> 
>>Is this server live on the internet or in a stand alone lab?
>>Is it possible that there maybe a new virus attaching your machine
>>when it connected to the net
> 
> It's live on the 'net. But I hope it's not a virus on Debian. Cuz I have
> no anti-virus on any of my Debian Boxes :)
> 
> 
> 
>>And what do you get in error logs? Does it crash while processing your
>>code or on directly in Apache?
>>
>>Where did you see DBNoMemory and who reported it? What drivers do you
>>use and how do you handle db connections?
> 
> The  DB Errors were from the bsddb.DBEnv runninng out of connections.
> Not sure why this would happen as it was supposedly designed for many
> connections. 
> But I'm currently using a global bsddb.DBEnv connection to a folder that
> holds many *.bdb files. These are opened stuffed into a queue. The are
> closed if older than 2 hrs.
> I don't get these errors  anymore. 
> 
> 
> 
>>How does Apache behave WRT memory ? There are known memory leaks in
>>mod_python version prior to 3.2.7, so 
> 
> Each apache process hovers between 3.2 and 4.7 % (readin from top). I
> ran some tests on my local server using /usr/sbin/ab, but it never went
> down. Not even when requests set to 50000. 
> 
> 
>>I would advise you to upgrade to 3.2.7 if you can
> 
> Now this is something I've been thinking of. I'll look into it later
> tonite. 
>  
> 
> 
>>>      MaxRequestsPerChild  500
>>
>>Make that 10000 or so. This low a number is only required if your
>>system is leaking lots of resources (which mod_python does not do).
> 
> Now I'm confused. Does it leak or not? How do I test for that?

It depends on how you define "lots". There is an issue in 3.1.4 with the 
use of PythonOption causing a leak. Each instance of PythonOption in 
your apache configuration leaks 25 bytes for each request. This will 
obviously only become a problem if your MaxRequestPerChild set at a very 
high number (or 0).

IIRC the other issue was a problem with garbage collecting circular 
references for the request object. Nicolas may have a better 
recollection, but it seems to me this was mainly a problem when 
registering a cleanup that held a reference to the request object. This 
was a more serious leak and could cause problems in the sort of time 
frame you are seeing.

There can also be related issue if a segfault occurs in the cleanup 
phase as a result of holding a reference to the request object which has 
already been garbage collected. The client would still see the result 
and not be aware of the problem, but the cleanup code never closes the 
db connection because it segfaults. A segfault would also explain why 
your memory use seems stable. It's sort of like having a really low 
MaxRequestsPerChild setting. ;)

A segfault in the cleanup code is obviously a real performance killer, 
so check that code carefully. As Nicolas suggested, you may want to post 
the code for your DBMSession subclass.

At any rate, 3.2.7 fixes these issues, and an upgrade is in order.

Jim