[mod_python] Apache processes "eating up" too much memory (leak?)

Jim Gallacher jpg at jgassociates.ca
Sat Dec 17 10:53:49 EST 2005


Sébastien Arnaud wrote:
> Thank you Graham and Jim for the pointers!
> 
> I guess I was directly affected by the bug in question in 3.1.4 !  Since 
> I was running 2 apps as virtual hosts, 10 PythonOption  instructions per 
> app, I also have 1 Load Balancer in front of those  app servers making 1 
> request each minute to poll each app, it was  making me loose 2x10x25 = 
> 500 bytes / min !

That would do it. ;)

> As soon as 3.2 is golden, I will grab it and run it on our test farm  
> before I push it out to our prod environment, sounds like 3.2 is full  
> of sweet bug fix and enhancements! Any idea when it will be  officially 
> released?

The 3.2.5b has been out in the wild since Nov 15. We had planned on a 
final release this week, but fixing MODPYTHON-99 may mean another beta. 
Note that this issue likely existed in 3.1 as well, but was only 
recently found. Personally I wouldn't hesitate to start using 3.2.5b on 
your test farm.

See http://issues.apache.org/jira/browse/MODPYTHON-99 for details.

> Also, I noticed that Apache had released 2.2 officially, how does  
> mod_python 3.2 handles this, will it work on apache 2.2? I am asking  
> this, because I just did a port sync on my Mac and I just saw that  
> apache 2.2 was already ready to replace my 2.0.55 install!

As Jorey mentioned, mp 3.2 doesn't support apache 2.2. We made the 
decision a few months ago to defer apache 2.2 support to mod_python 3.3. 
At that time we actually thought we'd have 3.2 out before apache 2.2 was 
available. Oh well.... :(.

Jim

> Thanks again for the help!
> 
> Sébastien Arnaud
> eMedia Library, inc
> sebastien at emedialibrary.org
> 
> 
> 
> On Dec 15, 2005, at 5:07 PM, Graham Dumpleton wrote:
> 
>> FWIW, you might consider upgrading to mod_python 3.2.5b if you can. I
>> believe that that version eliminates a few memory leaks in certain
>> circumstances in mod_python itself, although I don't remember the
>> details. This may or may not be contributing. My understanding was  that
>> the mod_python leaks weren't that great, but if you are handling a lot
>> of requests it may be noticeable. You also may want to configure  Apache
>> to recycle child processes after a certain number of requests to avoid
>> overblown processes.
>>
>> Sorry, I know this doesn't really help with tracking down the  source of
>> any problem if in your code. :-)
>>
>> Graham
>>
>> =?ISO-8859-1?Q?S=E9bastien_Arnaud?= wrote ..
>>
>>> Hello!
>>>
>>> I am running mod_python on 2 identical production server for 2 small
>>> apps I wrote which basically render over 100 of different SVG charts.
>>> I am running Gentoo on those servers, mod_python 3.1.4, apache 2.0.55
>>> in prefork mode, each server has 1 Go of mem.
>>>
>>> Both applications have been in beta testing for like 1 month or so,
>>> and I just noticed today that the memory usage was really high for
>>> each apache process running (max of 10 set in the conf file). When I
>>> started those apache process 30 days ago they stabilize at 2.0 % of
>>> memory used, but somehow after 30 days of users testing the app, it
>>> went to 10% per apache process, which made the swap usage go to 1Go
>>> (over 2Go avail). I am doing quite a lot of caching of XSL files in
>>> the app, but this is suppose to happen at launch time, meaning I
>>> instantiate all the objects I need at the first call of the
>>> mod_python handler and they persist through the life of those apache
>>> processes. Does somebody know anyway to inspect each apache process
>>> to see where the memory is being used? Basically I am trying here to
>>> figure out if I have some kind of memory leak going on, or if there
>>> is something I miscalculated when caching the large amount of XSL
>>> files in persistent Python objects.
>>>
>>> Apart from that, those 2 little applications fly in terms of speed! A
>>> lot of people (including me) got really blown away to witness the
>>> speed of those 2 apps on those "old" servers (which are quad PIII
>>> 550Mhz btw).
>>>
>>> Thank you in advance for any insights so that I can get to the bottom
>>> of this!
>>>
>>> Cheers!
>>>
>>> Sébastien Arnaud
>>> eMedia Library, inc
>>> sebastien at emedialibrary.org
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Mod_python mailing list
>>> Mod_python at modpython.org
>>> http://mailman.modpython.org/mailman/listinfo/mod_python
>>
>> _______________________________________________
>> Mod_python mailing list
>> Mod_python at modpython.org
>> http://mailman.modpython.org/mailman/listinfo/mod_python
> 
> 
> 



More information about the Mod_python mailing list