[mod_python] Mod_Python not releasing memory?

2008/8/19  <Scott.Chapman at verizonwireless.com>:
> I have Apache/2.2.8 (Unix) mod_python/3.3.1 Python/2.4.5 on Red Hat
> Enterprise Linux AS release 4 (Nahant Update 6).
> Apache compiled:
> ./configure --with-mpm=prefork --enable-mods-shared="most proxy
> charset_lite" --with-expat=system
>
> Mod Python:
> ./configure --with-apxs=/usr/local/apache2/bin/apxs
> --with-python=/usr/local/bin/python2.4
>
> Python:
> ./configure --enable-shared
>
> My application is using Cheetah to generate a very large HTML table and
> sending it back to the client, after retrieving a bunch of database rows.
>
> One hit to Apache to generate one of these big tables runs memory usage up
> to 30.6% of system RAM:
>
> $ ps aux | grep 30316 | grep -v grep
> USER       PID %CPU %MEM   VSZ  RSS TTY      STAT START   TIME COMMAND
> web      30316  2.7 27.7 1559948 1122772 ?   Rl   12:48   1:21
> /usr/local/apache2/bin/httpd -k start
>
> $ ps aux | grep 30316 | grep -v grep
> USER       PID %CPU %MEM   VSZ  RSS TTY      STAT START   TIME COMMAND
> web      30316  2.7 30.6 1559948 1237948 ?   Rl   12:48   1:21
> /usr/local/apache2/bin/httpd -k start
>
> After the query is done and the results delivered to the browser, the Apache
> process is still holding about 25% of system memory.
>
> I don't see any way to free that memory.
> Once I "return result" to the browser, I can't do "result=None" so I'm
> relying on Python to release the memory and it is taking way too long for
> that to happen.  I do set the database result set to None as soon as I can
> and that helps some.
>
> If I get a number of these big hits, OOM starts killing PIDs which is a "Bad
> Thing".
>
> Any clues on how to make that memory release sooner?

This is normal behaviour. You need to do some reading/research on how
memory is used in operating systems.

In short though, in general once a process needs to allocate memory
from the operating system it is then marked as being in use by that
process for the life of the process, even if the individual memory
fragments are freed by code. Such freed memory can be reused within
the same process, but isn't released back to operating system for
other processes to use.

Graham