[mod_python] Apache and mod_python memory usage

Eugene Morozov wrote ..
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hello,
> We're using apache and mod_python for running Pylons-based CMS. We're a
> little bit concerned with memory usage. With mod_python
> apache2-mpm-worker consumes a lot of memory:
> 
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> 23787 www-data 20 0 366m 48m 4104 S 0.0 12.5 0:09.23 apache2
> 
> 48Mb is a bit too big even for apache. And why it needs 366Mb of virtual
> memory?
> 
> If I disable mod_python these values are cut significantly (to 266Mb
> virtual and 21Mb RSS, which is still much, it seems).
> 
> Not that it's a huge problem now, but I'm curious why so much memory is
> consumed.

What are the figures immediately after Apache has started and before any requests
have been handled by mod_python? When determining this, ensure that not
PythonImport directives are being applied to preload/execute any Python code.

This figure will be the initial starting size with only the main Python interpreter
instance created. Beyond that, the size will grow based on what your own Python
code is doing, how many additional Python interpreter instances are created or
possibly due to memory leaks.

As far as memory leaks in mod_python go, ensure that you are using the latest
version of mod_python, ie., 3.3. There are memory leaks in older versions which
depending on what your code is doing may be getting triggered.

As to Python interpreter instances, how many virtual hosts are you running on
the server under which mod_python requests are being handled? Also, are you
using any of PythonInterpPerDirectory or PythonInterpPerDirective or
PythonInterpreter directives? If you are using these and depending on how
you use them, you could be unnecessarily creating additional Python interpreter
instances which may be unnecessarily consuming additional memory.

Finally, does this figure keep going up and up over time which may indicate
memory leaks in mod_python or your own Python code?

Graham

PS. I finally have net access again. Been trying to get an ADSL connection going
where I am staying for most of the week. Finally came down to fact that ISP didn't
tell me or state anywhere on their sparse documentation what the correct
@something was that had to be placed after the ADSL login ID. Only after I asked
them a second time did they give me the correct value. Even though back on net,
access still restricted. Luckily list has been quiet anyway. :-)