[mod_python] memory leak in mod_python?

Gregory (Grisha) Trubetskoy grisha at modpython.org
Tue Apr 13 09:52:08 EST 2004


Thanks for the detailed report, we'll check it out.

Grisha

On Tue, 13 Apr 2004 stietke at diw.de wrote:

> Hi guys,
>
> thank you for this extremely cool piece of software. I am very
> happy with it. However there is one little problem I encountered
> while playing with it. On my computer, the apache process seems
> to leak memory when used with mod_python in certain configurations.
>
> In particular, it seems to leak memory when I use PythonOption
> directives in a <Directory>...</Directory> context.
>
> I tried to figure out where the memory disapears using valgrind,
> but the hundreds of lines valgrind spit out left me rather clueless.
>
> Maybe somebody here has any idea what is going on.
>
> Thank you in advance + all the best,
> Stefan
>
>
> summary:
> --------
> Some mod_python directives, including PythonOption and PythonInterpreter,
> seem to cause apache or mod_python to lose a certain, constant amount
> of memory for each request mod_python handles. The number of bytes
> lost each request seems to scale with the number of directives used.
>
> The apache process seems to lose memory when the mod_python module
> is loaded with a certain configuration, independently of the actual
> request beeing handled by mod_python. (see results 6 for details)
>
> versions:
> ---------
> 	apache 2.0.49
> 	mod_python 3.1.3
> 	python 2.3.3
> 	linux kernel 2.4.19
> 	libc6 2.3.1
> 	debian woody with some updates
>
> apache, python and mod_python are compiled from source.
>
> apache is compiled with prefork mpm to make testing easier (no threads),
> but tests with worker mpm showed similar results in earlier tests.
>
> httpd.config:
> -------------
> $ diff conf/httpd.conf conf/httpd-std.conf
> 119,122c119,122
> < StartServers         1
> < MinSpareServers      1
> < MaxSpareServers      1
> < MaxClients           1
> ---
> > StartServers         5
> > MinSpareServers      5
> > MaxSpareServers     10
> > MaxClients         150
> 219c219
> < Listen 127.0.0.1:10080
> ---
> > Listen 80
> 233d232
> < LoadModule python_module
> /home/stefan/apache_test/apache_prefork/modules/mod_python.so
> 362,372d360
> <     #
> <     # mod_python
> <     #
> <     AddHandler mod_python .py
> <     PythonHandler testdummy
> <     #PythonInterpreter pylet_interp
> <     #PythonDebug On
> <     #PythonOption x a
> <     #PythonOption y b
> <     #PythonOption z c
> <
> 493c481
> < LogLevel debug
> ---
> > LogLevel warn
>
> mod_python handler:
> -------------------
> $ cat testdummy.py
> def handler(req):
>         return 0
>
> testprocedure:
> --------------
> 1) warm up: ../apache/bin/ab -n 50000 -k http://localhost:10080/test.py
>
> 2) test: cat /proc/2315/status > httpd_status.txt;
> 	 for x in 1 2 3 4 5; do
> 		../apache/bin/ab -n 100000 -k http://localhost:10080/test.py;
> 		cat /proc/2315/status >> httpd_status.txt;
> 	 done
>
> 3) see if the worker process grows in size: grep VmSize: httpd_status.txt
>
> results:
> --------
> 1) with the mod_python configuration from above, no growth.
> 2) with the line "PythonOption x a" enabled:
> 	VmSize:     8776 kB
> 	VmSize:    10348 kB	---> +1572
> 	VmSize:    11916 kB	---> +1568
> 	VmSize:    13488 kB	---> +1572
> 	VmSize:    15056 kB	---> +1568
> 	VmSize:    16624 kB	---> +1568
>    total: ((16624 - 8776) * 1024) / 500000 = 16.07270 bytes/request
> 3) with the PythonOption replaced with "PythonOption
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
> aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
> (that is 100*x and 100*a):
> 	result: exactly the same as 2).
> 4) with "PythonOption x a" and "PythonOption y b" enabled:
> 	VmSize:     9956 kB	---> +3924
> 	VmSize:    13880 kB	---> +3924
> 	VmSize:    17804 kB	---> +3924
> 	VmSize:    21728 kB	---> +3932
> 	VmSize:    25660 kB	---> +3924
> 	VmSize:    29584 kB
>    total: ((29584 - 9956) * 1024) / 500000 = 40.19814 bytes/request
> 5) with additionally "PythonOption z c" enabled:
> 	VmSize:    11532 kB
> 	VmSize:    18620 kB	---> +7088
> 	VmSize:    25708 kB	---> +7088
> 	VmSize:    32792 kB	---> +7084
> 	VmSize:    39880 kB	---> +7088
> 	VmSize:    46968 kB	---> +7088
>    total: ((46968 - 11532) * 1024) / 500000 = 72.57292 bytes/request
> 6) with all PythonOptions disabled, but with
>    "PythonInterpreter pylet_interp" enabled:
> 	result: exactly the same as 2).
>    BUT: after this test, i ran
>    ../apache/bin/ab -n 300000 -k http://localhost:10080/index.html.en
>    without restarting apache, and the VmSize grew again to 21328 kB.
>    ((21328 - 16624) * 1024) / 300000 = 16.05632
>    Apparently, apache loses memory already when the mod_python
>    module is loaded but does not handle the request at hand.
>
>
>
> _______________________________________________
> Mod_python mailing list
> Mod_python at modpython.org
> http://mailman.modpython.org/mailman/listinfo/mod_python
>


More information about the Mod_python mailing list