|
stietke at diw.de
stietke at diw.de
Tue Apr 13 12:55:34 EST 2004
Hi guys,
thank you for this extremely cool piece of software. I am very
happy with it. However there is one little problem I encountered
while playing with it. On my computer, the apache process seems
to leak memory when used with mod_python in certain configurations.
In particular, it seems to leak memory when I use PythonOption
directives in a <Directory>...</Directory> context.
I tried to figure out where the memory disapears using valgrind,
but the hundreds of lines valgrind spit out left me rather clueless.
Maybe somebody here has any idea what is going on.
Thank you in advance + all the best,
Stefan
summary:
--------
Some mod_python directives, including PythonOption and PythonInterpreter,
seem to cause apache or mod_python to lose a certain, constant amount
of memory for each request mod_python handles. The number of bytes
lost each request seems to scale with the number of directives used.
The apache process seems to lose memory when the mod_python module
is loaded with a certain configuration, independently of the actual
request beeing handled by mod_python. (see results 6 for details)
versions:
---------
apache 2.0.49
mod_python 3.1.3
python 2.3.3
linux kernel 2.4.19
libc6 2.3.1
debian woody with some updates
apache, python and mod_python are compiled from source.
apache is compiled with prefork mpm to make testing easier (no threads),
but tests with worker mpm showed similar results in earlier tests.
httpd.config:
-------------
$ diff conf/httpd.conf conf/httpd-std.conf
119,122c119,122
< StartServers 1
< MinSpareServers 1
< MaxSpareServers 1
< MaxClients 1
---
> StartServers 5
> MinSpareServers 5
> MaxSpareServers 10
> MaxClients 150
219c219
< Listen 127.0.0.1:10080
---
> Listen 80
233d232
< LoadModule python_module
/home/stefan/apache_test/apache_prefork/modules/mod_python.so
362,372d360
< #
< # mod_python
< #
< AddHandler mod_python .py
< PythonHandler testdummy
< #PythonInterpreter pylet_interp
< #PythonDebug On
< #PythonOption x a
< #PythonOption y b
< #PythonOption z c
<
493c481
< LogLevel debug
---
> LogLevel warn
mod_python handler:
-------------------
$ cat testdummy.py
def handler(req):
return 0
testprocedure:
--------------
1) warm up: ../apache/bin/ab -n 50000 -k http://localhost:10080/test.py
2) test: cat /proc/2315/status > httpd_status.txt;
for x in 1 2 3 4 5; do
../apache/bin/ab -n 100000 -k http://localhost:10080/test.py;
cat /proc/2315/status >> httpd_status.txt;
done
3) see if the worker process grows in size: grep VmSize: httpd_status.txt
results:
--------
1) with the mod_python configuration from above, no growth.
2) with the line "PythonOption x a" enabled:
VmSize: 8776 kB
VmSize: 10348 kB ---> +1572
VmSize: 11916 kB ---> +1568
VmSize: 13488 kB ---> +1572
VmSize: 15056 kB ---> +1568
VmSize: 16624 kB ---> +1568
total: ((16624 - 8776) * 1024) / 500000 = 16.07270 bytes/request
3) with the PythonOption replaced with "PythonOption
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
(that is 100*x and 100*a):
result: exactly the same as 2).
4) with "PythonOption x a" and "PythonOption y b" enabled:
VmSize: 9956 kB ---> +3924
VmSize: 13880 kB ---> +3924
VmSize: 17804 kB ---> +3924
VmSize: 21728 kB ---> +3932
VmSize: 25660 kB ---> +3924
VmSize: 29584 kB
total: ((29584 - 9956) * 1024) / 500000 = 40.19814 bytes/request
5) with additionally "PythonOption z c" enabled:
VmSize: 11532 kB
VmSize: 18620 kB ---> +7088
VmSize: 25708 kB ---> +7088
VmSize: 32792 kB ---> +7084
VmSize: 39880 kB ---> +7088
VmSize: 46968 kB ---> +7088
total: ((46968 - 11532) * 1024) / 500000 = 72.57292 bytes/request
6) with all PythonOptions disabled, but with
"PythonInterpreter pylet_interp" enabled:
result: exactly the same as 2).
BUT: after this test, i ran
../apache/bin/ab -n 300000 -k http://localhost:10080/index.html.en
without restarting apache, and the VmSize grew again to 21328 kB.
((21328 - 16624) * 1024) / 300000 = 16.05632
Apparently, apache loses memory already when the mod_python
module is loaded but does not handle the request at hand.
|