|
Jim Gallacher
jpg at jgassociates.ca
Sat Mar 3 19:31:34 EST 2007
Graham Dumpleton wrote: > > On 04/03/2007, at 9:30 AM, Graham Dumpleton wrote: > >> The result for mod_python request is (pid 405): >> >> 410 httpd 0.0% 0:00.00 1 8 39 212K 2.87M 888K >> 30.4M >> 408 httpd 0.0% 0:00.00 1 8 39 212K 2.87M 764K >> 30.4M >> 407 httpd 0.0% 0:00.00 1 8 39 216K 2.87M 768K >> 30.4M >> 406 httpd 0.0% 0:00.00 1 8 39 216K 2.87M 768K >> 30.4M >> 405 httpd 0.0% 0:00.37 1 10 86 2.40M 2.75M 3.94M >> 32.0M >> 404 httpd 0.0% 0:00.00 1 8 39 212K 2.87M 768K >> 30.4M >> 403 httpd 0.0% 0:00.06 1 11 39 32K 2.87M 1.30M >> 30.4M >> >> Thus, private memory use by Apache child process jumps up to 2.40M. >> Ie., about an extra 2.2M. >> >> For a mod_wsgi request however, the result is (pid 407): >> >> 410 httpd 0.0% 0:00.00 1 8 39 212K 2.87M 816K >> 30.4M >> 408 httpd 0.0% 0:00.00 1 8 39 212K 2.87M 764K >> 30.4M >> 407 httpd 0.0% 0:00.01 1 10 39 568K 2.87M 1.64M+ >> 30.4M >> 406 httpd 0.0% 0:00.00 1 8 39 216K 2.87M 768K >> 30.4M >> 405 httpd 0.0% 0:00.37 1 10 86 2.40M 2.75M 3.95M >> 32.0M >> 404 httpd 0.0% 0:00.00 1 8 39 212K 2.87M 768K >> 30.4M >> 403 httpd 0.0% 0:00.06 1 11 39 32K 2.87M 1.30M >> 30.4M >> >> That is, private memory use by Apache child process only jumps up to >> 568K. >> Ie., about an extra 350K. >> >> This is quite a big difference and really makes one wander what >> mod_python >> is actually doing. This will also have to be investigated, but >> probably still >> want to pursue this shared versus static library thing first. > > Part of the jump is the imports in mod_python.apache: > > import sys > import traceback > import time > import os > import pdb > import stat > import imp > import types > import cgi > import _apache > import threading > > I had been thinking they would have been done when mod_python loaded > for first interpreter, but they aren't done until time of first request. > > If I import the same modules into mod_wsgi example, then get (pid 429): > > 432 httpd 0.0% 0:00.00 1 8 39 212K 2.87M 904K > 30.4M > 430 httpd 0.0% 0:00.00 1 8 39 212K 2.87M 776K > 30.4M > 429 httpd 0.0% 0:00.18 1 10 67 1.95M 2.88M 3.33M > 31.5M > 428 httpd 0.0% 0:00.00 1 8 39 216K 2.87M 780K > 30.4M > 427 httpd 0.0% 0:00.00 1 8 39 216K 2.87M 780K > 30.4M > 426 httpd 0.0% 0:00.00 1 8 39 212K 2.87M 780K > 30.4M > 425 httpd 0.0% 0:00.07 1 11 39 32K 2.87M 1.34M > 30.4M > > Thus these modules alone causes it to jump up to 1.95M. This works out > about an extra 1.4MB. Because mod_wsgi is all in C code aren't relying on > any Python modules so base overhead is less. > > Interesting. How many of these modules could be avoided in mod_python > if we tried hard? I have no idea, but it's worth a look. Attached is a fun little script that uses pmap to examine the changes. You'll need to modify the $URL in the script so it points to an appropriate mod_python script. Results for before and after running a helloworld.py script: --- before/30006.log 2007-03-03 18:59:20.000000000 -0500 +++ after/30006.log 2007-03-03 18:59:20.000000000 -0500 @@ -2,7 +2,40 @@ Address Kbytes Mode Offset Device Mapping 08048000 364 r-x-- 0000000000000000 003:00004 apache2 080a3000 12 rw--- 000000000005a000 003:00004 apache2 -080a6000 1792 rw--- 00000000080a6000 000:00000 [ anon ] +080a6000 2592 rw--- 00000000080a6000 000:00000 [ anon ] +a9cd4000 4 r-x-- 0000000000000000 003:00004 _weakref.so +a9cd5000 4 rw--- 0000000000000000 003:00004 _weakref.so +a9cd6000 4 r-x-- 0000000000000000 003:00004 _bisect.so +a9cd7000 4 rw--- 0000000000001000 003:00004 _bisect.so +a9cd8000 260 rw--- 00000000a9cd8000 000:00000 [ anon ] +a9d19000 12 r-x-- 0000000000000000 003:00004 sha.so +a9d1c000 4 rw--- 0000000000002000 003:00004 sha.so +a9d1d000 8 r-x-- 0000000000000000 003:00004 md5.so +a9d1f000 4 rw--- 0000000000002000 003:00004 md5.so +a9d20000 388 rw--- 00000000a9d20000 000:00000 [ anon ] +a9d81000 56 r-x-- 0000000000000000 003:00004 datetime.so +a9d8f000 12 rw--- 000000000000d000 003:00004 datetime.so +a9d92000 260 rw--- 00000000a9d92000 000:00000 [ anon ] +a9dd3000 16 r-x-- 0000000000000000 003:00004 struct.so +a9dd7000 4 rw--- 0000000000004000 003:00004 struct.so +a9dd8000 12 r-x-- 0000000000000000 003:00004 collections.so +a9ddb000 4 rw--- 0000000000003000 003:00004 collections.so +a9ddc000 8 r-x-- 0000000000000000 003:00004 fcntl.so +a9dde000 4 rw--- 0000000000002000 003:00004 fcntl.so +a9ddf000 8 r-x-- 0000000000000000 003:00004 _random.so +a9de1000 4 rw--- 0000000000002000 003:00004 _random.so +a9de2000 12 r-x-- 0000000000000000 003:00004 binascii.so +a9de5000 8 rw--- 0000000000002000 003:00004 binascii.so +a9de7000 12 r-x-- 0000000000000000 003:00004 math.so +a9dea000 4 rw--- 0000000000002000 003:00004 math.so +a9deb000 260 rw--- 00000000a9deb000 000:00000 [ anon ] +a9e2c000 12 r-x-- 0000000000000000 003:00004 _ssl.so +a9e2f000 4 rw--- 0000000000003000 003:00004 _ssl.so +a9e30000 36 r-x-- 0000000000000000 003:00004 _socket.so +a9e39000 12 rw--- 0000000000009000 003:00004 _socket.so +a9e3c000 20 r-x-- 0000000000000000 003:00004 itertools.so +a9e41000 8 rw--- 0000000000004000 003:00004 itertools.so +a9e43000 260 rw--- 00000000a9e43000 000:00000 [ anon ] a9e84000 36 r-x-- 0000000000000000 003:00004 libgcc_s.so.1 a9e8d000 4 rw--- 0000000000009000 003:00004 libgcc_s.so.1 a9e8e000 4 ----- 00000000a9e8e000 000:00000 [ anon ] @@ -70,6 +103,12 @@ b777b000 888 r-x-- 0000000000000000 003:00004 libpython2.4.so.1.0 b7859000 156 rw--- 00000000000de000 003:00004 libpython2.4.so.1.0 b7880000 16 rw--- 00000000b7880000 000:00000 [ anon ] +b7884000 4 r-x-- 0000000000000000 003:00004 syslog.so +b7885000 4 rw--- 0000000000001000 003:00004 syslog.so +b7886000 16 r-x-- 0000000000000000 003:00004 strop.so +b788a000 8 rw--- 0000000000003000 003:00004 strop.so +b788c000 12 r-x-- 0000000000000000 003:00004 time.so +b788f000 8 rw--- 0000000000002000 003:00004 time.so b7891000 92 r-x-- 0000000000000000 003:00004 mod_python.so b78a8000 12 rw--- 0000000000017000 003:00004 mod_python.so b78ab000 136 r-x-- 0000000000000000 003:00004 libsvn_fs_base-1.so.0.0.0 @@ -82,6 +121,8 @@ b7914000 4 rw--- 0000000000006000 003:00004 libsvn_delta-1.so.0.0.0 b7915000 112 r-x-- 0000000000000000 003:00004 libsvn_repos-1.so.0.0.0 b7931000 4 rw--- 000000000001b000 003:00004 libsvn_repos-1.so.0.0.0 +b7932000 12 r-x-- 0000000000000000 003:00004 cStringIO.so +b7935000 4 rw--- 0000000000003000 003:00004 cStringIO.so b7936000 8 r-x-- 0000000000000000 003:00004 mod_userdir.so b7938000 4 rw--- 0000000000001000 003:00004 mod_userdir.so b7939000 8 r-x-- 0000000000000000 003:00004 libutil-2.3.6.so @@ -161,4 +202,4 @@ bfb4b000 80 rwx-- 00000000bfb4b000 000:00000 [ stack ] bfb5f000 8 rw--- 00000000bfb5f000 000:00000 [ anon ] ffffe000 4 ----- 0000000000000000 000:00000 [ anon ] -mapped: 232452K writeable/private: 224444K shared: 208K +mapped: 235048K writeable/private: 226776K shared: 208K So we are increasing memory usage by 4928K (2596K mapped + 2332K writable/private) which is in line with what Graham found using top. Jim -------------- next part -------------- A non-text attachment was scrubbed... Name: quickie-profile.sh Type: application/x-shellscript Size: 510 bytes Desc: not available Url : http://mm_cfg_has_not_been_edited_to_set_host_domains/pipermail/mod_python/attachments/20070303/038809bc/quickie-profile-0001.bin
|