|
Graham Dumpleton
graham.dumpleton at gmail.com
Wed Jun 27 19:15:22 EDT 2007
Is the version of mod_python you are using one you compiled yourself, or one from Ubuntu packages? Is the version of Python you are using one you compiled yourself, or one from Ubuntu packages? What is the size of mod_python.so? Does it use a shared Python library or static? Use 'ldd' on mod_python.so to see dependencies. Graham On 28/06/07, Aseem Mohanty <aseem at metaweb.com> wrote: > Yes, hence the exotic phraseology. Results for the various combinations are below: > > 1. mod_python only, no mod_auth* modules: leaks > 2. mod_auth_digest only, no other mod_auth*, no mod_python: does not leak > 3. mod_authz_host only, no other mod_auth*, no mod_python: does not leak > 4. mod_python + mod_authz_host, no mod_auth_digest: leaks > 5. mod_python + mod_auth_digest, no mod_authz_host: leaks < (1) or (4) > 6. mod_python + mod_auth_digest + mod_authz_host: does not leak > > NOTE: 'does not leak' implies ~50-100K leakage per graceful restart > (5) leaks ~500-600K per graceful restart > zero load on server > > As far as I can tell there is definitely some sinister interaction going on > between these three modules. Unfortunately not being conversant with apache > module development, I cannot really say what. > > Thanks. > AM > > > Jim Gallacher wrote: > > Aseem Mohanty wrote: > >> This may sound quite exotic > > > > > > Why yes, it does? Interaction between these modules resulting in a > > memory leak sure sounds weird. Did you try it with *only* mod_python > > loaded? (ie, without mod_authz_host) > > > > Jim > > > >> but here it is: > >> > >> In a vanilla apache2.2.x configuration if I only load mod_authz_host > >> and none of > >> the other mod_auth* modules, every graceful restart causes ~1Mb leak > >> in the > >> parent process. This is with mod_python loaded. Without mod_python the > >> memory profile remains unchanged. > >> > >> Steps to reproduce: > >> Gentoo: > >> 1. In /etc/apache2/httpd.conf comment out all "LoadModule auth*" > >> directives except > >> LoadModule authz_host_module modules/mod_authz_host.so > >> > >> Ubuntu Feisty (repos): > >> 1. In /etc/apache2/mods-enabled rename auth*.load files except > >> authz_host.load > >> to auth*.load.not > >> > >> Ubuntu Feisty (httpd built from source): > >> 1. In conf/httpd.conf comment out all "LoadModule auth*" directives > >> except > >> LoadModule authz_host_module modules/mod_authz_host.so > >> > >> 2. Start apache > >> 3. Track apache parent process: top -d1 -p $(pgrep -o apache/httpd) > >> 4. Hit apache instances with graceful restarts: > >> for i in $(seq 150); do echo $i; kill -USR1 $(pgrep -o apache/httpd); > >> sleep 2; done > >> > >> In this case the parent will leak ~1Mb per restart. If however in each > >> of the > >> configurations mod_auth_digest.so is also loaded the leak is only > >> about 150-200K > >> or so per graceful restart. This is on a server with 0 load. In my > >> case it > >> caused apache instances that were restarted every 55 minutes to grow > >> up to 300Mb > >> in a week. > >> > >> Tested on: > >> 1. Ubuntu Feisty (built from source httpd-2.2.4, mod_python-3.3.1): > >> ./configure --prefix=/opt --enable-modules=all > >> --enable-mods-shared=all --enable-so --with-included-apr > >> Server version: Apache/2.2.4 (Unix) > >> Server built: Jun 27 2007 09:34:16 > >> Server's Module Magic Number: 20051115:4 > >> Server loaded: APR 1.2.8, APR-Util 1.2.8 > >> Compiled using: APR 1.2.8, APR-Util 1.2.8 > >> Architecture: 32-bit > >> Server MPM: Prefork > >> threaded: no > >> forked: yes (variable process count) > >> Server compiled with.... > >> -D APACHE_MPM_DIR="server/mpm/prefork" > >> -D APR_HAS_SENDFILE > >> -D APR_HAS_MMAP > >> -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled) > >> -D APR_USE_SYSVSEM_SERIALIZE > >> -D APR_USE_PTHREAD_SERIALIZE > >> -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT > >> -D APR_HAS_OTHER_CHILD > >> -D AP_HAVE_RELIABLE_PIPED_LOGS > >> -D DYNAMIC_MODULE_LIMIT=128 > >> -D HTTPD_ROOT="/opt/share/apache2" > >> -D SUEXEC_BIN="/opt/share/apache2/bin/suexec" > >> -D DEFAULT_PIDLOG="logs/httpd.pid" > >> -D DEFAULT_SCOREBOARD="logs/apache_runtime_status" > >> -D DEFAULT_LOCKFILE="logs/accept.lock" > >> -D DEFAULT_ERRORLOG="logs/error_log" > >> -D AP_TYPES_CONFIG_FILE="conf/mime.types" > >> -D SERVER_CONFIG_FILE="conf/httpd.conf" > >> > >> > >> 2. gentoo (64 bit dual proc AMD): > >> Linux host1 2.6.16-gentoo-r9 #7 SMP Fri Mar 16 17:35:26 UTC 2007 > >> x86_64 AMD > >> Opteron(tm) Processor 248 AuthenticAMD GNU/Linux > >> > >> Server version: Apache/2.2.4 (Unix) > >> Server built: Jun 26 2007 19:29:33 > >> Server's Module Magic Number: 20051115:4 > >> Server loaded: APR 1.2.8, APR-Util 1.2.8 > >> Compiled using: APR 1.2.8, APR-Util 1.2.8 > >> Architecture: 64-bit > >> Server MPM: Prefork > >> threaded: no > >> forked: yes (variable process count) > >> Server compiled with.... > >> -D APACHE_MPM_DIR="server/mpm/prefork" > >> -D APR_HAS_SENDFILE > >> -D APR_HAS_MMAP > >> -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled) > >> -D APR_USE_SYSVSEM_SERIALIZE > >> -D APR_USE_PTHREAD_SERIALIZE > >> -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT > >> -D APR_HAS_OTHER_CHILD > >> -D AP_HAVE_RELIABLE_PIPED_LOGS > >> -D DYNAMIC_MODULE_LIMIT=128 > >> -D HTTPD_ROOT="/usr" > >> -D SUEXEC_BIN="/usr/sbin/suexec" > >> -D DEFAULT_PIDLOG="/var/run/httpd.pid" > >> -D DEFAULT_SCOREBOARD="logs/apache_runtime_status" > >> -D DEFAULT_LOCKFILE="/var/run/accept.lock" > >> -D DEFAULT_ERRORLOG="logs/error_log" > >> -D AP_TYPES_CONFIG_FILE="/etc/apache2/mime.types" > >> -D SERVER_CONFIG_FILE="/etc/apache2/httpd.conf" > >> > >> 3. Ubuntu Feisty (32 bit Intel core duo - ubuntu repos): > >> Linux host2 2.6.20-16-generic #2 SMP Thu Jun 7 20:19:32 UTC 2007 i686 > >> GNU/Linux > >> > >> Server version: Apache/2.2.3 > >> Server built: Jan 15 2007 18:14:50 > >> Server's Module Magic Number: 20051115:3 > >> Server loaded: APR 1.2.7, APR-Util 1.2.7 > >> Compiled using: APR 1.2.7, APR-Util 1.2.7 > >> Architecture: 32-bit > >> Server MPM: Prefork > >> threaded: no > >> forked: yes (variable process count) > >> Server compiled with.... > >> -D APACHE_MPM_DIR="server/mpm/prefork" > >> -D APR_HAS_SENDFILE > >> -D APR_HAS_MMAP > >> -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled) > >> -D APR_USE_SYSVSEM_SERIALIZE > >> -D APR_USE_PTHREAD_SERIALIZE > >> -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT > >> -D APR_HAS_OTHER_CHILD > >> -D AP_HAVE_RELIABLE_PIPED_LOGS > >> -D DYNAMIC_MODULE_LIMIT=128 > >> -D HTTPD_ROOT="" > >> -D SUEXEC_BIN="/usr/lib/apache2/suexec" > >> -D DEFAULT_PIDLOG="/var/run/apache2.pid" > >> -D DEFAULT_SCOREBOARD="logs/apache_runtime_status" > >> -D DEFAULT_LOCKFILE="/var/run/apache2/accept.lock" > >> -D DEFAULT_ERRORLOG="logs/error_log" > >> -D AP_TYPES_CONFIG_FILE="/etc/apache2/mime.types" > >> -D SERVER_CONFIG_FILE="/etc/apache2/apache2.conf" > >> > >> Bug or just an unexpected apache configuration?? I actually filed a > >> bug against apache core > >> (http://issues.apache.org/bugzilla/show_bug.cgi?id=42749) initially > >> not realizing that it could be a combination of factors that was > >> causing it and not just soe module not being loaded. > >> > >> Thanks. > >> AM > >> _______________________________________________ > >> Mod_python mailing list > >> Mod_python at modpython.org > >> http://mailman.modpython.org/mailman/listinfo/mod_python > >> > > > _______________________________________________ > Mod_python mailing list > Mod_python at modpython.org > http://mailman.modpython.org/mailman/listinfo/mod_python >
|