|
Graham Dumpleton
graham.dumpleton at gmail.com
Fri May 15 06:35:59 EDT 2009
2009/5/15 zsi <zsi at skolesys.dk>: > > > On Fri, 15 May 2009 20:11:38 +1000, Graham Dumpleton > <graham.dumpleton at gmail.com> wrote: >> 2009/5/15 zsi <zsi at skolesys.dk>: >>> Hey * >>> >>> I am creating a SOAP service that is based on ZSI and mod_python. I have >>> the service running and it works, but it's not performing well, so I >> made >>> did some output to the syslog from inside the handler: >>> >>> reloadchecker.py: >>> ---------------- >>> testvar=1 >>> ---------------- >>> >>> My handler: >>> ---------------- >>> import capa0_9.soap.sessionservice as sessionservice >>> import capa0_9.soap.zsi_handler as zsi_handler >>> from mod_python import apache >>> import capa0_9.tools.log as log >>> import os,reloadchecker >>> >>> mod = __import__('encodings.utf_8', globals(), locals(), '*') >>> mod = __import__('encodings.utf_16_be', globals(), locals(), '*') >>> >>> def handler(req): >>> >>> """ >>> The PythonHandler entry-point. This function must be present >> in >>> order to have mod_apache dispatch a python operation. >>> """ >>> zsi_handler.AsHandler(modules=(sessionservice,), request=req) >>> log.debug("REQUESTINFO: Interpreter Name = %s" % >>> str(req.interpreter)) >>> log.debug("REQUESTINFO: os.getpid() = %s" % >> str(os.getpid())) >>> log.debug("REQUESTINFO: reloadchecker = %s" % >>> str(reloadchecker.testvar)) >>> reloadchecker.testvar += 1 >>> return apache.OK >>> ----------------- >>> >>> I log the interpreter name, which according to section 4.1 should make >>> mod_python boost performance by keeping the interpreter alive as long as >>> the apache server processes. And I log the pid of the interpreter. As it >> is >>> clear to see in the output below a new pid is spawned each time, it does >>> not reuse the interpreter and therefore, I guess, it has to reload >>> everything for each request. Note that i also list the apache2 processes >>> before and after I bombard the server with requests, it shows that the >> same >>> processes are running before and after. I also made a dummy module >>> "reloadchecker" that contains one variable "testvar". If it is reloaded >> the >>> variable is initialized to 1, if it was already loaded it should be >>> increased by 1 for each request. >>> >>> Am I missing something, > > Thanks for the quick response. > >> >> Yes. >> >>> shouldn't mod_python optimize performance by >>> reusing previously loaded modules from earlier requests? >> >> Only if the request is actually handled by the same process. Since >> Apache is a multiprocess web server, the requests could be handled by >> any of the processes. You will obviously incur the load cost of the >> interpreter and application for the first request against each >> processor. Subsequent requests for a process should use the fact that >> stuff is already loaded. You likely didn't do enough requests to have >> touched all process and have a request actually go back to a process >> which was already used. >> >> Read: >> >> >> > http://www.dscpl.com.au/wiki/ModPython/Articles/TheProcessInterpreterModel >> >> Since you appear to be using prefork MPM, also read about the dangers >> of prefork MPM in: >> >> >> > http://blog.dscpl.com.au/2009/03/load-spikes-and-excessive-memory-usage.html >> >> If you want better control over how many processes are used for your >> application and particularly so you can limit number to small fixed >> number and/or use threading as well, then look at mod_wsgi daemon mode >> instead. >> >> BTW, the process IDs in your 'ps' output don't even match what your >> Python code is producing. Looks like you aren't even talking to the >> same Apache instance. > I wasn't expecting them to match. The process id's from the python code > are the interpreter's pid not the apaches. What I wanted to show by > that was that it is stille the same apache processes (8 of them) but > I have 18 different interpreter process ID's. So it is not because I > hit a new apache process every time. Did you read the document I told you to read about process/interpreter model of Apache/mod_python? As well as the blog entry? The Python sub interpreters run in the Apache child processes. Thus the process IDs from your Python code must have some overlap with the Apache processes you listed. Apache/mod_python does not create special Python interpreter processes. Graham >> So, try and validate your configuration and whether you might have >> multiple Apache installations running. Look for 'httpd' processes as >> well. > > Here is the server pool configuration (standard ubuntu mpm-prefork > configuration): > StartServers 5 > MinSpareServers 5 > MaxSpareServers 10 > MaxClients 150 > MaxRequestsPerChild 0 > > Best regards > Jakob Simon-Gaarde > >>> mikrov at capa-server:/srv/capa/www/0.9$ sudo ps aux | grep apache2 >>> mikrov 7396 0.0 0.1 3004 764 pts/2 S+ 11:11 >> 0:00 grep >>> apache2 >>> root 22343 0.0 1.0 13728 5336 ? Ss May14 >> 0:26 >>> /usr/sbin/apache2 -k start >>> root 22346 0.0 0.6 13728 3136 ? S May14 >> 0:04 >>> /usr/sbin/apache2 -k start >>> root 22347 0.0 0.6 13728 3136 ? S May14 >> 0:04 >>> /usr/sbin/apache2 -k start >>> root 22348 0.0 0.6 13728 3136 ? S May14 >> 0:04 >>> /usr/sbin/apache2 -k start >>> root 22349 0.0 0.6 13728 3136 ? S May14 >> 0:04 >>> /usr/sbin/apache2 -k start >>> root 22350 0.0 0.6 13728 3136 ? S May14 >> 0:04 >>> /usr/sbin/apache2 -k start >>> root 22383 0.0 0.6 13728 3136 ? S May14 >> 0:04 >>> /usr/sbin/apache2 -k start >>> root 22392 0.0 0.6 13728 3136 ? S May14 >> 0:04 >>> /usr/sbin/apache2 -k start >>> root 22407 0.0 0.6 13728 3136 ? S May14 >> 0:04 >>> /usr/sbin/apache2 -k start >>> root 22716 0.0 0.6 13728 3136 ? S May14 >> 0:03 >>> /usr/sbin/apache2 -k start >>> root 23356 0.0 0.6 13728 3136 ? S May14 >> 0:03 >>> /usr/sbin/apache2 -k start >>> >>> mikrov at capa-server:/srv/capa/www/0.9/sessionservice$ sudo tail -f >>> /var/log/syslog | grep REQUESTINFO >>> May 15 11:52:03 capa-server capa: REQUESTINFO: Interpreter Name = >>> /srv/capa/www/0.9/sessionservice/ >>> May 15 11:52:03 capa-server capa: REQUESTINFO: os.getpid() = >> 7953 >>> >>> May 15 11:52:03 capa-server capa: REQUESTINFO: reloadchecker = 1 >>> >>> May 15 11:52:03 capa-server capa: REQUESTINFO: Interpreter Name = >>> /srv/capa/www/0.9/sessionservice/ >>> May 15 11:52:03 capa-server capa: REQUESTINFO: os.getpid() = >> 7954 >>> >>> May 15 11:52:03 capa-server capa: REQUESTINFO: reloadchecker = 1 >>> >>> May 15 11:52:04 capa-server capa: REQUESTINFO: Interpreter Name = >>> /srv/capa/www/0.9/sessionservice/ >>> May 15 11:52:04 capa-server capa: REQUESTINFO: os.getpid() = >> 7955 >>> >>> May 15 11:52:04 capa-server capa: REQUESTINFO: reloadchecker = 1 >>> >>> May 15 11:52:05 capa-server capa: REQUESTINFO: Interpreter Name = >>> /srv/capa/www/0.9/sessionservice/ >>> May 15 11:52:05 capa-server capa: REQUESTINFO: os.getpid() = >> 7959 >>> >>> May 15 11:52:05 capa-server capa: REQUESTINFO: reloadchecker = 1 >>> >>> May 15 11:52:06 capa-server capa: REQUESTINFO: Interpreter Name = >>> /srv/capa/www/0.9/sessionservice/ >>> May 15 11:52:06 capa-server capa: REQUESTINFO: os.getpid() = >> 7960 >>> >>> May 15 11:52:06 capa-server capa: REQUESTINFO: reloadchecker = 1 >>> >>> May 15 11:52:07 capa-server capa: REQUESTINFO: Interpreter Name = >>> /srv/capa/www/0.9/sessionservice/ >>> May 15 11:52:07 capa-server capa: REQUESTINFO: os.getpid() = >> 7961 >>> >>> May 15 11:52:07 capa-server capa: REQUESTINFO: reloadchecker = 1 >>> May 15 11:52:08 capa-server capa: REQUESTINFO: Interpreter Name = >>> /srv/capa/www/0.9/sessionservice/ >>> May 15 11:52:08 capa-server capa: REQUESTINFO: os.getpid() = >> 7963 >>> May 15 11:52:08 capa-server capa: REQUESTINFO: reloadchecker = 1 >>> May 15 11:52:08 capa-server capa: REQUESTINFO: Interpreter Name = >>> /srv/capa/www/0.9/sessionservice/ >>> May 15 11:52:08 capa-server capa: REQUESTINFO: os.getpid() = >> 7964 >>> May 15 11:52:08 capa-server capa: REQUESTINFO: reloadchecker = 1 >>> May 15 11:52:09 capa-server capa: REQUESTINFO: Interpreter Name = >>> /srv/capa/www/0.9/sessionservice/ >>> May 15 11:52:09 capa-server capa: REQUESTINFO: os.getpid() = >> 7966 >>> May 15 11:52:09 capa-server capa: REQUESTINFO: reloadchecker = 1 >>> May 15 11:52:10 capa-server capa: REQUESTINFO: Interpreter Name = >>> /srv/capa/www/0.9/sessionservice/ >>> May 15 11:52:10 capa-server capa: REQUESTINFO: os.getpid() = >> 7968 >>> May 15 11:52:10 capa-server capa: REQUESTINFO: reloadchecker = 1 >>> May 15 11:52:11 capa-server capa: REQUESTINFO: Interpreter Name = >>> /srv/capa/www/0.9/sessionservice/ >>> May 15 11:52:11 capa-server capa: REQUESTINFO: os.getpid() = >> 7969 >>> May 15 11:52:11 capa-server capa: REQUESTINFO: reloadchecker = 1 >>> May 15 11:52:12 capa-server capa: REQUESTINFO: Interpreter Name = >>> /srv/capa/www/0.9/sessionservice/ >>> May 15 11:52:12 capa-server capa: REQUESTINFO: os.getpid() = >> 7970 >>> May 15 11:52:12 capa-server capa: REQUESTINFO: reloadchecker = 1 >>> May 15 11:52:13 capa-server capa: REQUESTINFO: Interpreter Name = >>> /srv/capa/www/0.9/sessionservice/ >>> May 15 11:52:13 capa-server capa: REQUESTINFO: os.getpid() = >> 7972 >>> May 15 11:52:13 capa-server capa: REQUESTINFO: reloadchecker = 1 >>> May 15 11:52:13 capa-server capa: REQUESTINFO: Interpreter Name = >>> /srv/capa/www/0.9/sessionservice/ >>> May 15 11:52:13 capa-server capa: REQUESTINFO: os.getpid() = >> 7973 >>> May 15 11:52:13 capa-server capa: REQUESTINFO: reloadchecker = 1 >>> May 15 11:52:14 capa-server capa: REQUESTINFO: Interpreter Name = >>> /srv/capa/www/0.9/sessionservice/ >>> May 15 11:52:14 capa-server capa: REQUESTINFO: os.getpid() = >> 7974 >>> May 15 11:52:14 capa-server capa: REQUESTINFO: reloadchecker = 1 >>> May 15 11:52:15 capa-server capa: REQUESTINFO: Interpreter Name = >>> /srv/capa/www/0.9/sessionservice/ >>> May 15 11:52:15 capa-server capa: REQUESTINFO: os.getpid() = >> 7976 >>> May 15 11:52:15 capa-server capa: REQUESTINFO: reloadchecker = 1 >>> May 15 11:52:16 capa-server capa: REQUESTINFO: Interpreter Name = >>> /srv/capa/www/0.9/sessionservice/ >>> May 15 11:52:16 capa-server capa: REQUESTINFO: os.getpid() = >> 7977 >>> May 15 11:52:16 capa-server capa: REQUESTINFO: reloadchecker = 1 >>> >>> mikrov at capa-server:/srv/capa/www/0.9$ sudo ps aux | grep apache2 >>> mikrov 7457 0.0 0.1 3004 764 pts/2 S+ 11:12 >> 0:00 grep >>> apache2 >>> root 22343 0.0 1.0 13728 5336 ? Ss May14 >> 0:26 >>> /usr/sbin/apache2 -k start >>> root 22346 0.0 0.6 13728 3136 ? S May14 >> 0:04 >>> /usr/sbin/apache2 -k start >>> root 22347 0.0 0.6 13728 3136 ? S May14 >> 0:04 >>> /usr/sbin/apache2 -k start >>> root 22348 0.0 0.6 13728 3136 ? S May14 >> 0:04 >>> /usr/sbin/apache2 -k start >>> root 22349 0.0 0.6 13728 3136 ? S May14 >> 0:04 >>> /usr/sbin/apache2 -k start >>> root 22350 0.0 0.6 13728 3136 ? S May14 >> 0:04 >>> /usr/sbin/apache2 -k start >>> root 22383 0.0 0.6 13728 3136 ? S May14 >> 0:04 >>> /usr/sbin/apache2 -k start >>> root 22392 0.0 0.6 13728 3136 ? S May14 >> 0:04 >>> /usr/sbin/apache2 -k start >>> root 22407 0.0 0.6 13728 3136 ? S May14 >> 0:04 >>> /usr/sbin/apache2 -k start >>> root 22716 0.0 0.6 13728 3136 ? S May14 >> 0:03 >>> /usr/sbin/apache2 -k start >>> root 23356 0.0 0.6 13728 3136 ? S May14 >> 0:03 >>> /usr/sbin/apache2 -k start > > _______________________________________________ > Mod_python mailing list > Mod_python at modpython.org > http://mailman.modpython.org/mailman/listinfo/mod_python >
|