|
zsi
zsi at skolesys.dk
Fri May 15 06:34:07 EDT 2009
On Fri, 15 May 2009 20:11:38 +1000, Graham Dumpleton <graham.dumpleton at gmail.com> wrote: > 2009/5/15 zsi <zsi at skolesys.dk>: >> Hey * >> >> I am creating a SOAP service that is based on ZSI and mod_python. I have >> the service running and it works, but it's not performing well, so I > made >> did some output to the syslog from inside the handler: >> >> reloadchecker.py: >> ---------------- >> testvar=1 >> ---------------- >> >> My handler: >> ---------------- >> import capa0_9.soap.sessionservice as sessionservice >> import capa0_9.soap.zsi_handler as zsi_handler >> from mod_python import apache >> import capa0_9.tools.log as log >> import os,reloadchecker >> >> mod = __import__('encodings.utf_8', globals(), locals(), '*') >> mod = __import__('encodings.utf_16_be', globals(), locals(), '*') >> >> def handler(req): >> >> """ >> The PythonHandler entry-point. This function must be present > in >> order to have mod_apache dispatch a python operation. >> """ >> zsi_handler.AsHandler(modules=(sessionservice,), request=req) >> log.debug("REQUESTINFO: Interpreter Name = %s" % >> str(req.interpreter)) >> log.debug("REQUESTINFO: os.getpid() = %s" % > str(os.getpid())) >> log.debug("REQUESTINFO: reloadchecker = %s" % >> str(reloadchecker.testvar)) >> reloadchecker.testvar += 1 >> return apache.OK >> ----------------- >> >> I log the interpreter name, which according to section 4.1 should make >> mod_python boost performance by keeping the interpreter alive as long as >> the apache server processes. And I log the pid of the interpreter. As it > is >> clear to see in the output below a new pid is spawned each time, it does >> not reuse the interpreter and therefore, I guess, it has to reload >> everything for each request. Note that i also list the apache2 processes >> before and after I bombard the server with requests, it shows that the > same >> processes are running before and after. I also made a dummy module >> "reloadchecker" that contains one variable "testvar". If it is reloaded > the >> variable is initialized to 1, if it was already loaded it should be >> increased by 1 for each request. >> >> Am I missing something, Thanks for the quick response. > > Yes. > >> shouldn't mod_python optimize performance by >> reusing previously loaded modules from earlier requests? > > Only if the request is actually handled by the same process. Since > Apache is a multiprocess web server, the requests could be handled by > any of the processes. You will obviously incur the load cost of the > interpreter and application for the first request against each > processor. Subsequent requests for a process should use the fact that > stuff is already loaded. You likely didn't do enough requests to have > touched all process and have a request actually go back to a process > which was already used. > > Read: > > > http://www.dscpl.com.au/wiki/ModPython/Articles/TheProcessInterpreterModel > > Since you appear to be using prefork MPM, also read about the dangers > of prefork MPM in: > > > http://blog.dscpl.com.au/2009/03/load-spikes-and-excessive-memory-usage.html > > If you want better control over how many processes are used for your > application and particularly so you can limit number to small fixed > number and/or use threading as well, then look at mod_wsgi daemon mode > instead. > > BTW, the process IDs in your 'ps' output don't even match what your > Python code is producing. Looks like you aren't even talking to the > same Apache instance. I wasn't expecting them to match. The process id's from the python code are the interpreter's pid not the apaches. What I wanted to show by that was that it is stille the same apache processes (8 of them) but I have 18 different interpreter process ID's. So it is not because I hit a new apache process every time. > So, try and validate your configuration and whether you might have > multiple Apache installations running. Look for 'httpd' processes as > well. Here is the server pool configuration (standard ubuntu mpm-prefork configuration): StartServers 5 MinSpareServers 5 MaxSpareServers 10 MaxClients 150 MaxRequestsPerChild 0 Best regards Jakob Simon-Gaarde >> mikrov at capa-server:/srv/capa/www/0.9$ sudo ps aux | grep apache2 >> mikrov 7396 0.0 0.1 3004 764 pts/2 S+ 11:11 > 0:00 grep >> apache2 >> root 22343 0.0 1.0 13728 5336 ? Ss May14 > 0:26 >> /usr/sbin/apache2 -k start >> root 22346 0.0 0.6 13728 3136 ? S May14 > 0:04 >> /usr/sbin/apache2 -k start >> root 22347 0.0 0.6 13728 3136 ? S May14 > 0:04 >> /usr/sbin/apache2 -k start >> root 22348 0.0 0.6 13728 3136 ? S May14 > 0:04 >> /usr/sbin/apache2 -k start >> root 22349 0.0 0.6 13728 3136 ? S May14 > 0:04 >> /usr/sbin/apache2 -k start >> root 22350 0.0 0.6 13728 3136 ? S May14 > 0:04 >> /usr/sbin/apache2 -k start >> root 22383 0.0 0.6 13728 3136 ? S May14 > 0:04 >> /usr/sbin/apache2 -k start >> root 22392 0.0 0.6 13728 3136 ? S May14 > 0:04 >> /usr/sbin/apache2 -k start >> root 22407 0.0 0.6 13728 3136 ? S May14 > 0:04 >> /usr/sbin/apache2 -k start >> root 22716 0.0 0.6 13728 3136 ? S May14 > 0:03 >> /usr/sbin/apache2 -k start >> root 23356 0.0 0.6 13728 3136 ? S May14 > 0:03 >> /usr/sbin/apache2 -k start >> >> mikrov at capa-server:/srv/capa/www/0.9/sessionservice$ sudo tail -f >> /var/log/syslog | grep REQUESTINFO >> May 15 11:52:03 capa-server capa: REQUESTINFO: Interpreter Name = >> /srv/capa/www/0.9/sessionservice/ >> May 15 11:52:03 capa-server capa: REQUESTINFO: os.getpid() = > 7953 >> >> May 15 11:52:03 capa-server capa: REQUESTINFO: reloadchecker = 1 >> >> May 15 11:52:03 capa-server capa: REQUESTINFO: Interpreter Name = >> /srv/capa/www/0.9/sessionservice/ >> May 15 11:52:03 capa-server capa: REQUESTINFO: os.getpid() = > 7954 >> >> May 15 11:52:03 capa-server capa: REQUESTINFO: reloadchecker = 1 >> >> May 15 11:52:04 capa-server capa: REQUESTINFO: Interpreter Name = >> /srv/capa/www/0.9/sessionservice/ >> May 15 11:52:04 capa-server capa: REQUESTINFO: os.getpid() = > 7955 >> >> May 15 11:52:04 capa-server capa: REQUESTINFO: reloadchecker = 1 >> >> May 15 11:52:05 capa-server capa: REQUESTINFO: Interpreter Name = >> /srv/capa/www/0.9/sessionservice/ >> May 15 11:52:05 capa-server capa: REQUESTINFO: os.getpid() = > 7959 >> >> May 15 11:52:05 capa-server capa: REQUESTINFO: reloadchecker = 1 >> >> May 15 11:52:06 capa-server capa: REQUESTINFO: Interpreter Name = >> /srv/capa/www/0.9/sessionservice/ >> May 15 11:52:06 capa-server capa: REQUESTINFO: os.getpid() = > 7960 >> >> May 15 11:52:06 capa-server capa: REQUESTINFO: reloadchecker = 1 >> >> May 15 11:52:07 capa-server capa: REQUESTINFO: Interpreter Name = >> /srv/capa/www/0.9/sessionservice/ >> May 15 11:52:07 capa-server capa: REQUESTINFO: os.getpid() = > 7961 >> >> May 15 11:52:07 capa-server capa: REQUESTINFO: reloadchecker = 1 >> May 15 11:52:08 capa-server capa: REQUESTINFO: Interpreter Name = >> /srv/capa/www/0.9/sessionservice/ >> May 15 11:52:08 capa-server capa: REQUESTINFO: os.getpid() = > 7963 >> May 15 11:52:08 capa-server capa: REQUESTINFO: reloadchecker = 1 >> May 15 11:52:08 capa-server capa: REQUESTINFO: Interpreter Name = >> /srv/capa/www/0.9/sessionservice/ >> May 15 11:52:08 capa-server capa: REQUESTINFO: os.getpid() = > 7964 >> May 15 11:52:08 capa-server capa: REQUESTINFO: reloadchecker = 1 >> May 15 11:52:09 capa-server capa: REQUESTINFO: Interpreter Name = >> /srv/capa/www/0.9/sessionservice/ >> May 15 11:52:09 capa-server capa: REQUESTINFO: os.getpid() = > 7966 >> May 15 11:52:09 capa-server capa: REQUESTINFO: reloadchecker = 1 >> May 15 11:52:10 capa-server capa: REQUESTINFO: Interpreter Name = >> /srv/capa/www/0.9/sessionservice/ >> May 15 11:52:10 capa-server capa: REQUESTINFO: os.getpid() = > 7968 >> May 15 11:52:10 capa-server capa: REQUESTINFO: reloadchecker = 1 >> May 15 11:52:11 capa-server capa: REQUESTINFO: Interpreter Name = >> /srv/capa/www/0.9/sessionservice/ >> May 15 11:52:11 capa-server capa: REQUESTINFO: os.getpid() = > 7969 >> May 15 11:52:11 capa-server capa: REQUESTINFO: reloadchecker = 1 >> May 15 11:52:12 capa-server capa: REQUESTINFO: Interpreter Name = >> /srv/capa/www/0.9/sessionservice/ >> May 15 11:52:12 capa-server capa: REQUESTINFO: os.getpid() = > 7970 >> May 15 11:52:12 capa-server capa: REQUESTINFO: reloadchecker = 1 >> May 15 11:52:13 capa-server capa: REQUESTINFO: Interpreter Name = >> /srv/capa/www/0.9/sessionservice/ >> May 15 11:52:13 capa-server capa: REQUESTINFO: os.getpid() = > 7972 >> May 15 11:52:13 capa-server capa: REQUESTINFO: reloadchecker = 1 >> May 15 11:52:13 capa-server capa: REQUESTINFO: Interpreter Name = >> /srv/capa/www/0.9/sessionservice/ >> May 15 11:52:13 capa-server capa: REQUESTINFO: os.getpid() = > 7973 >> May 15 11:52:13 capa-server capa: REQUESTINFO: reloadchecker = 1 >> May 15 11:52:14 capa-server capa: REQUESTINFO: Interpreter Name = >> /srv/capa/www/0.9/sessionservice/ >> May 15 11:52:14 capa-server capa: REQUESTINFO: os.getpid() = > 7974 >> May 15 11:52:14 capa-server capa: REQUESTINFO: reloadchecker = 1 >> May 15 11:52:15 capa-server capa: REQUESTINFO: Interpreter Name = >> /srv/capa/www/0.9/sessionservice/ >> May 15 11:52:15 capa-server capa: REQUESTINFO: os.getpid() = > 7976 >> May 15 11:52:15 capa-server capa: REQUESTINFO: reloadchecker = 1 >> May 15 11:52:16 capa-server capa: REQUESTINFO: Interpreter Name = >> /srv/capa/www/0.9/sessionservice/ >> May 15 11:52:16 capa-server capa: REQUESTINFO: os.getpid() = > 7977 >> May 15 11:52:16 capa-server capa: REQUESTINFO: reloadchecker = 1 >> >> mikrov at capa-server:/srv/capa/www/0.9$ sudo ps aux | grep apache2 >> mikrov 7457 0.0 0.1 3004 764 pts/2 S+ 11:12 > 0:00 grep >> apache2 >> root 22343 0.0 1.0 13728 5336 ? Ss May14 > 0:26 >> /usr/sbin/apache2 -k start >> root 22346 0.0 0.6 13728 3136 ? S May14 > 0:04 >> /usr/sbin/apache2 -k start >> root 22347 0.0 0.6 13728 3136 ? S May14 > 0:04 >> /usr/sbin/apache2 -k start >> root 22348 0.0 0.6 13728 3136 ? S May14 > 0:04 >> /usr/sbin/apache2 -k start >> root 22349 0.0 0.6 13728 3136 ? S May14 > 0:04 >> /usr/sbin/apache2 -k start >> root 22350 0.0 0.6 13728 3136 ? S May14 > 0:04 >> /usr/sbin/apache2 -k start >> root 22383 0.0 0.6 13728 3136 ? S May14 > 0:04 >> /usr/sbin/apache2 -k start >> root 22392 0.0 0.6 13728 3136 ? S May14 > 0:04 >> /usr/sbin/apache2 -k start >> root 22407 0.0 0.6 13728 3136 ? S May14 > 0:04 >> /usr/sbin/apache2 -k start >> root 22716 0.0 0.6 13728 3136 ? S May14 > 0:03 >> /usr/sbin/apache2 -k start >> root 23356 0.0 0.6 13728 3136 ? S May14 > 0:03 >> /usr/sbin/apache2 -k start
|