|
Graham Dumpleton
graham.dumpleton at gmail.com
Fri May 15 06:11:38 EDT 2009
2009/5/15 zsi <zsi at skolesys.dk>: > Hey * > > I am creating a SOAP service that is based on ZSI and mod_python. I have > the service running and it works, but it's not performing well, so I made > did some output to the syslog from inside the handler: > > reloadchecker.py: > ---------------- > testvar=1 > ---------------- > > My handler: > ---------------- > import capa0_9.soap.sessionservice as sessionservice > import capa0_9.soap.zsi_handler as zsi_handler > from mod_python import apache > import capa0_9.tools.log as log > import os,reloadchecker > > mod = __import__('encodings.utf_8', globals(), locals(), '*') > mod = __import__('encodings.utf_16_be', globals(), locals(), '*') > > def handler(req): > > """ > The PythonHandler entry-point. This function must be present in > order to have mod_apache dispatch a python operation. > """ > zsi_handler.AsHandler(modules=(sessionservice,), request=req) > log.debug("REQUESTINFO: Interpreter Name = %s" % > str(req.interpreter)) > log.debug("REQUESTINFO: os.getpid() = %s" % str(os.getpid())) > log.debug("REQUESTINFO: reloadchecker = %s" % > str(reloadchecker.testvar)) > reloadchecker.testvar += 1 > return apache.OK > ----------------- > > I log the interpreter name, which according to section 4.1 should make > mod_python boost performance by keeping the interpreter alive as long as > the apache server processes. And I log the pid of the interpreter. As it is > clear to see in the output below a new pid is spawned each time, it does > not reuse the interpreter and therefore, I guess, it has to reload > everything for each request. Note that i also list the apache2 processes > before and after I bombard the server with requests, it shows that the same > processes are running before and after. I also made a dummy module > "reloadchecker" that contains one variable "testvar". If it is reloaded the > variable is initialized to 1, if it was already loaded it should be > increased by 1 for each request. > > Am I missing something, Yes. > shouldn't mod_python optimize performance by > reusing previously loaded modules from earlier requests? Only if the request is actually handled by the same process. Since Apache is a multiprocess web server, the requests could be handled by any of the processes. You will obviously incur the load cost of the interpreter and application for the first request against each processor. Subsequent requests for a process should use the fact that stuff is already loaded. You likely didn't do enough requests to have touched all process and have a request actually go back to a process which was already used. Read: http://www.dscpl.com.au/wiki/ModPython/Articles/TheProcessInterpreterModel Since you appear to be using prefork MPM, also read about the dangers of prefork MPM in: http://blog.dscpl.com.au/2009/03/load-spikes-and-excessive-memory-usage.html If you want better control over how many processes are used for your application and particularly so you can limit number to small fixed number and/or use threading as well, then look at mod_wsgi daemon mode instead. BTW, the process IDs in your 'ps' output don't even match what your Python code is producing. Looks like you aren't even talking to the same Apache instance. Alternatively you have got some really silly Apache configuration setup, such a MaxRequestsPerchild set to 1. But even in that case, you would have expected some of those initial process to have been recycled. So, try and validate your configuration and whether you might have multiple Apache installations running. Look for 'httpd' processes as well. Graham > mikrov at capa-server:/srv/capa/www/0.9$ sudo ps aux | grep apache2 > mikrov 7396 0.0 0.1 3004 764 pts/2 S+ 11:11 0:00 grep > apache2 > root 22343 0.0 1.0 13728 5336 ? Ss May14 0:26 > /usr/sbin/apache2 -k start > root 22346 0.0 0.6 13728 3136 ? S May14 0:04 > /usr/sbin/apache2 -k start > root 22347 0.0 0.6 13728 3136 ? S May14 0:04 > /usr/sbin/apache2 -k start > root 22348 0.0 0.6 13728 3136 ? S May14 0:04 > /usr/sbin/apache2 -k start > root 22349 0.0 0.6 13728 3136 ? S May14 0:04 > /usr/sbin/apache2 -k start > root 22350 0.0 0.6 13728 3136 ? S May14 0:04 > /usr/sbin/apache2 -k start > root 22383 0.0 0.6 13728 3136 ? S May14 0:04 > /usr/sbin/apache2 -k start > root 22392 0.0 0.6 13728 3136 ? S May14 0:04 > /usr/sbin/apache2 -k start > root 22407 0.0 0.6 13728 3136 ? S May14 0:04 > /usr/sbin/apache2 -k start > root 22716 0.0 0.6 13728 3136 ? S May14 0:03 > /usr/sbin/apache2 -k start > root 23356 0.0 0.6 13728 3136 ? S May14 0:03 > /usr/sbin/apache2 -k start > > mikrov at capa-server:/srv/capa/www/0.9/sessionservice$ sudo tail -f > /var/log/syslog | grep REQUESTINFO > May 15 11:52:03 capa-server capa: REQUESTINFO: Interpreter Name = > /srv/capa/www/0.9/sessionservice/ > May 15 11:52:03 capa-server capa: REQUESTINFO: os.getpid() = 7953 > > May 15 11:52:03 capa-server capa: REQUESTINFO: reloadchecker = 1 > > May 15 11:52:03 capa-server capa: REQUESTINFO: Interpreter Name = > /srv/capa/www/0.9/sessionservice/ > May 15 11:52:03 capa-server capa: REQUESTINFO: os.getpid() = 7954 > > May 15 11:52:03 capa-server capa: REQUESTINFO: reloadchecker = 1 > > May 15 11:52:04 capa-server capa: REQUESTINFO: Interpreter Name = > /srv/capa/www/0.9/sessionservice/ > May 15 11:52:04 capa-server capa: REQUESTINFO: os.getpid() = 7955 > > May 15 11:52:04 capa-server capa: REQUESTINFO: reloadchecker = 1 > > May 15 11:52:05 capa-server capa: REQUESTINFO: Interpreter Name = > /srv/capa/www/0.9/sessionservice/ > May 15 11:52:05 capa-server capa: REQUESTINFO: os.getpid() = 7959 > > May 15 11:52:05 capa-server capa: REQUESTINFO: reloadchecker = 1 > > May 15 11:52:06 capa-server capa: REQUESTINFO: Interpreter Name = > /srv/capa/www/0.9/sessionservice/ > May 15 11:52:06 capa-server capa: REQUESTINFO: os.getpid() = 7960 > > May 15 11:52:06 capa-server capa: REQUESTINFO: reloadchecker = 1 > > May 15 11:52:07 capa-server capa: REQUESTINFO: Interpreter Name = > /srv/capa/www/0.9/sessionservice/ > May 15 11:52:07 capa-server capa: REQUESTINFO: os.getpid() = 7961 > > May 15 11:52:07 capa-server capa: REQUESTINFO: reloadchecker = 1 > May 15 11:52:08 capa-server capa: REQUESTINFO: Interpreter Name = > /srv/capa/www/0.9/sessionservice/ > May 15 11:52:08 capa-server capa: REQUESTINFO: os.getpid() = 7963 > May 15 11:52:08 capa-server capa: REQUESTINFO: reloadchecker = 1 > May 15 11:52:08 capa-server capa: REQUESTINFO: Interpreter Name = > /srv/capa/www/0.9/sessionservice/ > May 15 11:52:08 capa-server capa: REQUESTINFO: os.getpid() = 7964 > May 15 11:52:08 capa-server capa: REQUESTINFO: reloadchecker = 1 > May 15 11:52:09 capa-server capa: REQUESTINFO: Interpreter Name = > /srv/capa/www/0.9/sessionservice/ > May 15 11:52:09 capa-server capa: REQUESTINFO: os.getpid() = 7966 > May 15 11:52:09 capa-server capa: REQUESTINFO: reloadchecker = 1 > May 15 11:52:10 capa-server capa: REQUESTINFO: Interpreter Name = > /srv/capa/www/0.9/sessionservice/ > May 15 11:52:10 capa-server capa: REQUESTINFO: os.getpid() = 7968 > May 15 11:52:10 capa-server capa: REQUESTINFO: reloadchecker = 1 > May 15 11:52:11 capa-server capa: REQUESTINFO: Interpreter Name = > /srv/capa/www/0.9/sessionservice/ > May 15 11:52:11 capa-server capa: REQUESTINFO: os.getpid() = 7969 > May 15 11:52:11 capa-server capa: REQUESTINFO: reloadchecker = 1 > May 15 11:52:12 capa-server capa: REQUESTINFO: Interpreter Name = > /srv/capa/www/0.9/sessionservice/ > May 15 11:52:12 capa-server capa: REQUESTINFO: os.getpid() = 7970 > May 15 11:52:12 capa-server capa: REQUESTINFO: reloadchecker = 1 > May 15 11:52:13 capa-server capa: REQUESTINFO: Interpreter Name = > /srv/capa/www/0.9/sessionservice/ > May 15 11:52:13 capa-server capa: REQUESTINFO: os.getpid() = 7972 > May 15 11:52:13 capa-server capa: REQUESTINFO: reloadchecker = 1 > May 15 11:52:13 capa-server capa: REQUESTINFO: Interpreter Name = > /srv/capa/www/0.9/sessionservice/ > May 15 11:52:13 capa-server capa: REQUESTINFO: os.getpid() = 7973 > May 15 11:52:13 capa-server capa: REQUESTINFO: reloadchecker = 1 > May 15 11:52:14 capa-server capa: REQUESTINFO: Interpreter Name = > /srv/capa/www/0.9/sessionservice/ > May 15 11:52:14 capa-server capa: REQUESTINFO: os.getpid() = 7974 > May 15 11:52:14 capa-server capa: REQUESTINFO: reloadchecker = 1 > May 15 11:52:15 capa-server capa: REQUESTINFO: Interpreter Name = > /srv/capa/www/0.9/sessionservice/ > May 15 11:52:15 capa-server capa: REQUESTINFO: os.getpid() = 7976 > May 15 11:52:15 capa-server capa: REQUESTINFO: reloadchecker = 1 > May 15 11:52:16 capa-server capa: REQUESTINFO: Interpreter Name = > /srv/capa/www/0.9/sessionservice/ > May 15 11:52:16 capa-server capa: REQUESTINFO: os.getpid() = 7977 > May 15 11:52:16 capa-server capa: REQUESTINFO: reloadchecker = 1 > > mikrov at capa-server:/srv/capa/www/0.9$ sudo ps aux | grep apache2 > mikrov 7457 0.0 0.1 3004 764 pts/2 S+ 11:12 0:00 grep > apache2 > root 22343 0.0 1.0 13728 5336 ? Ss May14 0:26 > /usr/sbin/apache2 -k start > root 22346 0.0 0.6 13728 3136 ? S May14 0:04 > /usr/sbin/apache2 -k start > root 22347 0.0 0.6 13728 3136 ? S May14 0:04 > /usr/sbin/apache2 -k start > root 22348 0.0 0.6 13728 3136 ? S May14 0:04 > /usr/sbin/apache2 -k start > root 22349 0.0 0.6 13728 3136 ? S May14 0:04 > /usr/sbin/apache2 -k start > root 22350 0.0 0.6 13728 3136 ? S May14 0:04 > /usr/sbin/apache2 -k start > root 22383 0.0 0.6 13728 3136 ? S May14 0:04 > /usr/sbin/apache2 -k start > root 22392 0.0 0.6 13728 3136 ? S May14 0:04 > /usr/sbin/apache2 -k start > root 22407 0.0 0.6 13728 3136 ? S May14 0:04 > /usr/sbin/apache2 -k start > root 22716 0.0 0.6 13728 3136 ? S May14 0:03 > /usr/sbin/apache2 -k start > root 23356 0.0 0.6 13728 3136 ? S May14 0:03 > /usr/sbin/apache2 -k start > > Best regards > Jakob Simon-Gaarde > > _______________________________________________ > Mod_python mailing list > Mod_python at modpython.org > http://mailman.modpython.org/mailman/listinfo/mod_python >
|