nogradi at gmail.com
Sat Feb 11 15:39:55 EST 2006
Sorry everyone, a long post follows :) Many times there is need for a mod_python program to respond (send a page) to the client after receiving a request and at the same time start a subprocess which typically takes quite long and waiting for it to finish is not necessary. So the situation is this: (1) request comes from client (2) server processes the request (3) server starts a subprocess and don't wait for it to finish (4) send a response to the client. Actually, (3) and (4) could be interchanged. What is the best way to do this? One could of course use the exec* and spawn* family of functions, but in that case one would need to have a separate program to run and all variables (lists, classes, etc.) used by the program should be translated to command line arguments which can be cumbersome. I've learned from a recipe in the python cookbook, http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/66012, that this sort of thing in a python application would be done by forking the process twice (just like in unix) like this: import sys, os def daemonize ( stdin='/dev/null', stdout='/dev/null', stderr='/dev/null' ): # Perform first fork. try: pid = os.fork() if pid > 0: sys.exit(0) # Exit first parent. except OSError, e: sys.stderr.write("fork #1 failed: (%d) %s\n" % (e.errno, e.strerror)) sys.exit(1) # Decouple from parent environment. os.chdir("/") os.umask(0) os.setsid() # Perform second fork. try: pid = os.fork() if pid > 0: sys.exit(0) # Exit second parent. except OSError, e: sys.stderr.write("fork #2 failed: (%d) %s\n" % (e.errno, e.strerror)) sys.exit(1) # The process is now daemonized, redirect standard file descriptors. for f in sys.stdout, sys.stderr: f.flush() si = file(stdin, 'r') so = file(stdout, 'a+') se = file(stderr, 'a+', 0) os.dup2(si.fileno(), sys.stdin.fileno()) os.dup2(so.fileno(), sys.stdout.fileno()) os.dup2(se.fileno(), sys.stderr.fileno()) def this_will_run_as_a_daemon( ): # some stuff here daemonize( '/dev/null', '/tmp/daemon.log', '/tmp/daemon.log' ) this_will_run_as_a_daemon( ) All of this is working fine with python, the result is what one would expect, whatever is in this_will_run_as_a_daemon( ) will run as a daemon in the background. Now I have difficulty with doing a similar thing with mod_python and the publisher handler. I tried the following (the above daemonize function is in daemon.py): from daemon import * def foo( req ): daemonize( '/dev/null', '/tmp/daemon.log', '/tmp/daemon.log' ) # some stuff here that should run in the background # and the program should just start it and carry on return "okay" and then accessing it by a client gives strange behaviour, either does it what it supposed to do or gives the following error: Mod_python error: "PythonHandler mod_python.publisher" Traceback (most recent call last): File "/usr/lib/python2.3/site-packages/mod_python/apache.py", line 299, in HandlerDispatch result = object(req) File "/usr/lib/python2.3/site-packages/mod_python/publisher.py", line 136, in handler result = util.apply_fs_data(object, req.form, req=req) File "/usr/lib/python2.3/site-packages/mod_python/util.py", line 361, in apply_fs_data return object(**args) File "/var/www/html/python/test.py", line 5, in d daemonize('/dev/null', '/tmp/moddaemon.log', '/tmp/moddaemon.log') File "/var/www/html/python/daemon.py", line 27, in daemonize sys.exit(0) # Exit second parent. SystemExit: 0 I guess part of the problem is that anything after daemonize( ) really gets 'daemonized' including the return "okay" statement. But then how can it work once in a while? Anyway, I have the feeling that there must be a more sane and probably safe way of achieving what I would like by a much simpler method. Any ideas?