[mod_python] Re: forks, daemons, and other beasts

Daniel Nogradi writes:
> > >>>> The documentation for thread says that
> > >>>> When the main thread exits, it is system defined whether the other
> > >>>> threads survive. On SGI IRIX using the native thread
> > >>>> implementation,
> > >>>> they survive. On most other systems, they are killed without
> > >>>> executing
> > >>>> try ... finally clauses or executing object destructors.
> > >>>> I tested this and indeed whatever is called with
> > >>>> thread.start_new_thread it will die if execution reaches the end of
> > >>>> the original program.
> > >>>
> > >>> This may be completely off, but how about sending the
> > >>> notification, closing the connection to the client, then
> > >>> unzipping in the same thread?
> > >>
> > >> I doubt the client socket connection would be closed off if one
> > >> did that
> > >> and if no content length was specified, the client browser would
> > >> probably
> > >> hang waiting for socket connection to close and thus would not
> > >> necessarily
> > >> render what it had already received. In other words, the content
> > >> handler
> > >> has to return for socket connection to get closed off. That is
> > >> also presuming
> > >> that keep alive is not enabled.
> > >>
> > >> Graham
> > > I think you might've misunderstood what I meant, or else I said it
> > > wrong. I'm talking about sending some data along, then manually
> > > doing whatever the mod_python equivalent of shutdown(socket,
> > > SHUT_RDWR) is, then performing what you need to do (in this case
> > > unzipping). AFAIK, this guarantees that the connection is closed by
> > > sending FIN, rather than the alternative, simply closing it (which
> > > will leave the connection hanging if the fd is open in any other
> > > threads or processes).
> > >
> > > Another possibility is dumping the zip file into an incoming
> > > directory, then having a cron job or scheduled task periodically
> > > unzip any files in that directory. I don't know if this would work
> > > for your application, though.
> >
> > I knew what you meant. It simply isn't possible in mod_python/Apache
> > to say
> > shutdown the socket connection. There are potentially various layers
> > of Apache
> > filters and other stuff between the mod_python handler and the socket
> > and
> > mod_python doesn't have a direct handle on the socket that it can
> > manipulate.
> > Thus, in the context of mod_python, it isn't that simple.
> 
> One more thought: it seems that if MaxRequestsPerChild is 1, then
> anything threaded from the script handling the request will die when
> the script ends, but if MaxRequestsPerChild is N > 1, then it doesn't
> die, so everything seems to be okay. But what if the request is
> exactly the Nth for that particular child, then it will be the same as
> before, and the thread will die, won't it?


As has been discussed there are interoperability issues with apache if
you use spawn* calls, threading and I can't imagine what would happen
with exec* calls.

If you are on a unix system I think the easiest way to go is put your
post-processing in a script (a python script of course!) that traps
the TERM signal and then use the os.system() call in your handler.

For example, if you have this script, /tmp/foo.py:

    ############################################################
    # /tmp/foo.py

    import signal
    import time

    # ignore the TERM signal, so if apache kills the child process
    # that forked me I won't die.
    signal.signal(signal.SIGTERM, signal.SIG_IGN)

    # Your Code Here!
    # for this demo we'll sleep, so you have time to stop apache and
    # see this is still running
    time.sleep(30)

    # some output to prove we ran
    print "hello, world"

    #
    ############################################################


Then in your handler:

    os.system('python /tmp/foo.py >& /tmp/foo.log &')

If you're not familiar with unix: the above executes the script as if
you typed it in a terminal running a shell.  Stdout and stderr are
redirected to /tmp/foo.log, else they will be inherited from apache's,
which is probably not what we want.  The trailing ampersand will put
the job in the background, detaching from the parent process.

Hope this helps!

Daniel Popowich
---------------
http://home.comcast.net/~d.popowich/mpservlets/