[mod_python] Background threads in mod_python

Graham Dumpleton wrote:
>
> How do you know the thread is being killed? Are you sure it isn't the 
> process
> as a whole?
>
As I debugged the thread is being killed. There was a sample in the 
mailing list that caught an exception to test if anything is going 
wrong. Well there was an exception and that killed the thread. Once that 
problem was fixed, the thread continued on.
> There isn't any fundamental reason why a thread can't outlive the life 
> of the
> request handler, but there are some common mistakes.
>
Ah ok, that does help.
> Main issue that you cannot pass the mod_python request object to such 
> a thread
> and have it hold a reference to it and use the request object beyond 
> the life
> of the request handler. This is because the internals of the request 
> object
> are destroyed when the request handler returns and an access to the 
> request
> object may result in the Apache child process then subsequently crashing.
>
I kind of guessed that. Context: I used to write Apache modules. But 
thank-you for verifying.
> Alternatively, if the Apache child process doesn't crash, you might at 
> least
> cause a Python exception to occur because the state of something in the
> request object isn't as expected. This exception would effectively 
> cause the
> thread to exit unless the context in which the access was made to the 
> request
> object was protected by a try/except block.
>
I copied specific items from the request object to a Python object. 
Though this raises a question. If for example I assign the req.uri to a 
class instance data member is this ok? Will at the end of the request 
the data referenced by req.uri be released, or is req.uri a buffer 
managed by Python?
> In either case, if a Python exception occurs within the thread and it 
> isn't
> caught, it might not actually result in the details of the exception 
> being logged.
> This is because if the exception details are output, they go to 
> standard error
> and anything sent to standard error or standard output when running under
> mod_python isn't guaranteed to show up anywhere. It may if flushing 
> occurs,
> appear in the Apache error log, but I wouldn't depend on it. Sometimes 
> only
> such output ends up in the Apache error log when Apache itself is 
> shutdown
> and those streams get flushed at that point.
>
Thank-you for the reference information.
> In the above I talk about a Python exception occurring due to access to a
> no longer valid request object, but even an uncaught exception in your 
> code
> will cause the same thing, with the thread appearing to exit but with 
> nothing
> necessarily being logged anywhere. What you want to do is to put a 
> try/except
> block around everything the thread does and if an exception occurs, then
> use apache.log_error() to log something to the Apache error log 
> indicating
> that the exception occurs and the details of the exception.
>
Would you consider doing this as a general rule in mod_python? I can see 
the advantages, but am wondering if that could be considered a good 
programming practice?
> One final thing, are you sure that the thread isn't deadlocking on 
> something
> and isn't just stalled? Have you actually used apache.log_error() to log
> something at the exit point of the top level thread function to 
> confirm that
> it is really exiting? Note that exiting explicitly, or because of a 
> Python
> exception occurring within the thread is the only way the thread can 
> truly
> be stopped as there is no way in Python for a thread to be externally 
> killed
> by another thread anyway.
>
Thanks for you help....

Christian