[mod_python] Background threads in mod_python

Mike Looijmans wrote:
> It really depends on WHAT you intend to do. I can think of no reason 
> whatsoever to start a thread in the handler that somehow "survives" 
> the request. Once that thread has created some "answer", where can it 
> send that answer?
>
There is lots of needs for this in an Ajax world. For example Jetty 
(http://www.mortbay.com/MB/log/gregw/?permalink=ScalingConnections.html), 
Apache 2.2, and IIS a few years can do asynchronous processing. There 
are two ways to do asynchronous processing. The first is to lock onto 
the request and hold it and move it into a "secondary" processing area. 
The second is to start a task and then ask if any data has been 
generated. The running task will store in a cache that is then picked up 
by another request at another time.

The big idea here is to mimic an architecture where the server "calls" 
the client, which is very popular in Ajax.
> When Apache is using multiple processes, it will terminate child 
> processes for various reasons (for example, when 
> max_requests_per_child has been reached). That will also terminate any 
> thread you created in that process.
>
Yeah I was looking at this and it is pain. On Windows Apache does not do 
this. They use only two processes and use threads within those processes.
> I suspect your software gets a "job" from the user, and reports back 
> to the user that the job has been started. The job itself must keep 
> running for some time after that. The job creates a file or something 
> similar which can make the user conclude that the job has finished.
>
I would not say "job". I would say long running tasks generating data. 
For example I like to read real-time feeds, and want to generate the 
data. But the client is in control of the task using parameters that are 
sent to server.
> A web service used within a web server is not redundant, it's just a 
> way to delegate tasks to other machines or processes. This is 
> typically done for security reasons.
>
I don't really buy this argument. Let's say I do what you recommend and 
that is call an XML-RPC service. Why, do I need Apache in the first 
place? While I might get security I don't get any added value. If I am 
using web services then Apache is pretty well useless anyways because 
most web services don't use HTTP security. Most web service 
infrastructures use the WS-* specs, or some home-backed tokens that are 
added to the XML package.

Christian Gross