[mod_python] Background threads in mod_python

> So, why not just let the client wait for the data? Neither server nor client have any objection
> keeping a connection open for an hour or longer. Some proxies will appreciate it if you send some
> data every now and then.

Of course if the connection is just sitting idle you'll be wasting a
lot of expensive resources too (namely the open socket and Apache
process/thread slot).  If you find yourself in this situation, then it
is really when you're departing from the native semantics of HTTP
(which doesn't mean its wrong, just that you may need something
different from HTTP).

The way to cobble this onto HTTP of course is to use one request to
initiate a background task, and subsequent request(s) to poll for the
result.  (This is where Ajax techniques can hide this cobbled approach
from the user so it appears seamless even though it's not).  It's not
perfect and far from the most efficient way, but it can be made to
work.  When mixing with the pure Apache architecture though, you're
best served if that background task is done in an independent process
than Apache itself.

Of course if you are really doing this sort of complex communications
a lot, perhaps HTTP just isn't the best fit.  Perhaps it might be just
enough to use something a little less HTTP-pure than Apache; such as a
standalone Twisted server or similar.  Or even something more serious
like CORBA (or any of the newer re-inventions of it) may be worth
investigating.  Those technologies often provide much better
asynchronous service models--which anything layered on top of HTTP
(including "web services") can never do as well.

> There's more to security than authentication.
> ...
> Further, by restricting access to the DB so that only the "business
> layer" web service can access it, it is impossible to retrieve
> sensitive information from the system, even if the web server is
> compromised.

Sometimes this separation is even necessary.  For instance I run with
a rather strict SELinux environment (which is the Linux version of
Mandatory Access Control (MAC)).  The OS kernel actually prevents my
Apache process (and hence my mod_python handlers) from accessing most
of the system's resources.  Apache can't for instance actually touch
the database, except through a very narrow (security-wise) IPC
channel.  Likewise for many other resources like filesystems.  So I'm
actually forced to put background tasks (synchronous or not) into a
separate process.
-- 
Deron Meranda