[mod_python] CGI handler and multithreading: Can i have multiple sys.stdout objects?

mod_python has a nice "cgihandler", that allows me to run out old CGI code in
mod_python 'mode'. However, I have some problems with that, that some of you
may know a solution to.

The cgihandler will lock until the request is complete. This is OK, but it
restrains the number of simultaneous Python requests to just 1. In practice,
this means that the system gets a lot slower with mod_python, because there
are a few requests that send a few megabytes of data to the client, and this
can take a few minutes. The performance hit is unacceptable.

I'm running Apache 2.0.44 in "worker" thread mode on a Solaris 2.6 system.

A workaround is to compile apache with the "pre-fork" system. This however
yields another problem. The child processes all fire up their own interpreter,
and the memory load is about 10MB for each child (in 'worker' mode, only one
interpreter for 20 childs!). This quickly eats up all resources on the
machine. It also virtually eliminates the possibility to "recycle" the
database connection and similar objects. The net effect is that this setup is
also slower than the plain CGI implementation which just starts (runs and
kills) an interpreter for each CGI request.

I tried removing the thread lock in the cgihandler. All CGI scripts I have
don't care about the directory they are in (they get all they need from the
MySQL DB), and all scripts they import are fully re-entrant (or MT safe)
themselves. This left only one single problem: The "print" command.

Python's print command is being used to send the data to the client. The
cgihandler replaces sys.stdout and redirects it to the Request object, which
is ok. But there is only one "sys" module shared by all threads, so all the
print commands from all threads end up in the same request. In practice, if a
"long" request is running, and a shorter request interferes, the short one
will kill sys.stdout and the long one will abort with an exception because
sys.stdout had suddenly died.

One workaround would be to create a cgihandler 'clone' that passes an "output"
object to each script (which defaults to sys.stdout for "plain" CGI mode), and
let the scripts all write() to that object. Drawback is that I will have to
replace all "print" statements in about 10k+ lines of code. Also, I want the
scripts to remain CGI compatible for a few more weeks because a few shadow
systems cannot be upgraded to mod_python yet.

Is there a way that I can let the threads each "print" to their own output,
for example by having a seperate "sys" module in each thread? (could I delete
the sys module and re-import it or something similar?)

--
Mike Looijmans