[mod_python] Apache, Threading and Multi-Processing Modules

Mike Looijmans mike.looijmans at asml.com
Thu Jun 12 07:22:51 EST 2003

> For example, on a Windows platform where there is a single 
> multi-threaded Apache process (mpm_wint 
> [http://httpd.apache.org/docs-2.0/mod/mpm_winnt.html]) is it correct to 
> say that mod_python would not be able to take advantage of a 
> multi-processor machine due to the GIL?

I don't know what a "GIL" is, but Windows (NT) and apache are both definitely 
capable of taking advantage of multi-processor machines correctly.

> In another, given Apache running in the prefork MPM 
> [http://httpd.apache.org/docs-2.0/mod/prefork.html]- is it a) possible 
> or b) useful to have a global, per-Apache-process persitant data 
> strucuture sharing a pool of (threadsafe) database connections. 

As you already concluded, there is not shared data among the processes, so 
setting up communication between them is probably not worth considering. Just 
keep a single DB connection per process, something like:

db = None

def connect():
    if not db:
    	db = MySQLdb.connect(...)
    return db

> Taking the specific example of database connections (let me note I have 
> read and believe I understand FAQ 3.3) is it ever useful or possible to 
> share a pool of database connectors, rather than a single connector in 
> the global namespace. I assume that code such as that in FAQ 3.3 would 
> require additional locking mechanisms in order to function correctly in 
> a multi-threaded Apache environment?

It is both useful and possible to do so. Especially when the DBMS is on another 

Most databases allow only one session per thread, and as such the best thing to 
do is to make sure that a connection is used by only one thread at a time.
> I bet there must be some code in existing projects that does stuff like 
> this. Any pointers?

No pointers, but what I have been using here for months now on a "worker" mpm:

- Create a threading.Lock object.
- Create a dictionary (of lists) or list for the db connection pool (I use a 
dictionary because I have 4 databases where my clients want to grab data)

To get a connection, the lock is aquired and the first matching db connection is 
taken and removed from the pool, if possible. The lock is released and the 
connection returned. If the pool was empty, a new db conenction object is 
created and returned.

When a thread finishes handling a request, it returns the connection object back 
to the pool. (obtain lock, put connection into pool, release lock)

This guarantees that no two threads share the same connection, and still creates 
no more connections than neccesary.

Mike Looijmans
ASML: http://www5nl.asml.nl/~mlooijma
Private: http://www.milosoftware.com
-- MY text ends here --

The information contained in this communication and any attachments is confidential and may be privileged, and is for the sole use of the intended recipient(s). Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please notify the sender immediately by replying to this message and destroy all copies of this message and any attachments. ASML is neither liable for the proper and complete transmission of the information contained in this communication, nor for any delay in its receipt.

More information about the Mod_python mailing list