[mod_python] RE: Bug in mod_python.c causing multiple/redundantinterpreter creation.

Graham Dumpleton grahamd at dscpl.com.au
Sun Oct 17 16:54:08 EDT 2004

On 17/10/2004, at 9:32 AM, Graham Dumpleton wrote:

> I knew there was an issue with apache.py as well, with there being no 
> locking,
> but didn't mention it in my email with the fix as I wanted to check 
> that that
> was the source of the other problem I was seeing and make sure I came 
> up with
> a proper fix.

Turns out I had made the fix to the import_module() method as the first 
thing I
did, looks like I forgot. This is actually the way I uncovered that 
there was still
a further problem and thus the bug with creation of multiple 
interpreters. I spent
a good few hours tracking the problem done and then coming up with a 
fix. Drove me
nuts for a while as I knew exactly what I had to do to fix it but the 
result would
just hang. Finally realised the locking had to be done outside of the 
GIL locking
and not inside. Putting it inside later resulted in deadlock.

Anyway, the fix to apache.py I am running with at the moment is:

   from threading import Lock
   class Lock:
     def acquire(self): pass
     def release(self): pass

_lock = Lock()


def import_module(module_name, autoreload=1, log=0, path=None):
     return _unsafe_import_module(module_name, \

def _unsafe_import_module(module_name, autoreload=1, log=0, path=None):
     Get the module to handle the request. If
     autoreload is on, then the module will be reloaded
     if it has changed since the last import.


Ie., create the lock in a way that will work if no threading or using 
Python 2.2
and then rename existing import_module() and wrap that in safe locking 

Have done away with the funny check against reloading as doesn't seem 
because of the way apache.py is initially loaded. Theoretically it 
could get
screwed if someone did an import_module() call for "mod_python.apache" 
but that
is unlikely.

This obviously is to be used in conjunction with the mod_python.c fix 

I also worked out what the other strange thing that was happening and 
which had
me worried. Namely, even though I had fixed it so that only one 
was being created per process, was still seeing the apache.py module 
get loaded
more than once. Turned out to be that there were still two interpreters 
created. These were the one for the machine host and then later on the 
for the "main_interpreter". If PythonInterpreter was set to 
then only saw apache.py being loaded once as one would expect.

I am happy now that everything is working okay with these two changes 
and I
believe I can safely release another version of Vampire which works 
in multithreaded environment. I will just need to include details of 
these bugs
so people are aware of them and can fix them if they want to.

Next question is whether or not if I write a reimplemented 
import_module() method
which fixes the mod_python.publisher single module name instance 
problem and which
can optionally reload into a fresh module to avoid problems with code 
being changed
from underneath a separate executing thread, whether anyone is 
interested. I may
have to provide actual examples of the latter problem, as not sure 
people may
have been convinced of my previous description of the problem.

The other option I am investigating is how to integrate 
mod_python.publisher style
functionality into the structure of Vampire. This would avoid 
suggesting that
import_module() be rewritten as have already solved the problems in the 
module loading mechanism. The only thing is that I am not too keen on 
some aspects
of how mod_python.publisher works so have to come up with a mechanism 
for doing it
which sits right with my personal view on things. :-)

Graham Dumpleton (grahamd at dscpl.com.au)

More information about the Mod_python mailing list