[mod_python] Re: import and mod_python

Graham Dumpleton grahamd at dscpl.com.au
Wed Oct 12 21:30:10 EDT 2005

Wim Heirman wrote ..
> Hello,
> When doing a first toy project in mod_python, I stumbled upon the --
> apparently well-known problem -- of mod_python not reloading custom
> modules imported with 'import'. I think I've found a pretty descent
> solution for it, described below. In search for existing documentation
> on this I mainly found
> http://www.modpython.org/pipermail/mod_python/2004-October/016567.html
> and http://www.dscpl.com.au/articles/modpython-002.html (that's why I'm
> sending this to you), I'm not sure if this method or something similar
> might already be described elsewhere -- if not I suppose other people
> will be interested and I'll write out a descent description and put it
> on my website.
> So this is how it goes:
> Add this to the .htaccess of the directory where the modules are used:
> PythonHeaderParserHandler purge
> Create a purge.py file (in the same directory or somewhere in sys.path)
> with these contents:
> import sys
> from mod_python import apache
> def headerparserhandler(req):
>   prefix = '/user/wheirman'
>   for name, mod in sys.modules.items():
>     print name, mod
>     if getattr(mod, '__file__', '')[:len(prefix)] == prefix:
>       del sys.modules[name]
>   return apache.OK
> purge.py will unload all modules that are located in a certain directory
> (here: everything under my home directory, so not the ones from
> /usr/lib/python or the ones other people may be using on a shared
> production server). The PythonHeaderParserHandler makes that this purge
> is done just before the normal PythonHandler, so when importing a user
> module (using the normal 'import' statement) in the main script, it is
> reloaded from disk. This is done every time the user script is executed
> (so no need to manually browse to 'purge/doit') and for the right forked
> Apache process. In short: no need to modify any code, transparent to the
> developer, and reasonably cheap performance-wise.
> Any comments on this?
> Regards,
> Wim.
> -- 
> ir. Wim Heirman,
> ELIS Department, Ghent University, Belgium
> Phone: +32-9-264.95.27
> E-mail: wim.heirman at elis.UGent.be
> http://www.elis.UGent.be/~wheirman/

Note, response cc'd to mod_python mailing list. Preferable that you get
on the mod_python mailing list and post discussions on such topics

Anyway, deleting modules out of sys.modules like this is dangerous and
can cause various problems if a multithreaded MPM such as "winnt" on
Win32 systems or "worker" on UNIX is used. This is because you could be
deleting the module at the same time as something was first importing
it, or while a later request is executing code held within the module.

The first of these issues could be avoided by acquiring the importer
module lock while deleting modules, but second can't be avoided.
Other strange problems could also result.

The original suggestion I made about deleting stuff out of sys.modules
was intended only to be used in extreme cases where you had no choice
because Apache couldn't be restarted for some reason.

There are better solutions to this problem and Vampire indicates one
direction that can be taken, including having "import" layer on top of a
special purpose module importer so that "import" modules in
selected directories are reloaded when changed. See changes added
in mod_python 1.7 for details:


The whole module importer problem will hopefully be addressed in
mod_python 3.3 now that there is some idea of what can be done.


More information about the Mod_python mailing list