[mod_python] Udate python modules without restarting Apache

Graham Dumpleton writes:

> Now, the mpservlet system has its own module import and caching
> system for .mps files. In mod_python doing the reimport of
> servlet.py it will actually throw out the mpservlet cache contents
> so when handler() is called it will need to reload index.mps which
> in turn obtains _TutorialBase from the standard mod_python cache.
> 
> The problem here is that _TutorialBase used a normal "import" to get
> HTMLPage and it still holds a reference to the copy that was
> imported before "servlet.py" got reimported by mod_python. When the
> handler() method in servlet.py goes to check to see if the servlet
> loaded from index.mps derives from Servlet it doesn't match. This is
> because that in _TutorialBase is still using the Servlet base class
> from the older servlet module obtained using "import".
> 
> In other words, there are two copies of the servlet module in
> memory. The handler() method is checking against itself (ie., new
> copy), where as _TutorialBase is still using the old one.

Excellent description by Graham of what's going on.  You will see
this problem whenever the chain of "imports" is like this:

  mod_python =>
      import servlet =>
          execfile('someservlet.mps') =>
              [import indirectmodule.py =>]+
	          import servlet.py

It's the indirect importing of servlet.py that creates the subclassing
error.  My suggestions:

  0. The easiest way, of course, is "apachectl graceful", but that
     doesn't help you if you don't have privs on your hosting service.

  1. Avoid touching servlet.py.  You can avoid accidental reloading of
     servlet.py by mod_python by setting PythonAutoReload to "off"
     (which is recommended in a production environment).  If you do
     mean to modify servlet.py, consider subclassing Servlet or
     HTMLPage and placing this code in another module.  I would
     recommend subclassing anyway for two reasons: 1) it's the OO way
     to add/replace functionality; 2) as I provide updates you will
     have an easier time merging your modifications.

  2. Use apache.import_module() or some such that will reload modules
     automagically.

BTW, during development my favorite way to avoid these importing
issues is to set MaxRequestsPerChild to 1 in my apache configuration
(prefork MPM).  Apache will then start a new python interpreter per
request.  Performance can be sometimes doggish, but that's OK during
development.  I crank it up to 1000 during beta testing.

> BTW, mpservlet doesn't use "imp" module to do module importing of
> .mps files but instead uses execfile(). By using execfile() it
> effectively throws away the prior module before reimporting it.

The main reason I used execfile is because I wanted to build a cache
of servlet classes keyed on the full filename.  This means I avoid
problems of /somedir/index.mps and /somedir/subdir/index.mps
colliding.  Also, there are so many gnarly issues with using the
import system, I believe I've avoided many a head-ache (e.g., dealing
with pythonpath; wanting to import files that end in .mps).

Cheers!

Daniel Popowich
-----------------------------------------------
http://home.comcast.net/~d.popowich/mpservlets/