[mod_python] Udate python modules without restarting Apache

On Oct 07 17:10, Graham Dumpleton <grahamd at dscpl.com.au> wrote:
>
> Subject: Re: [mod_python] Udate python modules without restarting Apache
>
> On Oct 07 23:50, berry groenendijk <berry.groenendijk at gmail.com> wrote:
> >
> > Subject: Re: [mod_python] Udate python modules without restarting Apache
> >
> > I think the problem is in the servlet.py. Because, as soon as I touch
> > this file I get this error:
> > 
> > ValueError: index is not a subclass of Servlet (in E:/Program
> > Files/Apache Group/Apache2/htdocs/MySite/index.mps)
> > 
> > Before I touched servlet.py the index file loaded perfectly.
> > Apparently apache tried reloading the servlet file effectively
> > creating a new Servlet object in memory. Other pages that have already
> > created in memory and that are subclasses from Servlet have now
> > apparently lost their 'parent'.
> > 
> > Any suggestions how to solve this?
> 
> Hmmm, I haven't tried MPS as yet, looks like it might be time to do so
> in order to understand it a bit better.
> 
> One of the issues with reloading that isn't obvious is that it doesn't
> actually clean out the namespace of the existing loaded module before
> loading it again. In effect it lays the reloading module back over the top
> of the old one. You can thus still have strange things happen if you keep
> state in global variables or remove methods.
> 
> The easiest to explain is the removal of a method from a module which
> is imported by mod_python.publisher. Even though the method is removed
> from the code file, when it reloads the code module it doesn't actually
> delete the existing method, so it still exists and can still be accessed by
> a request.

Okay, have read your email properly now and have had a little play. What
I didn't note was that you had put mod_python.servlet into your directory
and touched it. A bit more of an explaination follows, but simple answer
is that you shouldn't need to touch servlet.py anyway as that isn't going to
help in the reloading of your code. I would also recommend that servlet.py
be installed properly somewhere else and not be in your document tree.
Preferably put it somewhere where its modification time will not be changed
inadvertantly while it is being used.

Anyway, what I think is happening is that in the most low level servlet class
in your common code modules, you have an ordinary import of something
from mpservlets. To use the demo tutorial that comes with mpservlets as an
example, it has _TutorialBase.py which after accomodating that you had
servlet.py in the same directory, has:

  from servlet import HTMLPage

Now in index.mps of the tutorial, if one has modified it to use import_module()
you would have:

  #from _TutorialBase import *

  from mod_python import apache
  _TutorialBase = apache.import_module("_TutorialBase")
  TutorialBase = _TutorialBase.TutorialBase

If Apache is freshly started, all will work okay. The problem now is that if servlet.py
is touched, mod_python will reimport servlet.py the next time a request comes in.

Now, the mpservlet system has its own module import and caching system for .mps
files. In mod_python doing the reimport of servlet.py it will actually throw out the
mpservlet cache contents so when handler() is called it will need to reload index.mps
which in turn obtains _TutorialBase from the standard mod_python cache.

The problem here is that _TutorialBase used a normal "import" to get HTMLPage and
it still holds a reference to the copy that was imported before "servlet.py" got reimported
by mod_python. When the handler() method in servlet.py goes to check to see if the
servlet loaded from index.mps derives from Servlet it doesn't match. This is because
that in _TutorialBase is still using the Servlet base class from the older servlet module
obtained using "import".

In other words, there are two copies of the servlet module in memory. The handler()
method is checking against itself (ie., new copy), where as _TutorialBase is still using
the old one.

Thus, for case where using mpservlet, simply don't touch the servlet.py. If you had
changed _TutorialBase, touch index.mps as well and both will be reloaded.

Although I suggest installing servlet.py somewhere else properly, you could possibly
also use import_module() to load in "servlet.py" as well. Ie.,

  # from servlet import HTMLPage

  from mod_python import apache
  servlet = apache.import_module("servlet")
  HTMLPage = servlet.HTMLPage

BTW, mpservlet doesn't use "imp" module to do module importing of .mps files but
instead uses execfile(). By using execfile() it effectively throws away the prior module before
reimporting it. This means you don't get issues with deleted methods still showing etc.
It also means though that you can't use global variables cleverly to preserve state
across module reloads. You would have to put such magic state in a separate modules
loaded by import_module() or "import" instead. Managing such state is still tricky though.

Hope this helps.

--
Graham Dumpleton (grahamd at dscpl.com.au)