[mod_python] mod_python executing old versions of my code

What you are seeing is normal behaviour for mod_python for the way your
code is set up. It is one of a number of cases which highlight the
shortcomings of the module importing and automatic module reloading
system of mod_python. You can read more about some of the problems
with module importing here:

  http://www.dscpl.com.au/articles/modpython-003.html

More information interspersed below.

James Paige wrote ..
> I'm new to mod_python... so maybe I have missed something fundamental 
> here, but mod_python seems to be keeping several copies of my old code
> in memory, so when I make changes and reload the page, sometimes I get
> my changes, and sometimes I get a seemingly-random version of the code
> that I have already changed.
> 
> Here is a simple test-case I set up.
> 
> Here is my .htaccess file:
> 
> -------------------------
> SetHandler python-program
> PythonHandler main
> PythonDebug On
> -------------------------
> 
> (I am using version 2.7.10 as shipped in Debian stable's 
> libapache-mod-python package, which is why I am using "python-program")
> 
> Here is my main.py
> 
> -----------------------------
> from mod_python import apache
> import testmodule
> 
> def handler(req):
>      req.write("Hello World!")
>      testmodule.foo(req)
>      return apache.OK
> -----------------------------
> 
> and here is the imported testmodule.py
> 
> --------------------------------
> def foo(req):
>     req.write('three blind mice')
> --------------------------------
> 
> The first time I load this test in my web browser, I see the expected 
> result:
> 
>    Hello World!three blind mice
> 
> I edit testmodule.py and change the string to something else:
> 
> --------------------------------------
> def foo(req):
>     req.write('mary had a little lamb')
> --------------------------------------
> 
> I then reload my web browser. Sometimes this results in "Hello 
> World!mary had a little lamb", and sometimes it reuslts in "Hello 
> World!three blind mice"

When you see what you expect, it is because the request was handled by a
different Apache child process which had never previously loaded your
code. Since it is the first time it has needed to load it, it will
obviously get the most recent version.

When you don't see what you expect, it is because the request was
handled by the the existing Apache child process which had already
loaded the code.

> I edit the string again, and hit reload again. Maybe I get the new 
> string.... or maybe I get one of the strings I used previously. I keep
> making changes to testmodule.py and I keep reloading the results, and 
> the results are almost always wrong.
> 
> I know this is not a browser cache problem. I have cleared/disabled my
> web-browser's cache.
> 
> I read about "Multiple Interpreters" in the documentation, and have 
> tried to force my code to run in a single interpreter by adding:
> 
> PythonInterpreter "test_interpreter"
> 
> to my .htaccess file, but that makes no difference.
> 
> This problem only occurs in the included module. I can edit main.py and
> my changes are always applied. I notice in the apache logs that 
> everytime I reload the page I see:
> 
>    [notice] mod_python: (Re)importing main from None

The automatic module reloading only applies to the top most module,
ie., main.py, and not to anything imported using "import".

> But I can find no way to force testmodule to be re-imported. What am I
> doing wrong?

You are not doing anything wrong, the current system simply doesn't
work very well.

The best you can get is to use:

  from mod_python import apache
  import os

  __here__ = os.path.dirname(__file__)
  
  def handler(req):
       req.write("Hello World!")
       testmodule = apache.import_module("testmodule,path=[__here__])
       testmodule.foo(req)
       return apache.OK

This uses apache.import_module() on each request to import the module.
In practice though, if the file has not changed it just uses what is already
in the cache. If it has changed, you should then see the change.

You can't use apache.import_module() at global scope in the module and
get the same effect as that only gets executed when main.py is imported.
Thus you don't get the check for changes to testmodule.py on every
request.

I have a completely rewritten module importing system for mod_python
which addresses nearly all the issues it has. I will be making it available for
trialing after mod_python 3.2.6 is officially released. It will require though
having 3.2.6, which means it will be useless to you with 2.7.11.

Whether this new module importing system gets adopted is another
matter entirely though. Some have argued in the past that the current
arrangement is adequate even though it has known problems. ;-(

Graham