[mod_python] Module reload/derived class problem

On 05/03/2005, at 5:42 AM, Alan Davies wrote:

> Now that I've realised why I get that error message, I can avoid the
> problem by brute force either by 'touch'ing 'B' every time I change
> 'A', or by restarting Apache after a change to 'A' (instead of a week
> later when I notice the problem).
>
> More advanced ideas for fixing the module reloading problem would be
> appreciated.

The problem derives from two short comings of the present module loading
system used by mod_python. The first is that module reloads are done on
top of the already loaded module. This in itself can cause some issues.
The second is that the module loading system isn't able to determine 
that
a parent module should be reloaded when a child module it depends on has
been changed.

The trigger for your problem is the second of these issues, but the 
cause
of the error actually derives from the first issue.

Namely, the parent module module has what are effectively stale 
references
to a type object which has since been superseded because the child 
module
was re-imported as a result of some other request. When the child module
code is called it no longer identifies the instance as being of the same
type because the type object for the child has since been changed.

FWIW, the module loading system in Vampire doesn't suffer the problem.
When I write the code as:

## testa.py

class A:
   def foo(self):
     return "A:foo"

## testdrive.py

import vampire
import os

directory = os.path.dirname(__file__)
testa = vampire.importModule("testa",directory,__req__)

class B(testa.A):
   def foo(self):
     return "B::foo", testa.A.foo(self)

b = B()

and using the new publisher support in version of Vampire I am working 
on,
if I access "/testdrive/b/foo" and then touch "testa.py" it still works
when I access "/testdrive/b/foo" again.

This is because Vampire has a means of determining that "testdrive.py"
depends on "testa.py" and thus when "testa.py" is modified, it will 
without
any prompting reload both "testdrive.py" and "testa.py" on the 
subsequent
request even though "testdrive.py" hadn't actually been touched.

This is illustrated by logged messages. After a fresh restart, access 
the
URL:

[Sat Mar 05 13:41:57 2005] [notice] mod_python: (Re)importing module 
'vampire'
[Sat Mar 05 13:41:57 2005] [notice] vampire: Importing module 
'/Users/grahamd/Sites/vampire/examples/publisher/testdrive.py'
[Sat Mar 05 13:41:57 2005] [notice] vampire: Importing module 
'/Users/grahamd/Sites/vampire/examples/publisher/testa.py'

Then run "touch testa.py". The "testdrive.py" file has not been touched 
or
modified in any way and thus still has same modification time. Then 
access
the URL again:

[Sat Mar 05 13:42:10 2005] [notice] vampire: Reimporting module 
'/Users/grahamd/Sites/vampire/examples/publisher/testdrive.py'
[Sat Mar 05 13:42:10 2005] [notice] vampire: Reimporting module 
'/Users/grahamd/Sites/vampire/examples/publisher/testa.py'

Thereby showing how both parent and child were reimported even though 
only
child had been changed.

It is because it was detected that the parent depended on the modified 
child and
was also reloaded, you don't suffer the sort of problem you are seeing.

Thus for most cases the Vampire module loading system is a bit more 
resilient
that the default provided with mod_python. I still know of some cases 
which
it wouldn't be able to handle, but it is impossible to make any 
reloading
system completely perfect because of the various ways that type objects 
and
class instances could be cached and then later used again in the 
context of
freshly reloaded code which has a different idea of what it then 
expects.

Thus, as to a technical solution, you might consider using Vampire. If 
you
want the publisher support, you'll either have to wait for the next 
version
of Vampire or indicate that you are prepared to work with a beta version
which some others are also trying.

Graham