[mod_python] Raising Exceptions

What you have uncovered is a symptom of some of the problems that
exist in the module importer in mod_python. The main thing this example
touches on is that the module importer tries to support itself the  
loading
of Python packages. This causes various problems and for that reason,
mod_python will not do that in mod_python 3.3 and will always defer to
the standard Python importer for importing of packages.

The consequences of this change and other changes in mod_python 3.3
though are that the way you are trying to structure all of your code  
in the
document tree as one big package will not necessarily work by default in
mod_python 3.3. To get it to work you will need to override module  
search
paths with the PythonPath directive and will have to be very careful not
to mix Python packages and file based modules in the top directory where
your PythonHandler directive is used.

Finally, these changes in mod_python 3.3 will mean that there will be no
automatic module reloading on Python code structured as Python
packages. Thus, you will need to restart Apache every time you make a
change to your code due to the way you use packages. Automatic
module reloading still works with file based Python modules though,
but your code isn't configured that way.

Anyway, the key problem arises from the fact that you have put the
definition of the MyError exception class in test/__init__.py. This  
combines
with problems in the module importer whereby it performs redundant  
reloads
of __init__.py files within a package. Turns out that this is a  
particular
issue I haven't noted before and thus it isn't documented in the list of
importer problems described at:

   http://www.dscpl.com.au/wiki/ModPython/Articles/ 
ModuleImportingIsBroken

That you are mixing import and apache.import_module() could still have
resulted in redundant reloads at other times though and thus similar
problems. See ISSUE 9 and 10 in that document.

Stepping through this, the first thing that happens when the request  
arrives
is that mod_python uses apache.import_module() to load your top level
handler module test.dispatcher and executes the handler in it.  
Because it
is part of a package, it first loads test/__init__.py and then the  
sub module
test/dispatcher.py to simulate true package loading. In test/ 
dispatcher.py it
uses import to get the class MyError. This comes from the test/ 
__init__.py
just loaded and already stored in sys.modules.

The next thing is that apache.import_module() is asked to import the
module test.handlers.testhandler. The apache.import_module() detects  
that
the module hasn't been loaded before and so needs to load it. The  
problem
is the importer doesn't check whether it has already loaded the  
__init__.py
files already for that directory or any parent directories of the  
package and
thus proceeds to load the test/__init__.py file (a second time) and  
then the
test/handlers/__init__.py file.

That test/__init__.py is loaded a second time is now the trigger for the
problem. This is because in reloading the file the definition of  
MyError is
replaced with a new instances loaded from the file. At this point,  
because
you imported MyError as:

   from test import MyError

the test/dispatcher.py file holds a reference to the version of  
MyError from
before the reload. Ie., not the same as that now in test/__init__.py.  
Now
when test.handlers.testhandler gets loaded, it will find the newer  
version
of MyError in test/__init__.py. When the testhandler handler raises  
MyError
it is using the new MyError whereas as the top level handler is  
expecting
to see the old MyError. Because they are different they will not  
match and
so it will get caught as an Exception instead.

How do you solve this with the current module importer? The quick answer
is to never put anything in the __init__.py files of packages being  
imported
using apache.import_module().

Graham

On 29/09/2006, at 8:09 PM, John Keyes wrote:

> Hi guys,
>
> I've included some test code (see below) for some weird
> behaviour I've noticed.
>
> I have dispatcher.py set up as a handler which dynamically
> imports another handler (testhandler.py) and then calls
> it's handler function.
>
> The handler function in testhandler throws a user defined
> exception (see __init__.py) which the dispatcher is
> explicity set up to catch (see except MyError).
>
> If I import testhandler using apache.import_module I
> get the following output:
>
>  test
>  testhandler - b4 raise
>  exception - MyError - jk
>
> Yet if I use __import__ and getattr I get the following
> output:
>
>  test
>  testhandler - b4 raise
>  myerror
>
> As you can see using import_module the incorrect except
> block is executed, but the class name of the exception
> is correct.
>
> Can anyone explain to me why this is the case?
>
> I've the code included inline but if you want to run the
> code I've attached a zip file as well.
>
> Apache     - 2.0.58
> mod_python - 3.2.8
> Python     - 2.4.3
>
> Cheers,
> -John K
>
> == test.__init__.py ==
> class MyError(Exception):
>    def __init__(self, msg):
>        self.msg = msg
>
>    def __str__(self):
>        return self.msg
>
> == test.dispatcher.py ==
> from mod_python import apache
>
> from test import MyError
>
> def handler(req):
>    req.write("\ntest")
>
>    try:
>        #test_handler = apache.import_module 
> ('test.handlers.testhandler')
>        test_handler = my_import('test.handlers.testhandler')
>        val = test_handler.handler(req)
>    except MyError, me:
>        req.write("\nmyerror")
>    except Exception, e:
>        req.write("\nexception - %s - %s" % (e.__class__.__name__, e))
>
>    return apache.OK
>
> def my_import(name):
>    mod = __import__(name)
>    components = name.split('.')
>    for comp in components[1:]:
>        mod = getattr(mod, comp)
>    return mod
>
> == test.handlers.testhandler.py ==
> from mod_python import apache
>
> from test import MyError
>
> def handler(req):
>    req.write("\ntesthandler - b4 raise")
>
>    raise MyError('jk')
>
>    req.write("\ntesthandler - af raise")
>    return apache.OK
> <import_test.zip>
> _______________________________________________
> Mod_python mailing list
> Mod_python at modpython.org
> http://mailman.modpython.org/mailman/listinfo/mod_python