[mod_python] Raising Exceptions

Jeez Graham that's a hefty response!

I moved the exceptions to a separate module and everything
works.

I'll have to re-read the entire response again to make sure
I understand what's going on here.

Thanks for your help,
-John K

On 9/30/06, Graham Dumpleton <grahamd at dscpl.com.au> wrote:
> What you have uncovered is a symptom of some of the problems that
> exist in the module importer in mod_python. The main thing this example
> touches on is that the module importer tries to support itself the
> loading
> of Python packages. This causes various problems and for that reason,
> mod_python will not do that in mod_python 3.3 and will always defer to
> the standard Python importer for importing of packages.
>
> The consequences of this change and other changes in mod_python 3.3
> though are that the way you are trying to structure all of your code
> in the
> document tree as one big package will not necessarily work by default in
> mod_python 3.3. To get it to work you will need to override module
> search
> paths with the PythonPath directive and will have to be very careful not
> to mix Python packages and file based modules in the top directory where
> your PythonHandler directive is used.
>
> Finally, these changes in mod_python 3.3 will mean that there will be no
> automatic module reloading on Python code structured as Python
> packages. Thus, you will need to restart Apache every time you make a
> change to your code due to the way you use packages. Automatic
> module reloading still works with file based Python modules though,
> but your code isn't configured that way.
>
> Anyway, the key problem arises from the fact that you have put the
> definition of the MyError exception class in test/__init__.py. This
> combines
> with problems in the module importer whereby it performs redundant
> reloads
> of __init__.py files within a package. Turns out that this is a
> particular
> issue I haven't noted before and thus it isn't documented in the list of
> importer problems described at:
>
>    http://www.dscpl.com.au/wiki/ModPython/Articles/
> ModuleImportingIsBroken
>
> That you are mixing import and apache.import_module() could still have
> resulted in redundant reloads at other times though and thus similar
> problems. See ISSUE 9 and 10 in that document.
>
> Stepping through this, the first thing that happens when the request
> arrives
> is that mod_python uses apache.import_module() to load your top level
> handler module test.dispatcher and executes the handler in it.
> Because it
> is part of a package, it first loads test/__init__.py and then the
> sub module
> test/dispatcher.py to simulate true package loading. In test/
> dispatcher.py it
> uses import to get the class MyError. This comes from the test/
> __init__.py
> just loaded and already stored in sys.modules.
>
> The next thing is that apache.import_module() is asked to import the
> module test.handlers.testhandler. The apache.import_module() detects
> that
> the module hasn't been loaded before and so needs to load it. The
> problem
> is the importer doesn't check whether it has already loaded the
> __init__.py
> files already for that directory or any parent directories of the
> package and
> thus proceeds to load the test/__init__.py file (a second time) and
> then the
> test/handlers/__init__.py file.
>
> That test/__init__.py is loaded a second time is now the trigger for the
> problem. This is because in reloading the file the definition of
> MyError is
> replaced with a new instances loaded from the file. At this point,
> because
> you imported MyError as:
>
>    from test import MyError
>
> the test/dispatcher.py file holds a reference to the version of
> MyError from
> before the reload. Ie., not the same as that now in test/__init__.py.
> Now
> when test.handlers.testhandler gets loaded, it will find the newer
> version
> of MyError in test/__init__.py. When the testhandler handler raises
> MyError
> it is using the new MyError whereas as the top level handler is
> expecting
> to see the old MyError. Because they are different they will not
> match and
> so it will get caught as an Exception instead.
>
> How do you solve this with the current module importer? The quick answer
> is to never put anything in the __init__.py files of packages being
> imported
> using apache.import_module().
>
> Graham
>
> On 29/09/2006, at 8:09 PM, John Keyes wrote:
>
> > Hi guys,
> >
> > I've included some test code (see below) for some weird
> > behaviour I've noticed.
> >
> > I have dispatcher.py set up as a handler which dynamically
> > imports another handler (testhandler.py) and then calls
> > it's handler function.
> >
> > The handler function in testhandler throws a user defined
> > exception (see __init__.py) which the dispatcher is
> > explicity set up to catch (see except MyError).
> >
> > If I import testhandler using apache.import_module I
> > get the following output:
> >
> >  test
> >  testhandler - b4 raise
> >  exception - MyError - jk
> >
> > Yet if I use __import__ and getattr I get the following
> > output:
> >
> >  test
> >  testhandler - b4 raise
> >  myerror
> >
> > As you can see using import_module the incorrect except
> > block is executed, but the class name of the exception
> > is correct.
> >
> > Can anyone explain to me why this is the case?
> >
> > I've the code included inline but if you want to run the
> > code I've attached a zip file as well.
> >
> > Apache     - 2.0.58
> > mod_python - 3.2.8
> > Python     - 2.4.3
> >
> > Cheers,
> > -John K
> >
> > == test.__init__.py ==
> > class MyError(Exception):
> >    def __init__(self, msg):
> >        self.msg = msg
> >
> >    def __str__(self):
> >        return self.msg
> >
> > == test.dispatcher.py ==
> > from mod_python import apache
> >
> > from test import MyError
> >
> > def handler(req):
> >    req.write("\ntest")
> >
> >    try:
> >        #test_handler = apache.import_module
> > ('test.handlers.testhandler')
> >        test_handler = my_import('test.handlers.testhandler')
> >        val = test_handler.handler(req)
> >    except MyError, me:
> >        req.write("\nmyerror")
> >    except Exception, e:
> >        req.write("\nexception - %s - %s" % (e.__class__.__name__, e))
> >
> >    return apache.OK
> >
> > def my_import(name):
> >    mod = __import__(name)
> >    components = name.split('.')
> >    for comp in components[1:]:
> >        mod = getattr(mod, comp)
> >    return mod
> >
> > == test.handlers.testhandler.py ==
> > from mod_python import apache
> >
> > from test import MyError
> >
> > def handler(req):
> >    req.write("\ntesthandler - b4 raise")
> >
> >    raise MyError('jk')
> >
> >    req.write("\ntesthandler - af raise")
> >    return apache.OK
> > <import_test.zip>
> > _______________________________________________
> > Mod_python mailing list
> > Mod_python at modpython.org
> > http://mailman.modpython.org/mailman/listinfo/mod_python
>