[mod_python] The new module loader

Dan Eloff wrote ..
> Thanks a bundle Graham, I get the gist of it. Tomorrow I'll back everything
> up and then start reworking it with all that in mind. Much appreciated.
> The
> pyserver stuff wasn't being reloaded because it's in site-packages, and
> the
> rest of the stuff was being loaded *AFTER* being placed in sys.path. I
> did a
> pretty good job of shooting myself in the foot there. I'll fix it all up
> and
> give it another go and let you know what happens.
> 
> You're right about the packages too, other than as a namespace and for
> convenience, I don't need it, and it makes more sense for it to live
> elsewhere. You put an awful lot of thought into this, I think it will be
> the
> death of all those auto reload woes.

If you are going to go off and restructure your code, I'll let you know of
one more thing that may be useful.

When you use apache.import_module(), like the directive in the Apache
configuration, you can use an absolute pathname instead of a straight
module name.

  mystuff1 = apache.import_module("/some/path/mystuff1.py")

Also, within a module managed by the new importer, as well as being
able to say:

  import mystuff2

or:

  mystuff2 = apache.import_module("mystuff2")

to get "mystuff2" from the same directory, you can also say:

  mystuff2 = apache.import_module("./mystuff2.py")

When using an absolute path or relative path, the file doesn't even
strictly need to have a .py extension. For example, if someone
wanted to rewrite high level dispatcher for mpservlets framework
to use new importer, they could use:

  servlet = apache.import_module("/some/path/servlet.mps")

Unless you have a good reason, I would suggest sticking with .py
extension.

You might perhaps see the next step from "./" above and that is that
relative paths in general work. Thus, to get module from a subdir
you can use:

  subdir_mystuff3 = apache.import_module("./subdir/mystuff3.py")

The "./" is important, you can't just have "subdir/..." it will not work
and will take it as a normal module import rather than a relative path.

Finally, if need be, you can say:

  parentdir_mystuff4 = apache.import_module("../parent/mystuff4.py")

This relative path importing should be sufficient to replace any of the
inter package imports used with a package.

BTW, if you stumble across "~/..." in source code of importer, ignore it
and do not try and use it. Originally I saw it as a useful shortcut, but I
now see problems with it, so it is going to go away.

Graham

> On 4/20/06, Graham Dumpleton <grahamd at dscpl.com.au> wrote:
> >
> > Graham Dumpleton wrote ..
> > > Dan Eloff wrote ..
> > > > <Directory />
> > > >     SetHandler mod_python
> > > >     PythonHandler PyServer.pyserver
> > >
> > > The new module importer completely ignores packages as it is practically
> > > impossible to get any form of automatic module reloading to work
> > > correctly with them when they are more than trivial. As such, packages
> > > are handed off to standard Python __import__ to deal with. That it
> even
> > > finds the package means that you have it installed in sys.path. Even
> if
> > > it was a file based module, because it is on sys.path and thus likely
> to
> > > be installed in a standard location, the new module importer would
> again
> > > ignore it as it leaves all sys.path modules up to Python __import__
> > > as too dangerous to be mixing importing schemes.
> > >
> > > Anyway, that all only applies if you were expecting PyServer.pyserver
> to
> > > automatically reload upon changes.
> >
> > BTW, that something outside of the document tree, possibly in sys.path,
> > is dealt with by Python __import__ doesn't mean you can't have module
> > reloading on stuff outside of the document tree. The idea is that if
> it is
> > part of the web application and needs to be reloadable, that it doesn't
> > really belong in standard Python directories anyway. People only install
> > it there at present because it is convenient.
> >
> > The better way of dealing with this with the new module importer is to
> > put your web application modules elsewhere, ie., not on sys.path. You
> then
> > specify an absolute path to the actual .py file in the handler directive.
> >
> > <Directory />
> >      SetHandler mod_python
> >      PythonHandler /path/to/web/application/PyServer/pserver.py
> >      ...
> >
> > Most cases I have seen is that people use packages purely to create a
> > namespace to group the modules. With the new module importer that
> > doesn't really need to be done anymore. That is because you can
> > directly reference an arbitrary module by its path. When you use the
> > "import" statement in files in that directory, one of the places it will
> > automatically look, without that directory needing to be in sys.path,
> > is the same directory the file is in. This achieves the same result as
> > what people are using packages for now but you can still have module
> > reloading work.
> >
> > If you need a working example of this let me know.
> >
> > Graham
> >
> >