[mod_python] Subinterpreters and index.py

Jorey Bump wrote:
> Nicolas Lehuen wrote:
> > Jorey Bump wrote:
> >
> >>Modules are namespaces, regardless of their physical 
> location. This is 
> >>very important, even within subinterpreters.
> > 
> > I do understand this, and that's why I think that the 
> module defined 
> > in /app/index.py should not be overwritten by the one defined in 
> > /app/subdir/index.py. Maybe the namespace of the module defined by 
> > /app/subdir/index.py should be created within the namespace of the 
> > module defined by /app/index.py, so that the latter would inherit 
> > names from the former ? It seems that the publisher load 
> each .py file 
> > as modules based on the filename only, hence the name 
> collisions. If 
> > directories or URIs could be taken into account, this 
> collision problem would disappear.
> 
> Modules of the same name cause collisions. It's that simple. 
> To avoid collisions, create packages and name them carefully. 
> It's like
> indentation: It imposes a discipline that has enormous 
> benefits that aren't immediately apparent.

So why doesn't mod_python.publisher automatically creates packages based on
directories ? Mod_python.publisher imposes its own limitations. It's not the
way Python is supposed to behave. You call it a feature of
mod_python.publisher, all right. But nothing prevents people from having
modules defined in files with the same name in Python, without name
collisions, provided they are defined within different packages.

Before switching to Python 18 months ago, I've been coding in Java for at
least 5 years (well, since Java 1.0 beta). So I quite get the idea of
packages, namespaces and so on. I also do have a whole bunch of Python code
organised in packages that I reuse in different projects. And I love coding
in Python ! But this mod_python behaviour has nothing to do with Python,
it's a limitation. Anyway, mod_python Servlets seems so solve this issue ; I
was thinking of a similar handler which would load .py files the way I want,
but Daniel already wrote it, so here I go.
 
> The file hierarchy is important to the mechanics of creating 
> your environment (setting the PYTHONPATH, creating packages, 
> etc.), but plays no part in the logic of your application. 
> Just as you can't give two files in the same directory the 
> same name, you can't give two modules in the top level of 
> your application the same name. If you want to reuse a name, 
> you have to distinguish it by placing it within a package. 
> There's no need to force mod_python to do this, when you can 
> indicate this yourself by placing __init__.py in the 
> directory you want to package. 
> Automagic packaging could break a lot more than it fixes 
> (you'd have to memorize every module or package in your path 
> before creating directories).

Yeah, Jorey, I do understand that, it's trivial stuff ; my problem is that
mod_python.publisher seems to load all published modules in the top level
namespace. Sorry, I don't want to be too critical since mod_python is really
a nice achievement (very useful, especially when combined with the threaded
Apache2 implementation so that you can share data between threads), but you
don't have to lecture me on what is a namespace when the namespace issues
reside in mod_python.publisher.

This could be fixed in mod_python.publisher if the module_name was built
using the path part, not only the file part (line 67 of publisher.py) and by
loading the module by directly specifying the file to load and not adding
the path of the file to the call to import_module (line 98).
apache.import_module should then be patched to also support the loading of a
specific file, not only a module name which is then passed to
imp.find_module (line 454 of apache.py).
 
But like I've just wrote, we can also keep the best parts of mod_python, put
away mod_python.publisher and use mod_python Servlets instead.

[snip]

> > How can I benefit from the behaviour you describe while 
> also declaring 
> > the libraries I need on the Python path ?
> 
> This depends on your environment and your privileges. First, 
> try the test I outlined to verify your path under 
> Apache/mod_python (remove your PythonPath directive for the test).
> 
> One solution is to create packages in your existing path. 
> Anything you put in your installation's site-packages 
> directory will be available to everyone on the system. In 
> fact, that's why it was created.

That's what I do right now, but I want to host many different applications
on my server (I have it configured in mass virtual hosting mode for that),
and I don't think it's a good practice to have the code for all my
applications available to every other application. That's why I wanted to do
something like this :

$WWW/pythonlib/                  --> code explicitely shared by all
applications
$WWW/hosts/host1/                --> virtual host 1        
$WWW/hosts/host1/pythonlib/      --> code shared by all apps and modules of
host1
$WWW/hosts/host1/index.py        --> redirects to the default application of
host1
$WWW/hosts/host1/app1/           --> first application on host1
$WWW/hosts/host1/app1/pythonlib/ --> code shared by all modules of
host1/app1
$WWW/hosts/host1/app1/index.py   --> default module of host1/app1
...

In fact by reading this, I realise that by defining .htaccess files with
specific PythonInterpreter and PythonPath directives, I can do precisely
that plus get an healthy insulation of interpreters across applications (or
at least across hosts). I'll try this.

[snip]

Regards,

Nicolas Lehuen