[mod_python] psp_site example doesn't work

On Mon, 24 Jan 2005 09:12:24 -0500, Jorey Bump <list at joreybump.com> wrote:
> Nicolas Lehuen wrote:
> > On Sun, 23 Jan 2005 18:38:43 -0500, Graham Dumpleton
> > <grahamd at dscpl.com.au> wrote:
> 
> >>Package such as mpservlets and Vampire implement their own module
> >>importing mechanisms in order to avoid this problem. Ie., with those
> >>packages it is safe to have Python code files which are named the same
> >>appearing in different directories managed by mod_python running
> >>under the same interpreter.
> >
> > We had a discussion a few months ago about this issue, and I still
> > think it's a bug. Like Daniel Popovitch with mpservlets and you with
> > Vampire, I had to implement my own import mechanism to work around the
> > bug. I think it's time to fix it... I'll add a bug report and see what
> > we can do about it.
> 
> I guess I'm on record as considering this to be normal Python behaviour,
> and not a bug. Talk of modifying Publisher's current import mechanism
> threatens the significant investment I have in the applications I've
> already developed. I'd rather see Publisher left alone, as other
> frameworks are maturing that address this issue.
> 
> We should be clear about what we are talking about:
> 
> Issue:
> 
> An interpreter imports a module's name into the namespace independent of
> its location in the filesystem.
> 
> Pros:
> 
> - This is consistent with normal Python behaviour, where an interpreter
> loads modules from the current directory or PYTHONPATH. [1]
> 
> - There is no ambiguity about imports, and no need to rewrite code when
> moving the module to another location.
> 
> - The Python language documentation serves as a good resource, as native
> behaviours are not contradicted. [2]
> 
> - General purpose code needs *very* little modification to run under
> mod_python.publisher.
> 
> - The default Python behaviour of caching imported modules can be used
> to great leverage, and plays a significant role in the performance boost
> offered by mod_python. [3]
> 
> - With careful planning, per-interpreter package repositories can be
> created.
> 
> Cons:
> 
> - This consistently trips up newbies, who are expecting PHP-like
> behaviour from their web applications.
> 
> - Within an interpreter (a VirtualHost, for example), such care must be
> taken to create unique module names that porting an application or
> working in a team environment is significantly more fragile than in
> other web technologies, where it usually suffices to throw the code in
> its own directory.
> 
> - Although a lot of issues can be addressed in .htaccess files, so much
> configuration occurs at the apache level that it is difficult to use
> mod_python effectively without apache admin privileges. [4]
> 
> - The Multi-Processing Modules (MPMs) in apache 2 may have a unique
> effect on an application. After throwing in the different platforms,
> Python versions, and developer styles, it can be very hard to reach a
> consensus on whether an issue even exists.
> 
> Possible solution:
> 
> Automatically package module based on the file path.
> 
> Pros:
> 
> - Hopefully, this would provide better separation of modules in namespace.
> 
> Cons:
> 
> - Serious potential to break existing Publisher applications.
> 
> - The same module might be loaded into a separate namespace for
> different applications, eliminating the advantage of module caching. [5]
> 
> - This encourages the reuse of module names (such as index.py), which is
> considered a bad programming practice in Python. [1]
> 
> - If the code assumes and uses a package name based on its location, it
> will need to be rewritten if it is moved.
> 
> - The possibility that autopackaging will arbitrarily create a namespace
> clash with a standard module is unacceptably high, especially where
> third party packages are installed.
> 
> - It severely limits the ability to test portions of the code from the
> command line, because it adds an element of unpredictability to the
> module search path.
> 
> - Seasoned Python programmers might consider mod_python.publisher to be
> "broken" because it doesn't conform to Python standards. [6]
> 
> [1] For a clear description of Python's module search path, see:
> http://www.python.org/doc/current/tut/node8.html#SECTION008110000000000000000
> 
> [2] The documentation for mod_python is sparse because it simply embeds
> an interpreter in apache. See the Python documentation for programming
> issues: http://www.python.org/doc/current/index.html
> 
> [3] This seems to be the main complaint levied against mod_python, in
> one form or the other. I consider it one of it's greatest strengths. For
> example, it allows me to import database handling code from a module
> once for that interpreter, allowing multiple applications to share the
> connection -- a poor man's database pool, if you will.
> 
> [4] The same can be said about php.ini, but noone ever needs to restart
> apache to develop PHP applications.
> 
> [5] What is the solution to the problems caused by module caching? It
> doesn't make much sense to force mod_python to act like it's spawning a
> new interpreter for every request. Run the app as a CGI instead of using
> mod_python.
> 
> [6] The strength of Publisher is that it leverages the advantages of
> Python with very little overhead, simply adding a number of conveniences
> specific to web application development. mod_python.psp is a much better
> area to radically depart from this paradigm, as it is expected to
> provide PHP-like behaviour.

Sorry, I don't have much time, so for now I'll give a short answer : I
agree that importing modules should be consistent between the standard
Python way (the import keyword) and the apache.import_module way.

The problem is that mod_python.publisher does not follow the standard
Python way in that it imports published modules as root-level modules.
It's as if mod_python.publisher changed sys.path for each and every
published module. This is not the standard Python way, and the source
for many errors.

Note that the current package system in Python is way, way smarter
than mod_python.publisher : in a package, you can import fellow
modules by giving their local names. It's not recommended by
pychecker, but it works. The smart part is that if I have this :

/
     package1.py
/subdir
     __init__.py
     package1.py
     package2.py

Then whatever I do, /subdir/package1.py is imported as subdir.package1
and does not collide with /package1.py. mod_python.publisher is not as
smart, this is very confusing and surprising, nearly everybody
complains about this and hence, I think this should be filed as a bug.

So maybe apache.import_module should not be modified (except for
performance, but that's another issue which is already in the bugs
database), but mod_python.publisher sure has to be changed.

As far as I'm concerned, this bug was sufficient to have me implement
a publisher which works the way I want ; Daniel Popovitch and Graham
Dumpleton did the same with different designs. If each and every
developer develops his own publisher, it may be time to think about
changing the built-in one.

Regards,
Nicolas