[mod_python] psp_site example doesn't work

On Mon, 24 Jan 2005 12:19:46 -0500, Daniel Popowich
<dpopowich at comcast.net> wrote:
> 
> I agree with Nicolas that this is a "bug".  Perhaps not a bug in the
> classic sense: something not working as advertised, but a design
> issue so gnarly, that it smells like bug.
> 
> Thank you Jorey for your detailed email, but I have to disagree with
> you when you wrote:
> 
>     This is consistent with normal Python behaviour, where an
>     interpreter loads modules from the current directory or
>     PYTHONPATH. [1]
> 
> I came to mod_python after *many* years of intensive python
> programming and as a newbie to mod_python was caught by this issue for
> days.  The way mod_python imports files all at the same root level is
> very UN-pythonic.  After much hair-pulling I finally realized I needed
> to roll my own for mpservlets.
> 
> I do think it's possible to patch publisher and keep it backwards
> compatible.  In mpservlets, I do something like this:
> 
>     cache = {}
>     ...
>     fname = full path of source file to be loaded, say, req.filename
>     ...
>     # check cache
>     code = cache.get(fname)
>     if not code:
>         code = {}
>         execfile(fname, code)
>         cache[fname] = code
>     # search code dictionary for what I'm after, e.g.:
>     handler = code.get('handler')
>     if not handler:
>        raise apache.SERVER_RETURN, apache.http_not_found
>     handler(req)
> 
> You get the idea...
>
> Since publisher should "own" the modules it will import (if well
> designed, IMHO, the files exist for mod_python, not general use) and
> knows exactly what it's looking for in the modules, I see no reason
> why such a caching scheme can't work.

That's what I also use, with locks to ensure thread safety, and it
works without a hitch on a production server. I do totally agree with
you when you write that published modules should not be mixed with
general purpose modules. Published modules and 'standard' modules do
not share the same semantics :

- published modules can be reloaded as soon as they are modified, or
loaded multiple times if you use the forked MPM ; the code it contains
must be aware of this fact, which is generally not true for standard
modules.
- some objects from the published modules are accessible through HTTP.
This means that those modules are potentially subject to security
problems. In mod_python.publisher, a non-callable object is simply
transformed into a string and returned to the HTTP client. It's not a
good idea, therefore, to have a variable named
MY_VERY_SECRET_PRIVATE_KEY in the module... It's not even a good idea
to import it ! Granted, some other publishers may secure what is
accessible through HTTP and what is not (mine does), but anyway, it's
a good idea to distinguish between published modules and the others.
- I personnaly think it's a good thing for published modules not to be
compiled into a .pyc or .pyo file. Like for the previous point, my
concern is about security : .pyc and .pyo file should not be
accessible through HTTP, lest you want your code be dissassembled and
your private key exposed. A simple .htaccess directive can solve this,
but what about the zillion people who will forget about it ?
 
> There is an issue of both the python module infrastructure and
> mod_python having their own copies of the same module if your
> publisher source file is imported by another module, e.g., index.py is
> imported by foo.py.  With mpservlets I took the path of enforcing
> servlets to be in a file that cannot be imported by other modules
> (they have to be named .mps); this did two things: 1) prevents the
> double caching problem and 2) promotes the separation of interface
> code from business logic code.

I totally agree with your design decision. My application are
separated between published modules which contain all the HTML and
interface logic, and standard, imported backend modules which contain
all the application logic. Much cleaner this way.

> I also agree with Nicolas that there seems to be a message being sent
> when you consider mpservlets, Vampire, his own handler and PSP are all
> doing their own importing mechanism.  Let's face it, importing in
> vanilla python is not the most clear-cut source-inclusion scheme
> around (this is the ONLY area where I think java got it right over
> python); when you imbed it in a process like apache...ouch...
> 
> As for Nicholas' suggestion of removing publisher from the
> project...that may be a bit drastic.  That being said, I've never
> thought it an intuitive place to start for newbies.

That was a bit provocative, I agree ;).

> But if mpservlets was adopted into the main-stream distro...!  :-)

I'd say, why not ? I think it's a good thing mod_python is designed
the way it is. mod_python.apache provides a very low-level interface
to the Apache mod API. We can then add higher level interfaces on top
: mod_python.publisher, mpservlets, Vampire... We could even package
in a WSGI implementation !

However, I think mod_python should provide a default high level
implementation, a little bit more convenient than
mod_python.publisher. For starters, I think, like Graham, that the
fact that the publisher want you to return document instead of writing
them to the request object is a bit surprising for the newbie, and
hampers the transition from handlers to published objects. Maybe by
working together on the python-dev list, we could design a new
publisher for mod_python ?

Regards,
Nicolas