[mod_python] focus on shared hosting?

One of the main advantages of mod_python in my eyes is that there is
"full control" over what is actually done after a request is received. I
know I use Apache a some sort of Intelligent socket interface to port
80, but as you look at my previous post today you'll understand that is
exactly what I need.

I see a lot of people be perfectly happy with PSP, (I have briefly
looked into it and never ever made one PSP page). For those people basic
mod_python support would be excellent.

I my eyes it is important to find out why people use stuff, why do they
need mod_python, I find that a lot of people buy a lorry of tools to
drill a hole in the wall for hanging up a painting.

The original reason for using tools in my eyes is that we do stuff using
http where http was never designed for (and HTML), next is that all the
browsers act differently, and most important off all we want to show how
good we are. Also everybody is writing stuff which never ever becomes
finished, Microsoft is a big forerunner of introducing new stuff nobody
needs but is purely a matter of "look what we achieved to get your
money" 90% of the windows users are (ms)office users (Word, Excell,
outlook and IE) and why the hell do they need to upgrade to Vista when
W2k (win9X/ME was not an OS) did that job perfectly years ago. I have a
PIII-800 (Celeron) with 384Mb laptop in my living room for surfing the
net, reading email end do some private office work. Using W2K, office
2000, and Firefox and an Xterminal emulator (IE still installed). I have
thought many times about buying a new one, but cannot see why.

I am a programmer for 30 years now and way back in time we had computers
with 1kb of memory, in that time people HAD to use the tools and methods
which fitted into memory, nowadays everybody is installing everything
without thinking, using parts of this and parts of that.

So when brainstorming about mod_whatever it would be nice to see what is
needed by people to get their stuff done, in combination with what the
effects are when they want to host stuff. So called design criteria.
Isn't it the time to start make things easier instead of making things
more complex by supporting everything possible? mod_wsgi could be the
solution for a lot of users to get things hosted, but I still see
trouble when it comes to the site-packages in the python configuration.
My application requires mod_python and MySQLdb in addition to the
standard python installation and everything to backup is in the MySQL
database. If I can get MySLQdb out of the site-package directory (I have
not looked into that yet)  and into my app dir, just mod_python and
access to .htaccess would be sufficient.

One of a few arguments I have heard about mod_python in a shared hosting
environment is that it leaks memory and that it therefore cannot be
supported, at the other hand Admins have to be afraid of customers
writing code on a shared server anyway especially when it is open source
based since everybody tends to use the latest beta version for
functionality, some open source projects are in beta all the time...and
no guaranties 

I am heavily testing mod_python 3.3.0b to go live with it in January,
something I should not do, I know, but this is a non commercial project
on my own server in rented rack space running this application only, So
I accept the risk in this case.

Martijn





-----Oorspronkelijk bericht-----
Van: mod_python-bounces at modpython.org
[mailto:mod_python-bounces at modpython.org] Namens Graham Dumpleton
Verzonden: Friday, December 22, 2006 1:12 PM
Aan: Jeff Hinrichs - DM&T
CC: mod_python at modpython.org
Onderwerp: Re: [mod_python] focus on shared hosting?

On 22/12/2006, at 11:47 AM, Jeff Hinrichs - DM&T wrote:

> John Udell blogged today about django and mod_python.  However the
> question he poses is pertinent to mod_python and projects that depend
> on mp like TG and Pylons, et al.
>
> ...
>
> I know that there are a few hosts that specialize in python based apps
> but what can be done to make mp a must have like php and ruby?  What
> can be done technically to make mp a no-brainer decision for most web
> hosting companies.  Are there really security or resource utilization
> issues that can't be over come?  Maybe the next big mp focus should be
> on what can be done to make administering/securing a shared mp install
> so that it is easier for shared hosts to implement it.

Parts of what you are talking about are things I have been thinking
about quite a lot lately, which is, possible future directions for the
concept of using Python in conjunction with Apache. Note though that I
don't mention mod_python when I say this. :-)

The first thing one has to realise about people who want to use Python
for web programming is that the majority don't actually care what
mod_python is. Where they do want to use Apache, they see mod_python
merely as a convenient hopping off point. As such, you have frameworks
like Django, TurboGears, CherryPy and applications like Trac (latest)
and MoinMoin, which don't actually use any of the handlers supplied with
mod_python, nor its capabilities to do automatic module reloading,
forms, session handling etc.

All these frameworks and applications want to do is specify a handler
to be called for the response phase, set up the Python module path and
possibly pass in a few options. When their handler is called, they then
promptly hide all the mod_python bits under a layer of code and the
user doesn't see them.

Further, where in the past these frameworks and applications built
separate adaptors for the various different web servers, they now all
tend to support WSGI. What one thus needs is not mod_python, but
mod_wsgi. The code needed to implement such a module would actually
be quite little in comparison to mod_python. In fact, none of the pure
Python code components of mod_python would even be required.

So what would a mod_wsgi need to contain. The first thing it needs to
be able to do is handle initialisation of Python and creation of
interpreter
instances. This code would actually be much simpler than in mod_python
as mod_python has to do extra stuff like initialising global mutexes for
session management, trigger imports of special internal modules when
each interpreter instances is created, as well as importing special user
modules as well. In mod_wsgi it wouldn't need to do any of that.

The next big difference with mod_wsgi in comparison to mod_python
would be that except to trigger its own initialisation, the only handler
phase it needs to hook and process is the response phase. Because it is
not hooking all phases, it would be much more efficient than mod_python
as it wouldn't be doing all that extra work for each request of checking
for mod_python handlers for each phase.

Next area where mod_python does more work than it needs to is with all
its configuration and options directives. Ie., PythonPath,
PythonAutoReload,
PythonDebug, PythonOption. Apache already has a SetEnv directive which
can be used to set key value pairs with the values being placed into the
req.subprocess_env table object. Since other parts of Apache already
deal
with doing that, mod_wsgi could just use SetEnv as the means of getting
any configuration information, with the req.subprocess_env table also
becoming the basis of the WSGI dictionary which gets passed through
to any WSGI handler. Even the definition of what the WSGI handler module
and function is, could be specified in this way and as such mod_wsgi
would
not need to define any directives at all, thus eliminating the need for
code
to handle them.

To then setup mod_wsgi to be used, in the Apache configuration you would
have:

  <Location /some/path>
  SetHandler mod_wsgi
  SetEnv mod_wsgi.application module::application
  SetEnv mod_wsgi.interpreter myapplication
  SetEnv mod_wsgi.directory /directory/containing/module
  </Location>

Now in mod_python when a handler needs to be executed, a call is first
made into a dispatch function implemented in Python and it is that which
sets up all the environment and then calls the handler function. With
WSGI though, because the API for calling a WSGI application is so simple
it would be better to implement the dispatch in C code. Thus the C code
of mod_wsgi would import the application module and make the calls into
the application as appropriate. By doing this you totally eliminate the
need for any separate Python code modules needing to be installed and
thus get rid of one of the setup problems with mod_python of not being
able to find those extra Python modules. This should make it much easier
to install.

One issue with web hosting environments especially is being able to
separate different users web applications. Although one can't have each
application run as a separate user, one can at least separate them into
distinct interpreters. To specify the interpreter one would use SetEnv
to set mod_wsgi.interpreter. Where you have a lot of applications
though, you might not want to have to manually set these. Although
mod_rewrite is itself a bit heavy weight, one of the things it can do is
set the same variables as SetEnv sets based on stuff which appears in
the URL. Thus one could with a mod_rewrite rule automatically specify
the interpreter name based on the request URL. This is something that
mod_python can't even do because it uses a directive for setting the
name of the interpreter.

One final issue with interpreters is the initialisation of Python and
the creation of the interpreter instances. In this area, mod_wsgi could
run in one of two modes. The first would be where mod_wsgi is the only
Python module installed, ie. mod_python is not installed. In this
situation it could perform the initialisation of Python all itself and
also handle creation of interpreter instances. The second mode would be
where it detects that mod_python is already loaded. In this situation it
would simply use the hooks added in mod_python 3.3 for using
mod_python's machinery for accessing Python interpreter instances. By
being able to operate in these two modes, the module could either exist
by itself, but if needed, also co-operate with mod_python. Thus, just
because you run mod_wsgi doesn't mean that you couldn't also run
mod_wsgi at the same time.

If you have followed what I am talking about, and understand mod_python
internals and Apache a bit, you will see that the code for mod_wsgi
would actually be quite simple and because it doesn't have to wrap any
Apache data structures in Python equivalents, it should be quite easy to
create a version which is capable of being compiled on Apache 1.3, 2.0
or 2.2.

As I mention above, most people don't need the full mod_python
and thus this would allow all these higher level applications to still
be able to be run (under WSGI) even though they are using an older
version of Apache. This would also make it much simpler for web hosting
services as well, as they can say that they support anything which is
WSGI compliant.

Now, the idea of mod_wsgi is only one part of what I have been thinking
about for future directions of Python with Apache. It is late now though
so
I'll go into my other ideas in the coming days, that is if I don't now
decide
to go and finish my mod_wsgi which I already have the basis for in place
and thus get diverted. :-)

Comments on this part of my plans for global domination most welcome.

Graham

_______________________________________________
Mod_python mailing list
Mod_python at modpython.org
http://mailman.modpython.org/mailman/listinfo/mod_python