[mod_python] focus on shared hosting?

Graham Dumpleton wrote:
> On 22/12/2006, at 11:47 AM, Jeff Hinrichs - DM&T wrote:
> 
>> John Udell blogged today about django and mod_python.  However the
>> question he poses is pertinent to mod_python and projects that depend
>> on mp like TG and Pylons, et al.
>>
>> ...
>>
>> I know that there are a few hosts that specialize in python based apps
>> but what can be done to make mp a must have like php and ruby?  What
>> can be done technically to make mp a no-brainer decision for most web
>> hosting companies.  Are there really security or resource utilization
>> issues that can't be over come?  Maybe the next big mp focus should be
>> on what can be done to make administering/securing a shared mp install
>> so that it is easier for shared hosts to implement it.
> 
> Parts of what you are talking about are things I have been thinking
> about quite a lot lately, which is, possible future directions for the
> concept of using Python in conjunction with Apache. Note though that I
> don't mention mod_python when I say this. :-)
> 
> The first thing one has to realise about people who want to use Python
> for web programming is that the majority don't actually care what
> mod_python is. Where they do want to use Apache, they see mod_python
> merely as a convenient hopping off point. As such, you have frameworks
> like Django, TurboGears, CherryPy and applications like Trac (latest)
> and MoinMoin, which don't actually use any of the handlers supplied with
> mod_python, nor its capabilities to do automatic module reloading,
> forms, session handling etc.
> 
> All these frameworks and applications want to do is specify a handler
> to be called for the response phase, set up the Python module path and
> possibly pass in a few options. When their handler is called, they then
> promptly hide all the mod_python bits under a layer of code and the
> user doesn't see them.
> 
> Further, where in the past these frameworks and applications built
> separate adaptors for the various different web servers, they now all
> tend to support WSGI. What one thus needs is not mod_python, but
> mod_wsgi. The code needed to implement such a module would actually
> be quite little in comparison to mod_python. In fact, none of the pure
> Python code components of mod_python would even be required.
> 
> So what would a mod_wsgi need to contain. The first thing it needs to
> be able to do is handle initialisation of Python and creation of interpreter
> instances. This code would actually be much simpler than in mod_python
> as mod_python has to do extra stuff like initialising global mutexes for
> session management, trigger imports of special internal modules when
> each interpreter instances is created, as well as importing special user
> modules as well. In mod_wsgi it wouldn't need to do any of that.
> 
> The next big difference with mod_wsgi in comparison to mod_python
> would be that except to trigger its own initialisation, the only handler
> phase it needs to hook and process is the response phase. Because it is
> not hooking all phases, it would be much more efficient than mod_python
> as it wouldn't be doing all that extra work for each request of checking
> for mod_python handlers for each phase.
> 
> Next area where mod_python does more work than it needs to is with all
> its configuration and options directives. Ie., PythonPath, PythonAutoReload,
> PythonDebug, PythonOption. Apache already has a SetEnv directive which
> can be used to set key value pairs with the values being placed into the
> req.subprocess_env table object. Since other parts of Apache already deal
> with doing that, mod_wsgi could just use SetEnv as the means of getting
> any configuration information, with the req.subprocess_env table also
> becoming the basis of the WSGI dictionary which gets passed through
> to any WSGI handler. Even the definition of what the WSGI handler module
> and function is, could be specified in this way and as such mod_wsgi would
> not need to define any directives at all, thus eliminating the need for code
> to handle them.
> 
> To then setup mod_wsgi to be used, in the Apache configuration you would
> have:
> 
>   <Location /some/path>
>   SetHandler mod_wsgi
>   SetEnv mod_wsgi.application module::application
>   SetEnv mod_wsgi.interpreter myapplication
>   SetEnv mod_wsgi.directory /directory/containing/module
>   </Location>
> 
> Now in mod_python when a handler needs to be executed, a call is first
> made into a dispatch function implemented in Python and it is that which
> sets up all the environment and then calls the handler function. With
> WSGI though, because the API for calling a WSGI application is so simple
> it would be better to implement the dispatch in C code. Thus the C code
> of mod_wsgi would import the application module and make the calls into
> the application as appropriate. By doing this you totally eliminate the
> need for any separate Python code modules needing to be installed and
> thus get rid of one of the setup problems with mod_python of not being
> able to find those extra Python modules. This should make it much easier
> to install.
> 
> One issue with web hosting environments especially is being able to
> separate different users web applications. Although one can't have each
> application run as a separate user, one can at least separate them into
> distinct interpreters. To specify the interpreter one would use SetEnv
> to set mod_wsgi.interpreter. Where you have a lot of applications
> though, you might not want to have to manually set these. Although
> mod_rewrite is itself a bit heavy weight, one of the things it can do is
> set the same variables as SetEnv sets based on stuff which appears in
> the URL. Thus one could with a mod_rewrite rule automatically specify
> the interpreter name based on the request URL. This is something that
> mod_python can't even do because it uses a directive for setting the
> name of the interpreter.
> 
> One final issue with interpreters is the initialisation of Python and
> the creation of the interpreter instances. In this area, mod_wsgi could
> run in one of two modes. The first would be where mod_wsgi is the only
> Python module installed, ie. mod_python is not installed. In this
> situation it could perform the initialisation of Python all itself and
> also handle creation of interpreter instances. The second mode would be
> where it detects that mod_python is already loaded. In this situation it
> would simply use the hooks added in mod_python 3.3 for using
> mod_python's machinery for accessing Python interpreter instances. By
> being able to operate in these two modes, the module could either exist
> by itself, but if needed, also co-operate with mod_python. Thus, just
> because you run mod_wsgi doesn't mean that you couldn't also run
> mod_wsgi at the same time.
> 
> If you have followed what I am talking about, and understand mod_python
> internals and Apache a bit, you will see that the code for mod_wsgi
> would actually be quite simple and because it doesn't have to wrap any
> Apache data structures in Python equivalents, it should be quite easy to
> create a version which is capable of being compiled on Apache 1.3, 2.0
> or 2.2.
> 
> As I mention above, most people don't need the full mod_python
> and thus this would allow all these higher level applications to still
> be able to be run (under WSGI) even though they are using an older
> version of Apache. This would also make it much simpler for web hosting
> services as well, as they can say that they support anything which is
> WSGI compliant.
> 
> Now, the idea of mod_wsgi is only one part of what I have been thinking
> about for future directions of Python with Apache. It is late now though so
> I'll go into my other ideas in the coming days, that is if I don't now decide
> to go and finish my mod_wsgi which I already have the basis for in place
> and thus get diverted. :-)
> 
> Comments on this part of my plans for global domination most welcome.

Wow, what a good idea. (both mod_wsgi and world domination). Mod_wsgi 
could pretty much be the mod_python-lite that Anthony spoke of earlier. 
Could mod_python be made forward compatible with mod_wsgi? That way a 
could site start with mod_wsgi but then easily switch to mod_python if 
its additional features where required.

This is definitely something would should talk about in the new year.

Jim