[mod_python] Articles on module importing.

Graham Dumpleton wrote:

> What is missing is my ideas of how the module importing system can be
> fixed. I would rather get documented what I feel needs to be done
> before I get drawn into an any arguments on how to fix it. Thus, feel
> free to comment on the list of problems/issues, letting me know of any
> other problems you know of or where you feel I am wrong in my analysis,
> but can we please defer the discussion on how to fix it for a while
> yet. :-)
> 
> The articles can be found along with my prior article on getting
> started with mod_python at:
> 
>   http://www.dscpl.com.au/articles.html
> 
> The two new articles which have been posted are:
> 
>   http://www.dscpl.com.au/articles/modpython-002.html
>   http://www.dscpl.com.au/articles/modpython-003.html
> 
> I have posted this message to the general list rather than just the
> developer list as the information may be of general interest in as
> much as it will raise awareness that there are actually problems.
> I suggest that when we later get down to discussions on how to fix
> it that it be done on the developer list and not here however.

Graham, this is great work. It's helpful to understand the inner 
workings of mod_python when confronting the issues that arise.

Most (but not all) of the issues you raise concern writing handlers. I'd 
hate to see someone leave with the wrong impression about how to deal 
with these issues in their applications, when the advice given is more 
appropriate at the handler level. While each issue you discuss clearly 
demonstrates a problem, we should identify the higher level problems 
that fixing the module import system would address (such as, "Editing 
imported modules doesn't take effect immediately.").

On the other side of the coin, it's important to identify what sort of 
functionality should be retained. For example, despite its faults, an 
application written for mod_python.publisher can look like any other 
python application. Attempts to use alternate importing mechanisms at 
this level would look ugly and, even worse, make most of the code 
proprietary to mod_python (using apache.import_module() instead of 
import, for example). Extending the path outside of the application, 
such as via PythonPath, plays an important role here in the localised 
environment used by many developers. It can be key for the portability 
of the code, especially to non-mod_python environments.

It should be an established goal that web applications written for any 
handler included with mod_python can be written using only the standard 
library, even if that's not always the practice. It would be a shame to 
see developers come to mod_python only to find obstacles that restrict 
their productivity (and I'll grant that the current import issues fall 
into that category).

I think that defining the higher level problems and goals now will help 
to arrive at solutions that might not necessarily involve fixing current 
components.

For example, let's imagine that all of mod_python's import problems can 
be solved with some kind of magic bullet:

1. We want the PythonPath extended automatically based on the location 
of a file.

2. We want imported modules to be unloaded/reloaded automatically if 
modified or replaced in any way.

3. Imports of application code should be handled in a way that supports 
the packaging and portablility of an application.

Perhaps this can be addressed by adding code that looks for a .htPython 
directory in a file's current directory. This could be used to hold 
imported modules and packages shared by other files in the current 
directory. Because it follows an established convention, mod_python 
could import it in a way that doesn't cause namespace collisions and 
allows it to check it explicitly for changes. Conventional import 
statements can be used because it simply becomes part of an 
application's path (but not for the entire virtual host). Finally, a 
little security is offered if the server is configured to deny direct 
access to .ht* resources (typical in many apache installations). A 
compromise, to be sure, but it does allow sensitive resources to be 
packaged with an application.

I know you didn't want to go into the mechanics of fixing the module 
import mechanism, and I offer this only as an example to highlight the 
benefits of determining *what* to fix before deciding *how* to fix it. A 
scheme such as the one I outline offers backwards compatibility for 
legacy applications at the expense of adding some code that offers 
important new functionality. Hopefully, any changes to mod_python can be 
done in a similar way that doesn't threaten to fork the code.