list at joreybump.com
Thu Jul 7 23:33:32 EDT 2005
Graham Dumpleton wrote: > Jorey Bump wrote .. > >>Most (but not all) of the issues you raise concern writing handlers. I'd >>hate to see someone leave with the wrong impression about how to deal >>with these issues in their applications, when the advice given is more >>appropriate at the handler level. > > If you feel that this primarily only affects those who are writing their own > high level handlers, as opposed to those who might instead be using the > mod_python.publisher handler, then I would suggest you are wrong. No, I understand that an application inherits the problems of the handler on which it was based. > The current mod_python.publisher handler uses the import_module() > function internally to load modules and so it is just as much afflicted by many > of these problems, more so when within a published module the code uses > the import_module() explicitly and even if only the "import" statement is > used within that module. Understood. My point is that alternate import mechanisms such as import_module() shouldn't be encouraged in *published* modules or modules that are based on other handlers. I think we both agree that we want the average Python programmer to be able to start using mod_python as seamlessly as possible. Unfortunately, they need to choose a handler, and usually aren't any more interested in writing their own handler than a PHP programmer would be. I like to use Publisher, and I have worked around every limitation except one: After editing modules imported by a published module, apache must be restarted. So I agree that this is a serious problem that must be fixed, and not only in the Publisher handler. >>I think that defining the higher level problems and goals now will help >>to arrive at solutions that might not necessarily involve fixing current >>components. >> >>For example, let's imagine that all of mod_python's import problems can >>be solved with some kind of magic bullet: >> >>1. We want the PythonPath extended automatically based on the location >>of a file. > > This is not a goal or requirement, it is indicative of a solution. To provide > the desired behaviour doesn't require PythonPath be extended as there are > actually better ways of achieving the same thing and still maintain the > required transparency that you require. > > The real goal you are probably alluding to is that within a Python code > file serving as a handler you want the "import" statement to be able to > pick up a module that resides in the same directory. Yes? No. I think it's poor practice to put any kind of library files in a directory that is accessible to browsers, even with other languages. The risk is too high that someone with knowledge of your directory structure can call your support modules directly, causing unintended side effects. What I would like to see is path extension that is private to the application (not just the handler) but is otherwise indistinguishable from other parts of the search path. Naturally, it should come first, somewhat *emulating* the way the current directory is added to sys.path when running python from the command line, but without the risk of placing the modules/packages where they can be exposed to a browser. Because mod_python runs as the apache user, and not the user who owns the files, it's impossible to establish a convention that places this extended path outside the DocumentRoot. But a convention is required, hence my .htPython suggestion. > At the moment, if using mod_python.publisher this only works where the > published module is in the same directory as where the PythonHandler > directive was specified, it does not work where the published module is > in a subdirectory. Thus, should the ability to use the "import" statement > in this way also work for a published module in a subdirectory of where > the handler directive was originally defined? No. For backwards compatibility, I'd leave this behaviour as-is. We use the phrase "published module" to refer to a module that we intend a browser to access, but mod_python.publisher makes no such distinction. I don't want to see mod_python automagically package subdirectories or add them to the path because it gives newbies more rope to hang themselves. How can mod_python know if a subdirectory contains published or importable modules? It's better to check for a specially named directory and let the handler add it to the path. And it's not enough to restrict this special directory to the location where the PythonHandler was specified, because we want to be able to easily install the application elsewhere. Therefore, it has to be in the same directory as the published module. For example, http://host/app/ might contain this: .htPython/module1.py .htPython/module2.py app1.py app2.py #app1.py import module1 import module2 def index a = module1.do(something) return module2.show(a) app1 imports modules from the special directory, but app2 doesn't. No big deal. What's nice is that app.tgz could be untarred anywhere that the appropriate PythonHandler is defined, and it wouldn't be necessary to manually extend the PythonPath with an explicit file specification. The only precaution left for the developer is to name imported modules/packages appropriately to prevent collisions, but this must be done for all Python apps, anyway. Ideally, the path would be modified per "published" module, but I don't know if that's even possible. >>2. We want imported modules to be unloaded/reloaded automatically if >>modified or replaced in any way. > > What sort of imported modules though? Should this only apply to modules > imported by import_module() whether as a high level handler or explicitly > by the user or within something like mod_python.publisher? Should it also > apply to modules imported using the "import" statement for (1) above? > Should there be depth based dependency checking such that if some > grand child module imported at global scope within a high level handler > is changed, that all parents of the module up to the high level handler > should be automatically reloaded? Depth is certainly an issue, but one could assume (or at least establish the convention) that the custom modules in .htPython mostly import stable modules that are unlikely to change, such as those in the standard library. The mere possibility of specifying a conventional and portable location for custom modules goes a long way towards solving the import problem. >>3. Imports of application code should be handled in a way that supports >>the packaging and portablility of an application. > > Packages are a big problem when it comes to making automatic module > reloading work properly. I have probably put in a few days worth of > effort just into this particular issue of late and still don't have a workable > solution. I got to the point where I could finally see what might yield > a workable solution and then stopped as the code was getting overly > complicated, so much so that one had to question whether it was worth > the trouble. Sorry, by packaging I meant simply creating a tarball or zipfile for distribution, not a Python package. > The problem with packages... I currently develop most of my Publisher applications as packages that are stored in a directory prefixed to PythonPath. It works fine and I couldn't bear to lose that functionality (although I obviously still need to restart apache after editing package code). Unfortunately, I don't know enough about the import mechanism to understand the difference between reloading modules vs. packages, so I don't know if my scheme offers any solution here (or even if it's feasible).