|
Tim Valenta
tonightslastsong at gmail.com
Tue Jan 13 01:10:30 EST 2009
Thank you for the links-- they prove useful.
Capitalization shouldn't be an issue here... When I began learning
Unix a good while back, I quickly cast off relying on the OS to
'figure it out'. I've been keeping everything case sensitive.
Backing up to something foundational: I seem to be able to use the
import_module() method without problems now, but practically any use
of standard python 'import'/'from' keywords fails. The only success I
get is when the desired module is in the same directory as the one
trying to use the 'import' keyword. Does this stem from the fact that
import_module isn't look for packages, but rather specific files? Is
it for that reason that I can't use
from subDir import C
or, the equally broken,
import subDir.C
?
Jiggering around with adding more entries in importer.path to my
subdirectories doesn't seem to have any effect. If the 'import'
keyword really is bound to it's working directory only, I will likely
in fact have to use import_module() is several places, rather than
just at the head of the request (./A.py)
Despite my general lack of success, I appreciate the comments you've
made along the way.
Tim
On Mon, Jan 12, 2009 at 10:40 PM, Graham Dumpleton
<graham.dumpleton at gmail.com> wrote:
> 2009/1/13 Tim Valenta <tonightslastsong at gmail.com>:
>>> You just need to ensure you set mod_python's own module search path,
>>> ie. mod_python.importer.path
>>
>> And yet, as I've now removed my www directory from my PythonPath, I
>> actually get limited functionality out of the mod_python imports.
>> I've got my apache conf file globally applying the option:
>>
>> PythonOption mod_python.importer.path "['~']"
>> Also have tried using an absolute path, which is producing the same
>> functionality:
>> PythonOption mod_python.importer.path "['V:/www/']"
>>
>> My main module ('A') is requested by Apache in my browser, which uses
>> import_module to find and import file B. File B is in the same
>> directory as A, and they only way it was found was through my apache
>> conf noted above. This part works just fine, but two cases develop
>> from here:
>>
>> 1) If I use module B to import a third module 'C', I can't seem to
>> make it find module C if it's in a subdirectory. For instance, doing
>> either 'import subDirectory.moduleC' or doing
>> 'apache.import_module("subDirectory/moduleC")' results in a failure to
>> find the module. Note that in this case of subdirectories, I'm
>> intentionally avoiding absolute paths to reach my modules. I only
>> want to get absolute in my Apache conf. On the flip side, if module C
>> is in the same local directory as A and B, then mod_python finds
>> module C just fine.
>
> Try:
>
> apache.import_module("./subDirectory/moduleC.py")
>
>> 2 ) If I place C in the same directory as A and B, so that mod_python
>> finds it, I tried using the natural Python 'import' keyword in B to
>> import C. According to what I understand, this should be fine for
>> auto-reloading, since A imports B via import_module, and so then B can
>> use either importing strategy to get C in the picture because
>> mod_python uses import_module behind the scenes anyway. In this
>> setup, my caching issue remains, and changes to C are definitely not
>> reflected in apache until a server restart. Again note that it
>> doesn't matter if I make B use the 'import' keyword or the
>> apache.import_module() method. Both result in failure to reload.
>>
>> I can't seem to diagnose what's even wrong, since the only reason
>> module A, B, or C is being found in the first place is because of my
>> importer.path variable is set in httpd.conf. I've been using
>> practically empty files to test this. Each file contains no more than
>> 5 lines to get the modules imported and then to write out a
>> confirmation to the screen that it worked.
>>
>> Wisdom to offer?
>
> Yes, from memory the module importer isn't properly case insensitive
> on Windows or MacOS X where file systems is case insensitive. Thus,
> make sure your imports match exactly the case of the directories/files
> in the file system.
>
> Also, use:
>
> from mod_python import apache
> apache.log_error('__name__ = %s' % repr(__name__))
> apache.log_error('__file__ = %s' % repr(__file__))
>
> in modules so that information about imported files is logged as
> imported. This will tell you which files are being imported.
>
> With PythonDebug On you should also see messages in Apache error logs
> describing when they are being imported, initially and when changed.
>
> You can also write some code which extracts out of the mod_python
> importer cache the details of everything that has been loaded and the
> relationships. The function to call is:
>
> from mod_python import apache
> data = apache.request_modules_graph()
>
> write that to a file and run:
>
> http://www.graphviz.org/
> http://www.graphviz.org/Download_windows.php
>
> on it.
>
> Graham
>
>>> Now you do realise you don't need to convert 'import' to
>>> import_module() everywhere?
>>>
>>> The 'import' directive when used inside a file which was already
>>> managed by mod_python calls import_module() internally anyway.
>>>
>>> You just need to ensure you set mod_python's own module search path,
>>> ie. mod_python.importer.path
>>>
>>> Graham
>>>
>>> 2009/1/12 Tim Valenta <tonightslastsong at gmail.com>:
>>> > Okay, I think I'm on the same page as you now-- I've been playing
>>> > around with it and it seems that when I leave my PythonPath to find my
>>> > import modules, I definitely get no auto reloading goodness, while
>>> > when I specify the root path to find them as files, rather than as
>>> > modules (as explained in that documentation you linked to), things
>>> > seem to work as expected.
>>> >
>>> > I wasn't getting any warnings in my apache log, and after putting
>>> > together the details, I shouldn't be expecting any of that behavior
>>> > with two of the same module in memory. My issue seems to be simply in
>>> > avoiding letting my PythonPath find my code.
>>> >
>>> > Thank you much,
>>> > Tim
>>> >
>>> > On Sun, Jan 11, 2009 at 9:39 PM, Graham Dumpleton
>>> > <graham.dumpleton at gmail.com> wrote:
>>> >> 2009/1/12 Tim Valenta <tonightslastsong at gmail.com>:
>>> >>> Okay, that makes sense. My PythonPath system var definitely includes
>>> >>> my www directory.
>>> >>
>>> >> That in itself may not be enough to cause problem. You should actually
>>> >> see a warning in Apache error log complaining you have done this. That
>>> >> is, overlapped sys.path with directories that mod_python module
>>> >> importer uses.
>>> >>
>>> >> The real problem with doing this overlapping is that a module which
>>> >> wasn't one managed by mod_python, ie., standard Python module, imports
>>> >> something from the www directory, which also happens to be a handler
>>> >> module, there will actually be too copies in memory and so can give
>>> >> unexpected results.
>>> >>
>>> >> Graham
>>> >>
>>> >>> I had been using a forced path in my apache conf
>>> >>> file to point to '~', for simple reference for the time being.
>>> >>>
>>> >>> I think my problem is simply that the modules I'm trying to import
>>> >>> aren't 'candidates' for reloading. Other than that, all other modules
>>> >>> should already have been chained together with these
>>> >>> apache.load_module calls. My test case was pretty simple-- just two
>>> >>> scripts, the first loading the second.
>>> >>>
>>> >>> Tim
>>> >>>
>>> >>> On Sun, Jan 11, 2009 at 8:25 PM, Graham Dumpleton
>>> >>> <graham.dumpleton at gmail.com> wrote:
>>> >>>> 2009/1/12 Tim Valenta <tonightslastsong at gmail.com>:
>>> >>>>> Apologies for more trouble on the caching issue-- I've been adapting
>>> >>>>> some of my scripts over to this apache.load_module function, but I
>>> >>>>> don't think I'm seeing resolution to the issue. If I've read the
>>> >>>>> documentation properly, it seems that the PythonOption for
>>> >>>>> auto-reloading is set to On by default, so to be clear, I haven't put
>>> >>>>> a line for that in my httpd.conf file. Are there any other tricks, or
>>> >>>>> have I simply misread the documentation? I still have to restart the
>>> >>>>> Apache service in order to push the changes.
>>> >>>>
>>> >>>> Reloading does not work for any Python modules/packages installed in
>>> >>>> standard Python module search path. Thus, if you have set PythonPath
>>> >>>> to some directory where you store your modules, or if they are in
>>> >>>> system site-packages directory, they are not candidates for reloading.
>>> >>>>
>>> >>>> Where are the modules you are modifying and expecting to be reloading
>>> >>>> located? Is that location on sys.path, or have you explicitly setup
>>> >>>> mod_python module importer path to tell it where they are?
>>> >>>>
>>> >>>> From within running application, print out __name__ from the module
>>> >>>> you have loaded and are expecting to be reloadable, What is it? Does
>>> >>>> it look like a normal module name or something magic?
>>> >>>>
>>> >>>> How was that module imported? There needs to be a chain from
>>> >>>> mod_python handler for imports right down through to all modules you
>>> >>>> want reloadable. If there aren't, for example, you are using
>>> >>>> import_module() from a normal Python module which isn't a candidate
>>> >>>> for reloading, you will not see the depth checking for reloading
>>> >>>> occurring as you might expect.
>>> >>>>
>>> >>>> Graham
>>> >>>>
>>> >>>>> Tim
>>> >>>>>
>>> >>>>> On Sun, Jan 11, 2009 at 2:32 PM, Tim Valenta <tonightslastsong at gmail.com> wrote:
>>> >>>>>> Despite the jab at Windows, the site will be deployed on a Unix
>>> >>>>>> machine. I just prefer Windows for ease of use, especially since my
>>> >>>>>> wife has zero experience on a unix terminal or in the dearth of truly
>>> >>>>>> professional software found on the Unix/Linux platform. So yes, in
>>> >>>>>> the end it will be on a "real" operating system, But for now I'm
>>> >>>>>> willingly sticking to a more user-friendly environment for the simple
>>> >>>>>> folk.
>>> >>>>>>
>>> >>>>>> Given the characteristics of PHP that allow it to circumvent this
>>> >>>>>> caching stuff, I still think a nice little friendly red-flag reminder
>>> >>>>>> on the main mod_python tutorial should point out this limitation of an
>>> >>>>>> Apache module like this, for those of us like me, who would have gone
>>> >>>>>> many months more before having every accidentally found this manual
>>> >>>>>> import method.
>>> >>>>>>
>>> >>>>>> On Sun, Jan 11, 2009 at 2:23 PM, Graham Dumpleton
>>> >>>>>> <graham.dumpleton at gmail.com> wrote:
>>> >>>>>>>
>>> >>>>>>> 2009/1/12 Tim Valenta <tonightslastsong at gmail.com>:
>>> >>>>>>> >> Still not short enough, so much so I stopped part way through.
>>> >>>>>>> >
>>> >>>>>>> > And for that I apologize. I do appreciate the straight answer given,
>>> >>>>>>> > though I would appreciate much more the addition of that fact in some
>>> >>>>>>> > point-blank documentation that was easy to find. I have scoured the
>>> >>>>>>> > mod_python archives by search engine and came out utterly empty
>>> >>>>>>> > handed. I knew what the problem was, but I could find nothing to
>>> >>>>>>> > remedy the issue. I haven't encountered anything quite like this in
>>> >>>>>>> > my adventures with PHP, perl, or other similar scripting languages
>>> >>>>>>> > piloted by Apache. It raises a valid concern to which I ideally
>>> >>>>>>> > should have found answers very quickly.
>>> >>>>>>> >
>>> >>>>>>> > I hope this cures the subsequent issue given in the latter half of my message.
>>> >>>>>>>
>>> >>>>>>> PHP is specifically designed for web applications and it purposely
>>> >>>>>>> throws away all code at the end of each request and thus is reloading
>>> >>>>>>> it all on every request. Have a read of:
>>> >>>>>>>
>>> >>>>>>> http://blog.ianbicking.org/2008/01/12/what-php-deployment-gets-right/
>>> >>>>>>>
>>> >>>>>>> I would be very surprised if mod_perl auto reloaded code as it is
>>> >>>>>>> similar to Python in that you are applying a non web language to the
>>> >>>>>>> web. Only way perl code would be reload is if you were running them as
>>> >>>>>>> CGI scripts. If you run Python as CGI scripts you will get reloading
>>> >>>>>>> as well, but as with anything CGI, much slower.
>>> >>>>>>>
>>> >>>>>>> BTW, maybe read:
>>> >>>>>>>
>>> >>>>>>> http://blog.dscpl.com.au/2008/12/using-modwsgi-when-developing-django.html
>>> >>>>>>>
>>> >>>>>>> For WSGI hosted Python web application, this provides auto reloading
>>> >>>>>>> on code changes. You do need to use a real operating system though,
>>> >>>>>>> and not Windows.
>>> >>>>>>>
>>> >>>>>>> Graham
>>> >>>>>>>
>>> >>>>>>> > Tim
>>> >>>>>>> >
>>> >>>>>>> > On Sat, Jan 10, 2009 at 10:23 PM, Graham Dumpleton
>>> >>>>>>> > <graham.dumpleton at gmail.com> wrote:
>>> >>>>>>> >>
>>> >>>>>>> >> 2009/1/11 Tim Valenta <tonightslastsong at gmail.com>:
>>> >>>>>>> >> > Hello all-- I've been experiencing a caching issue from the
>>> >>>>>>> >> > very beginning of my use of mod_python... It's been at least 2 months now,
>>> >>>>>>> >> > and I keep running into actual issues that prevent me from coding.
>>> >>>>>>> >> > I have a knack for over-explaining, so I'll try to keep this concise yet
>>> >>>>>>> >> > descriptive.
>>> >>>>>>> >>
>>> >>>>>>> >> Still not short enough, so much so I stopped part way through.
>>> >>>>>>> >>
>>> >>>>>>> >> The simple matter of it is that mod_python does not do deep checking
>>> >>>>>>> >> of code for changes, nor does it automatically restart the process
>>> >>>>>>> >> when code is changed. Thus the need to restart Apache when you make
>>> >>>>>>> >> code changes to anything imported from sys.path is expected and normal
>>> >>>>>>> >> behaviour.
>>> >>>>>>> >>
>>> >>>>>>> >> The only time any code is automatically reloaded is the direct code
>>> >>>>>>> >> files imported by mod_python using its own special module importer.
>>> >>>>>>> >> This is documented under 'import_module()' function in:
>>> >>>>>>> >>
>>> >>>>>>> >> http://www.modpython.org/live/current/doc-html/pyapi-apmeth.html
>>> >>>>>>> >>
>>> >>>>>>> >> Graham
>>> >>>>>>> >>
>>> >>>>>>> >> > I'm developing a site on my local machine, Windows Vista, using Apache 2.2.x
>>> >>>>>>> >> > and mod_python 3.3.1 . I'm a programmer for a living, and I'm not quick to
>>> >>>>>>> >> > point the finger at the language, *but* (you knew that was coming, eh?) I
>>> >>>>>>> >> > know my code isn't to blame for the issue:
>>> >>>>>>> >> > I write some basic code for an 'index.py' file, using the
>>> >>>>>>> >> > mod_python.publisher handler. 'index.py' includes other modules which I've
>>> >>>>>>> >> > coded from that same location, etc, etc. Nothing fancy. Any changes I make
>>> >>>>>>> >> > in the 'index.py' file will be reflected immediately on my local web server.
>>> >>>>>>> >> > On the other hand, any changes I make to the modules included via import
>>> >>>>>>> >> > from within 'index.py' are completely ignored by the web server. It took me
>>> >>>>>>> >> > a while to realize that my pages didn't reflect my code. I also discovered
>>> >>>>>>> >> > that the .pyc files had nothing to do with it. I finally just restarted the
>>> >>>>>>> >> > apache web service (which in fact runs as a service on my machine), and then
>>> >>>>>>> >> > my code finally gets pushed through to the web server.
>>> >>>>>>> >> > I've been looking around practically every other day for documentation on
>>> >>>>>>> >> > how to make apache/mod_python simply cut it out and stop caching my python
>>> >>>>>>> >> > code, but I've found nothing. You can imagine the annoyance this presents,
>>> >>>>>>> >> > since I have to restart my web server every single time I make even the
>>> >>>>>>> >> > slightest change to a 'utility.py' file, etc.
>>> >>>>>>> >> > I've been coping with the problem for a while now, but then I've found far
>>> >>>>>>> >> > more annoying issues recently. To abbreviate the problem into short terms,
>>> >>>>>>> >> > I've got a main module 'MAIN' which imports another module for a class
>>> >>>>>>> >> > 'CLASS'. CLASS also has a few imports, such as 'backend' stuff for
>>> >>>>>>> >> > interfacing with various databases, etc. We'll call the 'backend' module
>>> >>>>>>> >> > "BACKEND". Given the setup, any changes I make to CLASS or BACKEND require
>>> >>>>>>> >> > an apache restart in order to take effect.
>>> >>>>>>> >> > I have a function in CLASS which calls a function from it's imported BACKEND
>>> >>>>>>> >> > module. I tried adding a parameter to the BACKEND function in question, and
>>> >>>>>>> >> > properly passed said parameter while in CLASS, yet the mod_python debugger
>>> >>>>>>> >> > spits out an error about me having passed 3 arguments, when the BACKEND
>>> >>>>>>> >> > function takes exactly 2. This is outright false, since my function in
>>> >>>>>>> >> > BACKEND looks like:
>>> >>>>>>> >> > def getUsers(self, req, terms):
>>> >>>>>>> >> >
>>> >>>>>>> >> > and I'm calling it with
>>> >>>>>>> >> > self.backend.getUsers(self.req, search)
>>> >>>>>>> >> >
>>> >>>>>>> >> > In reality, my code dictates that I'm passing 3 (including the implicit
>>> >>>>>>> >> > 'self' argument), and BACKEND's 'getUsers' does in fact take exactly 3
>>> >>>>>>> >> > arguments. Yet, the debugger is telling me that it takes only 2.
>>> >>>>>>> >> > I was trying to pass it 'req' because I wanted to investigate a little error
>>> >>>>>>> >> > in the code by printing something to the output HTML. So, my attempt is
>>> >>>>>>> >> > foiled, since somewhere something isn't being updated to what my most
>>> >>>>>>> >> > current code actually says.
>>> >>>>>>> >> > Just to test, I made the 'getUsers' function return immediately with a
>>> >>>>>>> >> > string of gibberish, like 'return "adsfasdfadsfa"'. this should make my
>>> >>>>>>> >> > other code spin wildly out of control and encounter errors, yet when I
>>> >>>>>>> >> > restart apache and test it... lo and behold, it's completely ignoring my
>>> >>>>>>> >> > goofy 'return' statement. The 'getUsers' function is still somehow
>>> >>>>>>> >> > returning valid data, as if the 'return' wasn't there at all!
>>> >>>>>>> >> > So then I tried causing actual syntax errors. The debugger caught this,
>>> >>>>>>> >> > much to my inner joy. So I tried causing a semantic error instead:
>>> >>>>>>> >> > referencing a non-existent attribute of a non-existent variable:
>>> >>>>>>> >> > madeUpVar.moo = 42
>>> >>>>>>> >> > Syntactically, nothing wrong, but at run time it should most definitely
>>> >>>>>>> >> > encounter a NameError or something equally as realistic. But I restart
>>> >>>>>>> >> > apache, and... nothing. The line is completely ignored.
>>> >>>>>>> >> > Which leads me to believe that it's not actually being 'ignored' per se, but
>>> >>>>>>> >> > rather the code being compiled is not the same as the code in play within
>>> >>>>>>> >> > the web server. When I delete my .pyc files and restart apache and visit
>>> >>>>>>> >> > the URL that triggers my python code, my .py files are in fact being
>>> >>>>>>> >> > recompiled down to their byte code .pyc files. And clearly the interpreter
>>> >>>>>>> >> > is processing my code, since it flags me on improper syntax. Yet, no matter
>>> >>>>>>> >> > what kind of syntactically-sound nonsense I put into my code, the changes
>>> >>>>>>> >> > aren't being reflected in my web server.
>>> >>>>>>> >> > These problems come and go, and I've go better explanation than over zealous
>>> >>>>>>> >> > caching. I imagine that by tomorrow sometime when I start my computer up,
>>> >>>>>>> >> > the problem will have disappeared for the time being.
>>> >>>>>>> >> > I've cursed this computer up and down as I've tried to figure out ANYTHING
>>> >>>>>>> >> > that I can do to alleviate the issue, by to no avail.
>>> >>>>>>> >> > Anybody with counsel to spare my tired brain is welcome to share...
>>> >>>>>>> >> > _______________________________________________
>>> >>>>>>> >> > Mod_python mailing list
>>> >>>>>>> >> > Mod_python at modpython.org
>>> >>>>>>> >> > http://mailman.modpython.org/mailman/listinfo/mod_python
>>> >>>>>>> >> >
>>> >>>>>>> >> >
>>> >>>>>>> >
>>> >>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>> _______________________________________________
>>> >>>>> Mod_python mailing list
>>> >>>>> Mod_python at modpython.org
>>> >>>>> http://mailman.modpython.org/mailman/listinfo/mod_python
|