|
Dan Eloff
dan.eloff at gmail.com
Thu Apr 20 22:29:41 EDT 2006
Neat, it works like you said it would, very nice. I'm not sure how you managed to reload parent when child changes, I'd like to take a peek at that later, good job. __purge__ is called if __clone__ raises and exception, to notify the original module that it wasn't able to be cloned, and now it's just going to be dumped without the data being copied over. I think I figured out what you used the intialized flag for in child.py, it let's you avoid initializing the parts that get overwritten by __clone__() in the event that __clone__ is called, afterall, it could be an expensive operation. I don't understand why you keep a reference in a temporary location and then lock the object and copy outside of __clone__, why not lock and copy inside __clone__? Thanks, -Dan On 4/20/06, Graham Dumpleton <grahamd at dscpl.com.au> wrote: > > Dan Eloff wrote .. > > Yay! I've got it now, and it compiled fine. I suspect something that > will > > still be a hitch is the need for that __clone__() method you talked > about. > > How do I use it, is it documented anywhere, or is there an example > > __clone__() method I can see? > > > > Thanks, > > -Dan > > Note that the __clone__() method is something that I added because of > a need I saw. I doubt that anyone else has burrowed into the code for > the importer deep enough yet to understand what it is necessarily all > about and thus the whole concept hasn't been peer reviewed as such. > Thus whether it is included at all, or in the form as it currently exists > is not final. > > That said, I have attached a very contrived set of examples of migrating > data from an old module to a new. It looks messy purely because of > the different examples and support for multithreading. If a module is > complicated and you want to be able to support multithreading and > module reloading, it will be complicated. I've seen a bit of dubious code > posted in the past which given the right conditions would blow up, > but without a proper solution have generally refrained from pointing > out the problems. > > Put the attached file in a directory configured for mod_python.publisher > with a configuration something like: > > PythonInterpreter testing > AddHandler mod_python .py > PythonHandler mod_python.publisher > PythonDebug On > PythonAutoReload On > > The "testing" interpreter is the one I have the new module importer > enabled for. Ie., in main Apache configuration, outside of all container > directives have: > > PythonOption mod_python.future.importer testing > > Check that all works by accessing page as "child.py". Once it works, > copy "child.py" to "parent.py" and add at the start the line: > > import child > > Then access "parent.py". > > Now for some fun, update modification time on "parent.py" by touching > it or resaving it. Watch how reload counter increments. Touch it again > and see how transfer counter increments. Work out from code what > each of the bits is doing and why certain things are done certain ways. > If you don't understand, ask. > > Now for some tricky stuff. Touch the "child.py" file, but still access > "parent.py". See how the parent module gets reloaded. Keep touching > "child.py" if you don't believe what is happening. Also access "child.py" > from browser as well to see how was also reloaded. > > What is happening here is that new module importer doesn't just look > at page you are accessing, but looks at modules it imports and if the > child changes, the parent gets reloaded automatically along with the > child. With this working, you avoid all the horrible problems that exist > now of having to force reloads by touching all files or restarting Apache. > > Finally, note how the reload is occurring even though child was imported > using "import" statement. This is because underneath it is magically > mapping to apache.import_module(). Change the "import" from: > > import child > > to: > > child = apache.import_module("child") > > and you will see it works the same. > > The rules are as per that incomplete document I referred you to > about the module importer. > > BTW, you cant totally avoid Apache restarts when you start using > __clone__() unless you are quite careful that it handles gracefully > not finding the data in an existing module that it is looking to copy > across. Anyway, experiment and you will see what I mean. > > You might also see if you can work out what __purge__() function > does by looking at source code. :-) > > Hope this provides you with hours of fun. > > Graham > > > On 4/20/06, Graham Dumpleton <grahamd at dscpl.com.au> wrote: > > > > > > Quoting from some of Jim's notes: > > > To access the Subversion repositories anonymously, you will need a > > > Subversion client. > > > > > > Choose the branch you would like to work on and check it out. For > > > example, to get the > > > > > > current development branch (trunk), use: > > > > > > svn co http://svn.apache.org/repos/asf/httpd/mod_python/trunk > > > mod_python > > > > > > Graham > > > > > > On 21/04/2006, at 4:17 AM, Dan Eloff wrote: > > > > > > > Excellent, now I just have to figure out how to get it from the > > > > svn. There's no instructions (that I found) on the mod_python site, > > > > probably because it's something everybody but me knows.I downloaded > > > > TortoiseSVN because that seems to be what you need on windows, and > > > > it when I click on checkout it wants to know the url of the > > > > repository. > > > > > > > > http://svn.apache.org and http://svn.apache.org/httpd/mod_python/ > > > > do not work. > > > > > > > > What's the url (and am I even going about this the right way?) > > > > > > > > Thanks, > > > > -Dan > > > > > > > > On 4/19/06, Graham Dumpleton <grahamd at dscpl.com.au> wrote: Dan > > > > Eloff wrote .. > > > > > It sounds interesting, it might just finally solve all my > > > > importing woes. > > > > > How do I enable this UMI? > > > > > > > > > > Thanks, > > > > > -Dan > > > > > > > > See: > > > > > > > > > http://issues.apache.org/jira/browse/MODPYTHON-143#action_12370976 > > > > > > > > Graham > > > > > > > > > On 4/19/06, Graham Dumpleton <grahamd at dscpl.com.au> wrote: > > > > > > > > > > > > Dan Eloff wrote .. > > > > > > > I am interested in how this unified module importer works and > > > > how it > > > > > > avoids > > > > > > > these problems. > > > > > > > > > > > > > > I'd really appreciate a brief description of it's > > > > architecture, not > > > > > that > > > > > > > you > > > > > > > usually need all that much encouraging ;) > > > > > > > > > > > > > > Thanks, > > > > > > > -Dan > > > > > > > > > > > > Bit busy today to elaborate too much, but you can start by > > > > reading my > > > > > > far from complete document on it: > > > > > > > > > > > > http://www.dscpl.com.au/articles/modpython-007.html > > > > > > > > > > > > Graham > > > > > > > > > > > > > On 4/19/06, Graham Dumpleton <grahamd at dscpl.com.au> wrote: > > > > > > > > > > > > > > > > > > > > > > > > On 20/04/2006, at 8:06 AM, Dan Eloff wrote: > > > > > > > > > > > > > > > > > I went digging through the mod_python files and > > > > discovered how > > > > > the > > > > > > > new > > > > > > > > > publisher loader works (using ModuleCache) and I borrowed > > > > the idea > > > > > > > to > > > > > > > > > import modules for my handler. > > > > > > > > > > > > > > > > Note that mod_python 3.3 will like be discarding that > > > > module loader > > > > > > > > and the original one and replacing it with a grand unified > > > > module > > > > > > > > reloader. :-) > > > > > > > > > > > > > > > > > The purpose of ModuleCache was to avoid sys.modules right? > > > > > > > > > > > > > > > > Yes. > > > > > > > > > > > > > > > > > I'm having some trouble with this, and I eventually > > > > tracked it > > > > > down > > > > > > > to > > > > > > > > > multiple versions of a dependancy module being loaded. > > > > > > > > > > > > > > > > Known issue. See: > > > > > > > > > > > > > > > > > > > > > > > > http://www.dscpl.com.au/articles/ > > > > modpython-003.html#multiple-module- > > > > > > > > instances > > > > > > > > > > > > > > > > > My handler loads one version, with empty module > > > > variables, while > > > > > the > > > > > > > > > actual modules for the pages load different version(s) > > > > and populate > > > > > > > > > them with data. The changes are never reflected in the > > > > version > > > > > kept > > > > > > > by > > > > > > > > > my handler (because they are not the same) > > > > > > > > > > > > > > > > > > This all makes sense, and is probably the intended > > > > effect, but > > > > > what > > > > > > > do > > > > > > > > > you do for modules like Session in mod_python, where > > > > MemorySession > > > > > > > > > uses a static class member to store sessions? Wouldn't > > > > the page > > > > > > > > > modules each get their own version and thusly prevent > > > > sessions > > > > > from > > > > > > > > > working? > > > > > > > > > > > > > > > > Also see: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://www.dscpl.com.au/articles/modpython-003.html#transfer-of- > > > > module- > > > > > > > > data > > > > > > > > > > > > > > > > > I need to be able to reload modified modules, but also to > > > > make > > > > > use > > > > > > > of > > > > > > > > > shared modules. These seem to be conflicting needs, > > > > however. Does > > > > > > > > > anybody have any inisght into this situation that they > > > > can share? > > > > > > > > > > > > > > > > Only that mod_python 3.3 will hopefully fix a lot of this > > > > stuff. > > > > > If > > > > > > you > > > > > > > > are prepared to give 3.3 source code out of subversion > > > > repository > > > > > a go > > > > > > > > to see if things work as you expect, let me know and will > > > > tell you > > > > > the > > > > > > > > magic to enable the unified module importer in 3.3 as at > > > > moment it > > > > > is > > > > > > > > still off by default. > > > > > > > > > > > > > > > > Note though that even in 3.3, the "transfer of module data" > > > > issue > > > > > > still > > > > > > > > has to be explicitly dealt with in some way. Although not > > > > confirmed > > > > > > how > > > > > > > > this will be done, the new module importer as it stands now > > > > allows > > > > > a > > > > > > > > module to define a __clone__() method which will be called > > > > and which > > > > > > > > can be used to transfer data from an old module to a new > > > > where data > > > > > > > > needs > > > > > > > > to be preserved across reloads. > > > > > > > > > > > > > > > > As I said, if interested I'll go into it more later. > > > > > > > > > > > > > > > > Graham > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mm_cfg_has_not_been_edited_to_set_host_domains/pipermail/mod_python/attachments/20060420/dbc7dbe0/attachment-0001.html
|