[mod_python] shared memory and module import problems

Nicolas Lehuen nicolas at lehuen.com
Tue Feb 21 14:40:43 EST 2006


Kevin

Your analysis is correct. By using Fedora you are certainly using by
default the prefork MPM. This causes the problem you are observing.

Regards,
Nicolas


2006/2/21, Kevin J. Smith <hockeysk8 at gmail.com>:
> In trying to answer the question of which MPM model the server uses I think
> I may have come across my answer.  In Graham's article
> http://www.dscpl.com.au/articles/modpython-004.html (not
> sure why I did run across this article in my many Google searches for an
> answer) he has explained the behaviour of mod_python with the various MPM
> models.
>
> It is most likely that I am using a prefork model because the server is
> running out-of-the-box fedora (don't have access to the server at the
> moment.)  And therefore, if I have understood Graham's article correctly,
> with a prefork model, even though the interpreter may be named the same,
> different requests may be picked up by different apache child processes
> therefore they will not be sharing the same memory.  To quote from the
> article:
>
> "This ability to access the same global data from any request handler and
> for that data to persist for the lifetime of that instance of Apache is not
> present when either of the "prefork" or "worker" modes are used. This is
> because handlers can be executing within the context of distinct child
> processes, each with their own set of global data unique to that child
> process. Further, the global data present within a particular child process
> only exists for the lifetime of that child process and Apache could shutdown
> and kill off a child process at any time."
>
> This certainly makes sense within the prefork model.  I am not sure why it
> wouldn't work for worker threads because in my mind they should be sharing
> the same memory?  Nonetheless, I need my handler to be portable therefore I
> will have to implement my own memory cache with something like memcached
> (http://www.danga.com/memcached/) or the like (Django uses
> memcached for a similar purpose.)
>
> Somebody please correct me if I have any of this wrong.
>
> Cheers
>
>
> On 2/21/06, Nicolas Lehuen <nicolas at lehuen.com> wrote:
> >
> > Hi Kevin,
> >
> > 1) Please give us your full configuration : OS, Python version, Apache
> > version and the MPM you use. Your problem may be related to the fact
> > that you are using a forking MPM.
> >
> > 2) Are you sure your module is continually reimported ? Do you see
> > traces in the error log about that ? Your problem may be related to a
> > bug in your cache code.
> >
> > 3) If you want to re-use the importing mechanism that
> > mod_python.publisher uses, you can have a look at the ModuleCache
> > class found in mod_python/cache.py. Moreover, this package can be
> > re-used for all kind of caching issues (see the source code).
> >
> > Regards,
> > Nicolas
> >
> > 2006/2/21, Kevin J. Smith < hockeysk8 at gmail.com>:
> > > Hi,
> > >
> > > I have a page controller that all handlers import with "from
> > > myapp.gui.pagecontroller import PageController".  Each handler is a
> class
> > > derived from the PageController.
> > >
> > > In the page controller, I have attempted to set up a shared memory cache
> for
> > > all the controllers with code very similar to the following:
> > >
> > > #...
> > > import threading
> > >
> > > CACHE_LOCK = threading.Lock()
> > > cache = {}
> > >
> > > def _fillCache():
> > >     cache['one'] = '1'
> > >     cache['two'] = '2'
> > >
> > > class PageController:
> > > # ...
> > >     def __init__(self):
> > >         #....
> > >         if not len(cache.keys()):
> > >             CACHE_LOCK.acquire()
> > >             _fillCache()
> > >             CACHE_LOCK.release()
> > >         #...
> > >
> > >
> > > However, I have discovered that this module is continually reimported
> > > because it ALWAYS calls the _fillCache() function despite all the
> handlers
> > > running within the same interpreter.
> > >
> > > I have upgraded to mod_python 3.2.7 noticing that in the change log
> there is
> > > a lot about fixing module import problems but my app is still behaving
> the
> > > same way.
> > >
> > > The flow of my app is that all requests are handled by a frontController
> > > handler (this is explicitly set in the httpd config) which, after
> checking
> > > the session and authentication and what-not and based on the request
> path,
> > > forwards the request onto one of my pageController handlers via
> > > util.redirect().  If I have followed Graham's notes about module
> importing
> > > correctly, the frontController and pageController classes would be
> imported
> > > via import_module() whereas the actual pageController module is imported
> via
> > > the regular python import mechanism.
> > >
> > > I have spent some time reading about the various "uniquenesses" of
> module
> > > importing within mod_python but it hurt my head slightly and I didn't
> know
> > > if most of it reflected the state of affairs in version 3.2.7.  From
> reading
> > > Graham's page I don't believe I should have a problem with the
> > > PageController handler continually being reimported but it certainly
> appears
> > > to be doing just that.
> > >
> > > Can someone shed some light on what I may be doing wrong or a possible
> > > solution to setting up a shared cache within an interpreter?  I know
> that if
> > > I run similar code outside of mod_python my cache design works fine.
> > >
> > > Cheers
> > >
> > >
> > > _______________________________________________
> > > Mod_python mailing list
> > > Mod_python at modpython.org
> > >
> http://mailman.modpython.org/mailman/listinfo/mod_python
> > >
> > >
> > >
> >
> > _______________________________________________
> > Mod_python mailing list
> > Mod_python at modpython.org
> > http://mailman.modpython.org/mailman/listinfo/mod_python
> >
>
>



More information about the Mod_python mailing list