Fwd: [mod_python] Connection Pools

Following missed the mailing list. Included in case anyone was
following this issue.

---------- Forwarded message ----------
From: j a <inboundfilter at gmail.com>
Date: 18 Jan 2008 07:53
Subject: Re: [mod_python] Connection Pools
To: Graham Dumpleton <graham.dumpleton at gmail.com>


Argh... just realized I did reply earlier, instead of reply all. Sorry
about that.

Thanks very much for your help, it definitely solidifies what I had
floating around in my head about how these things worked together.

-John

On Jan 17, 2008 3:41 PM, Graham Dumpleton <graham.dumpleton at gmail.com> wrote:
> On 18/01/2008, j a <inboundfilter at gmail.com> wrote:
> > > > What I'm unsure of:
> > > > 1) How many pools could I potentially end up with? One pool per httpd
> > > > process, or per interpreter?
> > >
> > > One pool per interpreter...
> > OK, I understand that, but...
> >
> > > ... per process
> > >
> > > Noting that default for mod_python is one interpreter for all requests
> > > against a virtual host.
> >
> > I'm confused on this point. Section 4.1 in the doc says "...scripts in
> > the same virtual server execute in the same subinterpreter", which
> > makes sense until "...exists until the Apache process dies". So say I
> > have two virtual hosts, each with an app that instantiates a pool, and
> > two httpd processes (from the default mpm_worker config). Would I get
> > four interpreters (and four pools)?
>
> Yes. Each process would have a sub interpreter instance for each virtual host.
>
> See:
>
>   http://www.dscpl.com.au/wiki/ModPython/Articles/TheProcessInterpreterModel
>
> > > > 3) Will imports in the applications overwrite the pool object
> > > > initialized by prior calls to init_pool() or get_connection()? Related
> > > > to #2? Does module caching deal with this?
> > >
> > > Depends on whether the module is a candidate for module reloading or
> > > not. See about __mp_clone__ hooks in import_module() documentation of:
> > >
> > >   http://www.modpython.org/live/current/doc-html/pyapi-apmeth.html
> >
> > This would only be an issue in a dev environment, correct? With auto
> > reloading disabled in production, would I have to even worry about
> > this at all?
>
> Better to be safe than sorry. There is an unfixed bug in mod_python
> which can in certain race conditions cause the module to be reloaded
> even if reloading is off.
>
> > That section raises another question: without auto reloading, but
> > running under mpm_worker, do I still need to contend with thread
> > locks?
>
> That depends on your code and how you use global data.
>
> > > The easiest thing to do is separate out the pool management code into
> > > separate module which is import from Python module search path, and
> > > thus not using mod_python specific module reloading mechanism and you
> > > will not have that issue.
> >
> > I *think* this is what I'm doing. I have the pool management code (the
> > three functions I describer earlier) in a separate module
> > (pool_manager.py). Another module (data.py) imports that; it contains
> > functions for retrieving, updating, inserting specific data from/to
> > the database. The app imports that second module. (just trying to keep
> > the db stuff out of the app itself).
>
> But is depends on where the files live. If you are using publisher and
> in the same directory as index.py, then may be managed by mod_python.
>
> Print out __name__ in each loaded module. It if it is '_mp_??????', it
> is managed by mod_python and reloadable.
>
> > > > 4) Will the pool(s)/connections be closed when Apache kills off a
> > > > process? Automatically? Will I need a specific cleanup function?
> > > > How/when would that be called()?
> > >
> > > See register_cleanup() in same documentation.
> > >
> > > Note that it is not reliably called given that process could crash or
> > > not get shutdown properly.
> > >
> > > Any atexit registered functions are not called in mod_python as
> > > mod_python isn't implemented properly to ensure they are.
> >
> > I gave this a try a while back but I didn't have a way to test it. I'm
> > writing a small test app that makes a lot of requests for a URL (a
> > function in index.py) which opens/closes db connections. I'm hoping
> > all these requests will cause apache to kill off a process and start
> > up another, and to put some load across the threads under each
> > process.
>
> Set MaxRequestsPerChild directive in Apache to a small number as that
> will force processes to be recycled after that many requests.
>
> Graham
>