[mod_python] Connection Pools

On 17/01/2008, j a <inboundfilter at gmail.com> wrote:
> The topic has been covered many times before, but I'm hoping someone
> can clarify what to expect when using the technique described in this
> thread: http://www.modpython.org/pipermail/mod_python/2005-October/019287.html
>
> Specifically:
> > Thus, it is preferable that resource allocation and the storage of the
> > handle be done in a Python module that is not imported using the module
> > loading system of mod_python but that "import" be used instead. Such a
> > module should preferably not be in the document tree but elsewhere on
> > the Python path. As described in the link above, use an access function
> > in that module which returns, and creates as necessary, the resource
> > handle.
>
> That's basically what I'm doing, under Apache 2.2 with mpm_worker,
> mod_python 3.3, Publisher handler, postgres, psycopg2. In my
> implementation, the module has one top-level name for a
> psycopg2.PersistentConnectionPool object, an init_pool() function to
> set that object, a get_connection() function which returns a reference
> to it, and a release(conn) function to return the connection to the
> pool. Additionally, get_connection() will call init_pool() if the pool
> object is None. The applications use only the get_connection() and
> release(conn) functions. As far as I can tell, this is working fine.
>
> What I'm unsure of:
> 1) How many pools could I potentially end up with? One pool per httpd
> process, or per interpreter?

One pool per interpreter per process.

Noting that default for mod_python is one interpreter for all requests
against a virtual host.

> 2) Are those pools and their connections available to all the threads
> in a given process?

No, only those in the same interpreter context.

> 3) Will imports in the applications overwrite the pool object
> initialized by prior calls to init_pool() or get_connection()? Related
> to #2? Does module caching deal with this?

Depends on whether the module is a candidate for module reloading or
not. See about __mp_clone__ hooks in import_module() documentation of:

  http://www.modpython.org/live/current/doc-html/pyapi-apmeth.html

The easiest thing to do is separate out the pool management code into
separate module which is import from Python module search path, and
thus not using mod_python specific module reloading mechanism and you
will not have that issue.

> 4) Will the pool(s)/connections be closed when Apache kills off a
> process? Automatically? Will I need a specific cleanup function?
> How/when would that be called()?

See register_cleanup() in same documentation.

Note that it is not reliably called given that process could crash or
not get shutdown properly.

Any atexit registered functions are not called in mod_python as
mod_python isn't implemented properly to ensure they are.

Graham

> I've also read this whole thread:
> http://www.modpython.org/pipermail/mod_python/2005-September/019092.html,
> and this article: http://www.dscpl.com.au/articles/modpython-004.html,
> both of which were helpful with how to deal with the connections, but
> don't really talk about what I'm asking here.
> _______________________________________________
> Mod_python mailing list
> Mod_python at modpython.org
> http://mailman.modpython.org/mailman/listinfo/mod_python
>