[mod_python] having modpython make a data base connection

Graham Dumpleton graham.dumpleton at gmail.com
Wed Jun 13 03:31:07 EDT 2007

On 13/06/07, David Bear <David.Bear at asu.edu> wrote:
> On Wed, Jun 13, 2007 at 11:58:06AM +1000, Graham Dumpleton wrote:
> > For starters, if you haven't read:
> >
> >  http://www.dscpl.com.au/wiki/ModPython/Articles/TheProcessInterpreterModel
> >
> > do so.
> If I structure the program to exist in a single file, make my db
> connections from there, does this imply that I could have 'multiple'
> connections made to the data base?
> I am using the worker mode of apache on linux. This means that I do
> have multiple child apaches and threads in each. So, I'm not sure if I
> have 5 children and 2 thread per child that would imply 10 connections
> to the data base?


The handler module is only loaded once for each process (ignoring
reloads at this point), and all the threads in each process execute
against the same loaded module. If the database client supports one
connection and then multiple sessions across that one connection, then
you can create and/or cache the database connection object at global
scope within the handler module. In that case, there would only be one
connection per process.

BTW, 2 threads per child in worker is very low, normally it would be
25 by default. Also, I strongly discourage creating database
connections as a side effect of importing the handler module as a
failure to connect can then leave the web application unusable as
nothing is going to re-trigger the module import to allow the
connection code to rerun. Thus, even though it delays the creation of
the connection, I would only do it the first time a handler requires
the connection. This does mean that connection creation and caching
has to be thread protected to stop more than one being created at the
same time.

Now, if instead you create the connection for every request as I am
sure some will promote as the simplest approach and for some database
also more than adequate, then every request can result in a separate
connection thus meaning as many as number of process times number of
threads. Noting that Apache may create more than 2 child processes
when under heavy load.

> > Second, be very careful about putting database connection objects in
> > modules which are candidates for reloading. For stuff about how to
> > deal with this if you want to do that, read documentation for
> > import_module() in:
> I don't know why a module would be reloaded. I'm still thinking in
> terms of the cgi model where the script runs, connection made to db,
> data is updated, connection closed, script dies, done.
> Under what conditions would a module be reloaded?

For a handler module which is located in the document tree (not on
sys.path), it will be automatically reloaded when its code file, or
the code file of any related module also in the document tree or found
on the mod_python module importers path has changed. Same applied to
publisher modules.

The issue with reloading is that the module is reimported into a new
module instance and the old data thrown away. If however the old data
isn't able to clean itself up properly through destructors, ie. close
database connections, you will get resource leakage. The documentation
I pointed you to shows how you can supply hooks to copy data across
from old module to new module when a reload occurs, thus allowing
already open database connection object to be reused.

If you don't want to deal with the possibilities of module reloading,
then turn it off using the directive:

  PythonAutoReload Off

Do this though and you will need to restart Apache when you make any
code changes to modules even if they are in the document tree (not on

> And if I use the
> handler mode (no publisher or psp) and the data base connection is
> made in the handler function, does that imply the connection is made
> when apache is started (and the script is run for the first time) and
> lives untill apache dies?

Depends on whether you are then caching the database connection object
at global scope in module as I mentioned before. Again note that such
caching has to be thread protected to stop multiple requests trying to
do it at the same time in the same process.


> >  http://www.modpython.org/live/current/doc-html/pyapi-apmeth.html
> >
> > I'll let others argue the merits of database connection pooling versus
> > a connection per request as not an area I have direct experience with.
> As they say, I'm against premature optimization -- I don't know enough
> to care at this point. I just want to run in 'safest way', that means
> the most predictable.
> >
> > Graham
> >
> > On 13/06/07, David Bear <David.Bear at asu.edu> wrote:
> > >I don't want to start a flame ware here, but I'm starting to develop
> > >and application that will connect to a postgresql server.
> > >
> > >I realize there are probably a dozen things I should know about
> > >modpython before I begin.
> > >
> > >I will be use the standard python handler. I won't be using publisher
> > >or psp (at least I think).
> > >
> > >I am afraid of issues like having modpython make a db connection
> > >object that is not shared properly --. The more terrifying question is
> > >what should be be asking.
> > >
> > >I wonder if someone has develop a top ten rules of thumb when using
> > >modpython with sql data base connections?
> > >
> > >
> > >--
> > >David Bear
> > >phone:  602-496-0424
> > >fax:    602-496-0955
> > >College of Public Programs/ASU
> > >University Center Rm 622
> > >411 N Central
> > >Phoenix, AZ 85007-0685
> > > "Beware the IP portfolio, everyone will be suspect of trespassing"
> > >_______________________________________________
> > >Mod_python mailing list
> > >Mod_python at modpython.org
> > >http://mailman.modpython.org/mailman/listinfo/mod_python
> > >
> --
> David Bear
> phone:  602-496-0424
> fax:    602-496-0955
> College of Public Programs/ASU
> University Center Rm 622
> 411 N Central
> Phoenix, AZ 85007-0685
>  "Beware the IP portfolio, everyone will be suspect of trespassing"

More information about the Mod_python mailing list