[mod_python] mod_python and db connections - architectural advice needed

On 18/02/2005, at 4:08 AM, Gustavo Córdova Avila wrote:

> BUT, I'd really like to hear other people's impressions on this.  One 
> of the things I've bumped into is when creating the first connection, 
> sometimes, upon initial import, many connections are made 
> simultaneously and then abandoned, but not destroyed.  This really 
> irks me; I've tried gating access to the base DB connection object 
> with a lock, but it's no good.
>
> The base DB object is a module-global reference to a psycopg 
> connection, and I keep around a lock whenever I need to change it 
> (because I use apache2 configured with worker). I don't know if I'm 
> stepping on my own toes or what, but it works:

If you are using worker MPM, sounds like the abandoned and not 
destroyed connections
may relate to the thread locking and duplicate interpreter creation 
bugs I highlighted
end of last year.

The problem I am talking about was one where the same module could be 
imported into two
different threads at almost the same time. If the database connections 
were global data
and new imports just replaced the old, the first set to be created 
would effectively
be thrown away, but may not be shutdown properly.

The other problem is that two whole interpreters could be created at 
the same time for
the same part of the document tree. Again, one would get thrown away, 
but stuff like
database connections may not get shutdown properly.

Both these problems would manifest when using a multithreaded MPM and 
on startup there
was enough access going on to trigger the locking problems.

See:

   http://www.dscpl.com.au/projects/vampire/PATCHES

for the code fixes.

The problem may also be caused by using auto reload and not ensuring 
that you treat the
database connection pool specially so that the old doesn't just get 
reloaded on top of
with it being lost but not shutdown.

Graham