[mod_python] Custom handler thread safety

Thanks Graham, your response helped a lot.  I wanted to ask another question
related to my original post.  If I am taking advantage of the shared memory
that using multithreaded MPM and storing objects such as db connections and
parsed page templates, do you see a threat with not locking the thread when
making use of those shared resources.

To be more clear, if I have resource object A in the shared memory and
without any kind of thread locking in place, Thread 1 is using lets say the
database connection object, at the same time Thread 2 needs the same
connection object, would that cause any type of a Race Condition?

Do you think it's a good idea to wrap usage of those shared object such as
when I am retrieving it from the cache with the lock.acquire() call on the
thread?

You are of great help, thanks a lot.

Hozi

-----Original Message-----
From: Graham Dumpleton [mailto:grahamd at dscpl.com.au] 
Sent: Thursday, February 03, 2005 4:31 PM
To: Huzaifa Tapal
Cc: mod_python at modpython.org
Subject: Re: [mod_python] Custom handler thread safety

There are bugs in mod_python if using threaded MPM. See:

  http://www.dscpl.com.au/projects/vampire/PATCHES

This may in part contribute to your problems.

Original discussion about it is in mailing list archive. See:

  http://www.modpython.org/pipermail/mod_python/2004-October/016634.html

There are emails about the second part of the bug as well, but can't
spot them right now.

There are also various gotchas when using autoreloading that may also
be affecting you.

Look at the patches as a starting point and then we can perhaps go
step by step through some of your issues and how you are implementing
your code.

Graham

Huzaifa Tapal wrote ..
> Hello All,
> 
>  
> 
> I wrote a custom mod_python handler that kind of mimicks the cgihandler
> but
> really is an extension to a web framework we have in house.  Just to be
> safe, I implemented thread locking into the handler before any request
> is
> processed.  The application is running on a debian box running Apache 2
> multithreaded with mod_python 3.13.  We are gaining huge performance
> increases by caching our template objects and db connection objects.
> 
>  
> 
> The problem I am running into is that if I run through the application,
> each
> request takes on average 300 ms to process.  However, when we benchmark
> with
> 20 concurrent users, the average goes up to around 2200 ms.  I am very
> sure
> that this is due to a thread locking shared objects in memory which
results
> in another thread waiting for the lock to be released.
> 
>  
> 
> If I take the thread locking mechanism out then we run into problems with
> there being too many connections being made to the MySQL db if the cached
> connection is being used and then the db starts dropping connections.
> 
>  
> 
> Any help would be much appreciated.  Even if there are any other good
> handlers out there other than cgihandler and publisher.
> 
>  
> 
> Hozi