[mod_python] Session Hanging Problems

On Nov 26, 2007, at 1:38 PM, Graham Dumpleton wrote:

> On 27/11/2007, Harish Agarwal <harish at octopart.com> wrote:
>> I'm using session handling with ModPython 3.3.1.  Originally I was
>> using DbmSession and have since transitioned to a custom MySQL  
>> Session
>> handler.  With both session types, however, I've noticed that session
>> initialization intermittently hangs (not forever, but takes as long  
>> as
>> four minutes to complete), at a low-ish (a handful of times every
>> hour, while receiving, say, on order of a thousand or so requests
>> every hour) frequency which seems to scale with the amount of traffic
>> we're receiving.  I had read that long DbmSession cleanups can cause
>> problems, which is why I transitioned to the MySQL system, which  
>> takes
>> < 1 second to complete, but I'm still noticing the long session init
>> times.
>>
>>
>> I put some debugging statements into the code and it seems to be
>> related to session locking.  In particular, it is this function call
>> in the lock method of the BaseSession class:
>>
>>
>> _apache._global_lock(self._req.server, self._sid)
>>
>>
>> which is taking some time to complete. I'm not familiar with
>> _apache._global_lock (is it used to apply a mutex lock to the
>> session?) and am having trouble finding information describing its
>> usage - but it seems likely that this is the root cause.  In the past
>> I've had problems with session locking but have since transitioned  
>> the
>> code to ensure that only one session is created per request, as such:
>>
>> if not hasattr(req,'session'):
>>             req.session = Session.MySQLSession(req)
>>
>>
>> Can anyone tell me if this kind of behavior is normal or is  
>> indicative
>> of some common configuration or coding error?  Any help would be
>> greatly appreciated.
>
> Ignoring hangs, how long does your longest request normally take to
> execute? Are you perhaps performing file uploads that take a
> significant amount of time and are holding a session locked for the
> whole period of the request?
>
> To at least allow some level of concurrency, from memory mod_python
> holds a small pool of cross process mutex locks. Based on the session
> ID (I think) it should consistently pick the same mutex lock each
> time. Thus, if a request comes in with the same session ID it would be
> blocked while the existing request for that session runs. If however
> another request comes in with a different session ID, but where it
> maps to the same mutex lock from the pool, it will also be blocked
> until the request holding that lock has completed.
>
> What this means is that the number of mutex locks in the pool
> effectively dictates how many parallel session based requests you can
> have executed across the whole of the Apache server. Thus, if you have
> requests that take a long time while still holding the session lock,
> it can lock out other requests until it completes.
>
> One can makes things a bit better by increasing the number of mutex
> locks in the pool, but you have to be careful not to create too many
> in case it is using sysvsem and your OS doesn't allocate enough.
>
> The only other thing to do is to release the lock explicitly as soon
> as you no longer need it and don't rely on the cleanup handler for the
> request to unlock it.
>
> In other words, what your application is doing, your own code and how
> you have written it could be the problem, and not necessarily
> mod_python itself.
>
> If my memory has totally faded and my description is wrong, someone
> please correct me. :-)
>
> Graham


I put in some more debugging statements and I'm finding, very oddly,  
that the hanging is most likely due to a req.write taking quite a bit  
of time to complete.  I'm able to view the exact same webpage without  
experiencing a hang, so the hang seems to be fairly sporadic.

Any ideas?