harish at octopart.com
Tue Dec 4 00:35:36 EST 2007
On Nov 26, 2007, at 1:38 PM, Graham Dumpleton wrote: > On 27/11/2007, Harish Agarwal <harish at octopart.com> wrote: >> I'm using session handling with ModPython 3.3.1. Originally I was >> using DbmSession and have since transitioned to a custom MySQL >> Session >> handler. With both session types, however, I've noticed that session >> initialization intermittently hangs (not forever, but takes as long >> as >> four minutes to complete), at a low-ish (a handful of times every >> hour, while receiving, say, on order of a thousand or so requests >> every hour) frequency which seems to scale with the amount of traffic >> we're receiving. I had read that long DbmSession cleanups can cause >> problems, which is why I transitioned to the MySQL system, which >> takes >> < 1 second to complete, but I'm still noticing the long session init >> times. >> >> >> I put some debugging statements into the code and it seems to be >> related to session locking. In particular, it is this function call >> in the lock method of the BaseSession class: >> >> >> _apache._global_lock(self._req.server, self._sid) >> >> >> which is taking some time to complete. I'm not familiar with >> _apache._global_lock (is it used to apply a mutex lock to the >> session?) and am having trouble finding information describing its >> usage - but it seems likely that this is the root cause. In the past >> I've had problems with session locking but have since transitioned >> the >> code to ensure that only one session is created per request, as such: >> >> if not hasattr(req,'session'): >> req.session = Session.MySQLSession(req) >> >> >> Can anyone tell me if this kind of behavior is normal or is >> indicative >> of some common configuration or coding error? Any help would be >> greatly appreciated. > > Ignoring hangs, how long does your longest request normally take to > execute? Are you perhaps performing file uploads that take a > significant amount of time and are holding a session locked for the > whole period of the request? > > To at least allow some level of concurrency, from memory mod_python > holds a small pool of cross process mutex locks. Based on the session > ID (I think) it should consistently pick the same mutex lock each > time. Thus, if a request comes in with the same session ID it would be > blocked while the existing request for that session runs. If however > another request comes in with a different session ID, but where it > maps to the same mutex lock from the pool, it will also be blocked > until the request holding that lock has completed. > > What this means is that the number of mutex locks in the pool > effectively dictates how many parallel session based requests you can > have executed across the whole of the Apache server. Thus, if you have > requests that take a long time while still holding the session lock, > it can lock out other requests until it completes. > > One can makes things a bit better by increasing the number of mutex > locks in the pool, but you have to be careful not to create too many > in case it is using sysvsem and your OS doesn't allocate enough. > > The only other thing to do is to release the lock explicitly as soon > as you no longer need it and don't rely on the cleanup handler for the > request to unlock it. > > In other words, what your application is doing, your own code and how > you have written it could be the problem, and not necessarily > mod_python itself. > > If my memory has totally faded and my description is wrong, someone > please correct me. :-) > > Graham I put in some more debugging statements and I'm finding, very oddly, that the hanging is most likely due to a req.write taking quite a bit of time to complete. I'm able to view the exact same webpage without experiencing a hang, so the hang seems to be fairly sporadic. Any ideas?