[mod_python] Session Hanging Problems

Harish Agarwal wrote:
> Have you noticed any performance issues?  I want to make sure to keep 
> things fast and am worried that departing from the Apache lock system 
> will degrade performance.  Why was mod python designed to use the apache 
> system and not use session specific locks?

I wasn't around when session handling was first introduced, but I assume 
the apache mutexes where used so that mod_python would be as 
cross-platform as possible. Python's fcntl.flock() is only available on 
*nix platforms.

The big drawback with apache's mutexes is that we are dealing with 
limited resource.

> Also a note for the list - I'm now explicitly unlocking the session 
> after each save to ensure that the session isn't locked during a 
> req.write call.  I've seen one hang since then, but in twelve hours, 
> that is much much better than the issues I was seeing before.

Are you re-locking the session before modifying the session object? If 
not then you've got a possible race condition and you risk losing 
session data. The only safe way to do this multiple times within a 
single request is:

session.lock()
session.read()
session['stuff'] = 'whatever'
session.save()
session.unlock()

Each read / save pair results in a disk access and unpickling/pickling 
the data, which may not be all that great for high performance.

You'll need to make sure the above sequence happens every time you 
modify the session object. If your code is complicated it would be 
pretty easy to lose some session data. You'll also need to make darn 
sure that there are matching lock/unlock pairs. If not you'll deadlock 
the session and in pretty short order you'll end up with a self 
inflicted denial of service as apache runs out of mutexes. Make sure you 
put a try/finally block in there with an unlock() in the finally section.

> I think that this problem directly contradicts the general philosophy 
> behind using the mod_python session handling scheme - I feel like I've 
> read, a few times now, that there is no reason to have to explicitly 
> unlock sessions, 

Then you missed comments about the problem with long running requests 
and session locking, such as the case of a large download. There are 
times when it really is advantageous to explicitly unlock a session. 
It's always best if you can finish with your session object before 
sending the file.

session['stuff'] = 'whatever'
session.save()
session.unlock()
req.sendfile("my_big_file.iso")

> but this is an instance where running everything in its 
> default configuration (not unlocking sessions, flushing the buffer on 
> req.writes) causes serious hanging problems...  In addition, unlocking 
> the session after each save doesn't sit well with me - I don't think the 
> session is thread-safe now (given that there can be multiple save's per 
> request).

It's not even a question of being thread safe. With prefork or worker 
mpms you are also dealing with multiple processes. If we just had to 
worry about one process with many threads life would be much easier.

Jim

> -Harish
> 
> 
> On Dec 4, 2007, at 1:48 AM, David Janes wrote:
> 
>> I'll note for the list that I put in the changes I outlined above ---
>> that is, handled locking entirely ourselves rather than using Apache's
>> lock function -- and I haven't had a hang since.
>>
>> Regards, etc...
>>
>> --David Janes
>> Founder, BlogMatrix
>> http://www.blogmatrix.com
>> http://blogmatrix.blogmatrix.com
>> _______________________________________________
>> Mod_python mailing list
>> Mod_python at modpython.org
>> http://mailman.modpython.org/mailman/listinfo/mod_python
> 
> _______________________________________________
> Mod_python mailing list
> Mod_python at modpython.org
> http://mailman.modpython.org/mailman/listinfo/mod_python
>