[mod_python] Object persistence?

Fredrik Sandin wrote:
> On Sat, 26 Aug 2006 05:36:14 +0200, Graham Dumpleton 
> <grahamd at dscpl.com.au> wrote:
>>
>> On 25/08/2006, at 11:16 PM, Fredrik Sandin wrote:
>>
>>>
>>> Hello!
>>>
>>> I am new to mod_python and could need some advice. I want to create an
>>> online GUI with mod_python for an existing application written in 
>>> Python.
>>>
>>> The Python application organizes distributed computations and therefore
>>> maintains network connections to a number of computers. Users post jobs,
>>> which are sent somewhere for execution and then the result is sent back
>>> by a callback function. Illustration of principle:
>>>
>>> class MyComputerCluster:
>>>     def __init__(self, callback):
>>>         self.callback = callback
>>>         ...
>>>     def post_job(self, description):
>>>         ...
>>>
>>> def mycallback(userid, result):
>>>     print 'User %d requested: %s' % (userid, result)
>>>
>>> mycluster = MyComputerCluster(mycallback)
>>> mycluster.post_job('compute the meaning of 42')
>>> ...
>>>
>>> Now, there is one major issue - Is it possible to create an object of 
>>> type
>>> MyComputerCluster for each user of the online GUI and keep it in memory
>>> between calls, without spoiling the network connections maintained by 
>>> the
>>> object? Session variables can be used as reference for a persistent 
>>> object,
>>> but how to create it?
>>>
>>> A solution would be to write an independent service that creates the
>>> objects and listens for requests (post_job, get_result), but that takes
>>> time...
>>
>> Does the code above you give as an example already exist and a part of
>> your existing Python application? What framework is your application for
>> performing the distributed jobs written in, or is it some home grown 
>> custom
>> framework?
>>
>> I have a sense of what you want to do, but because no real information is
>> provided about the existing application or what framework it uses, it 
>> is hard
>> to give any advice.
>>
>> Anyway, if the issue is more about how handlers in mod_python might
>> communicate with your application, you might look at XML-RPC using the
>> xmlrpclib module in Python. Depending on the framework the application
>> uses though, it may not be possible to embed an XML-RPC server in the
>> application which could respond to the handlers run under mod_python.
>>
>> Can you perhaps be clearer about the separation between your mod_python
>> handlers and your application and how you intend communicating between
>> the two.
>>
>> Graham
> 
> 
> Dear Graham,
> 
> yes, the code I gave as an example represents an existing Python
> application. It is home-gown, so your suggestion to embed an XML-RPC
> server could do the trick, will have a closer look at it. Thanks!
> 
> By the way, why is it so difficult to support persistent objects,
> a "state", from one call to the next? The mod_php application itself
> has a state, and each user can be associated with a sessid, so the
> problem must be that a user could be served by different mod_php
> processes from one call to the next?

Correct, assuming you s/mod_php/mod_python/. :) Each process gets its 
own interpreter with a separate copy of all your objects. It's exactly 
the same as launching multiple python interpreters from the command 
line. Each one is completely independent, and I'm sure that you wouldn't 
expect a change to some object in one interpreter to change a 
corresponding object in another interpreter? In mod_python it all comes 
down to the mpm you are using. The only one where that has a single 
process where objects could easily be shared across threads is 
mpm-winnt, but this obviously limited to Windows. There is no generic 
mod_python solution that will work with all of the Apache mpm's.

> In that case, would it not be possible to use the sessid to make
> sure that a user is served by the same mod_php process as long
> as the session is valid?

No. Apache selects an available process to handle a request pretty much 
at random. (It may be the same process if there is a KeepAlive, but you 
shouldn't count on it). Apache itself has no knowledge (or interest) in 
the internal workings of your application, such as a session cookie, so 
there is no way associate a process with a session. It works the other 
way around, with the mod_python handler associating a session cookie 
with some data in a persistent store.

Jim