[mod_python] Problem keeping persistent object in memory (global)

Thank you Graham for the details on the proper use and issues with  
using a singleton.

I have modified my code and it seems that now the object mysitepool  
is staying in memory yeah!

The only other issue is that it seems the Site object I am inserting  
in the Pool is getting initialized every-time... It is like the  
Queueing mechanism is not working properly and initializing using the  
Contructor every-time.

I am using the pool.py written by Andy dustman (http://dustman.net/ 
andy/python/), and basically the mysitepool is now being initialized  
as shown below. What I am noticing is that the Site objects which are  
created in the Pool (basically in the Queue) are initialized each  
time you call mysitepool.get(). The behavior I am looking for is to  
keep all X Site objects in the Pool to stay in memory. FYI the Site  
object initialize a connection to the DB among other things, so it is  
critical that it is initialized and destroyed cleanly.

Thanks for sharing your wisdom once more ;)

Sébastien

handler.py (the one that mod_python points to via PythonHandler in  
the .htaccess)
----------
from mod_python import apache
from heracles.mysitepool import mysitepool

def handler(req):
     """
     Standard mod_python handler code for Heracles Web Application  
Framework
     """
     mySite = mysitepool.get()
     return mySite(req)
     mysitepool.put(mySite)

mysitepool.py
-------------

from mod_python import apache
from heracles.pool import Pool, Constructor
from heracles.site import Site

if not globals().has_attr("mysitepool"):
     global mysitepool
     mysitepool = Pool(Constructor(Site), apache.mpm_query(6))

pool.py
-------
# Credits: http://dustman.net/andy/python/

from Queue import Queue, Full, Empty

class Pool(Queue):
     """Manage a fixed-size pool of reusable, identical objects."""
     def __init__(self, constructor, poolsize=5):
         Queue.__init__(self, poolsize)
         self.constructor = constructor
     def get(self, block=1):
         """Get an object from the pool or a new one if empty."""
         try:
             return self.empty() and self.constructor() or Queue.get 
(self, block)
         except Empty:
             return self.constructor()
     def put(self, obj, block=1):
         """Put an object into the pool if it is not full. The caller  
must
         not use the object after this."""
         try:
             return self.full() and None or Queue.put(self, obj, block)
         except Full:
             pass

class Constructor:
     """Returns a constructor that returns apply(function, args, kwargs)
     when called."""
     def __init__(self, function, *args, **kwargs):
         self.f = function
         self.args = args
         self.kwargs = kwargs
     def __call__(self):
         return apply(self.f, self.args, self.kwargs)




On Jun 6, 2005, at 5:08 PM, Graham Dumpleton wrote:

> On 07/06/2005, at 3:02 AM, Huzaifa Tapal wrote:
>
>
>> One option that I use is to create a singleton in the module that  
>> I want to be global at the first import of that module.  So lets  
>> take your mySitePool module for example:
>>
>> class mySitePool:
>>    def __init__(self):
>>          ' suite
>>
>> ....
>>
>> global mysitepool
>> mysitepool = mySitePool()
>>
>
> Using singleton approach as described will help, now for some  
> really obscure
> stuff in case later on you find other strange problems.
>
> First is that if auto reloading of modules is on and the singleton  
> is stored
> in a module which at any time might get reloaded because the file  
> is changed,
> the act of reloading will replace the singleton object, resetting  
> it in the
> process.
>
> This means that the previously created pool object will become  
> inaccessible
> and potentially resources it uses in the way of database  
> connections might
> not get shutdown properly if that required an explicit step not  
> performed by
> normal Python object deletion.
>
> At the same time, a new pool object will be created and perhaps  
> another set
> of database connections. If old ones aren't cleaned up in the  
> inaccessible
> pool and new ones are created due to reloads, you might end up with  
> too many
> connections and not be able to create any more.
>
> To make the code durable and resistant to reloads, intended or not,  
> it is
> useful to write code as:
>
> if not globals().has_attr("mysitepool"):
>   mysitepool = mySitePool()
>
> What happens is that reload occurs on top of the existing module.  
> By seeing
> if the pool object already exists in globals, you simply avoid  
> creating it a
> second time.
>
> Also note that there are some threading issues in mod_python as  
> well, and
> if you don't do this fiddle but module reloading is turned off, you  
> can still
> have a problem. This is because the threading issues can result in  
> a module
> being loaded twice by mistake, as if a reload had occurred.
>
> For this, see patches at:
>
>   http://www.dscpl.com.au/projects/vampire/patches.html
>
> Graham
>
>