[mod_python] Using shared memory to do global persistence

APR has facilities to do shared memory in a portable fashion; APR is part of
Apache 2.0, so there isn't much reason to go grab any other library.

You can also use APRUTIL's apr_rmm.h to manage sub-allocations within the
shared memory segments. The problem is that a shared memory segment could be
mapped to different addresses in different processes. Thus, you want to hold
onto offsets into a shared memory segment. apr_rmm.h helps with managing
these subblocks and working with offsets rather than direct pointers. Note
that apr_rmm also handles locking so that you can have multiple processes
allocating (simultaneously) from a shared mem segment.

You can then layer additional Python facilities on top of this substrate.

Cheers,
-g

On Wed, Jun 25, 2003 at 07:32:14AM -0700, Jonathan Gardner wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> I am thinking of implementing a shared memory module that does more than just 
> "read" and "write". Instead, I want something more transparent to the 
> developer, so that you say "Go get this dict at this shared memory key" or 
> "Create a new dict in shared memory at this key", and have the entire dict 
> available like a namespace.
> 
> Obvious applications would include the ever-present problem of shared data 
> between apache mod_python processes. This would solve it once and for all. 
> Perhaps one day we could get it working with RPC and have processes on 
> seperate machines sharing data as well.
> 
> The scope of this project would be implimenting new types that derive from all 
> of the basic types so that:
> 
> 1) When accessing the data in the type, it would gain a lock on the shared 
> memory segment that stores its data using a semaphore, read the data and 
> unserialize it, then return that data.
> 2) When storing the data in that type, it would gain a lock on the shared 
> memory segment, then store the data over the existing data.
> 
> I am thinking there are really two ways to go about this.
> 1) Implementing it all in Python. The idea here would be that we would 
> serialize and unserialize the data using cPickle.
> 2) Implementing it all in C. The idea here is that we would reuse the same 
> data structures that the basic data types use. We would store and read the 
> binary data behind the basic data types so that they can be as easily stored 
> in shared memory as they are stored in real memory.
> 
> Does anyone know if something like this is implemented or in the process of 
> being implemented? Does anyone want to join me in this endeavor? 
> 
> - -- 
> Jonathan Gardner <jgardner at jonathangardner.net>
> (was jgardn at alumni.washington.edu)
> Live Free, Use Linux!
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.2.1 (GNU/Linux)
> 
> iD8DBQE++bJuWgwF3QvpWNwRAq0pAJ4ipACeWt63fSbtheNmxQfc217sQwCgj4D7
> /ID5Uq/oK8hAYQfmvvIGliE=
> =4zo4
> -----END PGP SIGNATURE-----
> 
> _______________________________________________
> Mod_python mailing list
> Mod_python at modpython.org
> http://mailman.modpython.org/mailman/listinfo/mod_python

-- 
Greg Stein, http://www.lyra.org/