[mod_python] Storing large amounts of data in RAM

Aaron Gallagher wrote:
> I'm working on a web-application port of a text game, and I'm just 
> wondering what the best way for me to implement this is.
>
> Game data is stored in a single python object which is pickleable. I'm 
> not sure of the exact size of the data in memory, but the pickled 
> files are between 600 and 900k uncompressed. I would like to store 
> each object in memory rather than saving and loading the object for 
> each request. So, what is the best way to do this? I'm pretty sure I 
> remember from earlier posts that it's not possible to have memory 
> that's shared between multiple interpreters. Is it possible to only 
> use one interpreter and keep all of the data objects in that interpreter?
>
> Also, other than having a cron job send an HTTP request to the server 
> intermittently, is there a way to have data objects save themselves to 
> a file if their player hasn't made a request in a certain amount of time?

You could try to use in-memory caching, but if the files are that small, 
particularly if you are on linux, the OS will cache some of the files 
automatically.  As an example, if you have a system that has a 
reasonable amount of RAM, try reading a text file that is a few hundred 
Mb.  Then, immediately read it again.  The second time around will be 
faster, likely, due to caching. 

Sean