[mod_python] Cache management in a mod_python environment

2009/9/5 Joel Welling <welling at psc.edu>:
> Hi folks;
>  I've written a web utility which generates images in mod_python, and
> I'd like to keep recently generated images in a LRU cache.  I don't
> understand the threading environment, however, and if many instances of
> my app try to manage the cache simultaneously they'll get it wrong.
>  If I run a mod_python page under apache, each httpd ends up creating
> its own instance of the mod_python program, correct?

Each process has its own Python interpreter instance.

> If a given httpd
> services two requests at the same time, does a separate instance of the
> module get created or does the function generating the page get run in
> two different threads?

For a thread MPM such as worker MPM for Apache, multiple threads
execute in context of same interpreter and same loaded code. In single
thread prefork MPM for Apache, all requests are serialised, so not an
issue.

> Is there any data structure I can attach cache
> information to that will survive between requests within the same httpd?

Within same httpd process, yes. Basically place it as global data in a
standard Python module. If multithreaded, updates and maybe reads may
need to be threaded protected of requires multiple operations which
together aren't atomic.

Have a read of:

  http://www.dscpl.com.au/wiki/ModPython/Articles/TheProcessInterpreterModel
  http://www.modpython.org/live/current/doc-html/pyapi-apmeth.html

In the latter, read about import_module() function. This is relevant
as that is what is used to handle imports of mod_python script files
and there is a section in there about preserving cached data across
reloads of the script when changes are made to script.

Graham