[mod_python] Session class or Memcache to reduce load?

> -----Original Message-----
> From: Graham Dumpleton [mailto:graham.dumpleton at gmail.com]
> Sent: Monday, August 06, 2007 2:25 AM
> > Hello, I am sorry if my question is too basic. I would like to reduce
> the
> > load on apache 2.0 (on 2.6.9 linux) that is running with prefork MPM.
> > There are two main things that are causing the load:
> 
> How do you know that these two things are causing the load?
> 

Because all this apache server (this physical machine) does is serving
thumbnails. 

> > a) Thumbnail images that are requested repeatedly
> 
> Are you serving up the thumbnails from Python code dynamically or
> allowing Apache to serve them up form a static file?

Dynamically.
 
> 
> > b) A simple DB query is necessary to locate an image file after the
> request.
> > The result of the query does not change. DB is located on another
> machine.
> >
> > My first question, do we need to cache \thumbnail images at all, or
> the
> > file-caching by the OS is sufficient?
> 
> OS file caching probably will not make much difference. Where one can
> waste a lot of cycles though is by using Python code to serve up the
> images. Thus, if Python is involved in serving up the images, how is
> it being done?
> 

The URI in the client's request contains an image code. Python queries the
DB (on a separate machine) to convert the URI into the image file location,
and then uses 

        f =  open(file_path)
        im_data = f.read()
        f.close()

then it determines image type using PIL

	  im = Image.open(file_path)
	  im_type = im.format.lower()

(Image.open() is a lazy operation in PIL, so I think it does not have to do
much to determine the image format)
Then python writes the data to the client

        self.req.content_type = 'image/' + im_type
        self.req.headers_out['Content-Length'] = str(len(im_data))
        self.dict['bin_data'] = im_data

 
> > Second question, to cache the results of the query, should we use
> > mod_python's Session class ( wich will use DbmSession since we are
> using
> > prefork MPM), or memcache?
> 
> Traditionally people use memcached for this.
> 
Why? For performance reasons?

> > For certain reasons in the application logic we cannot use apache's
> > mod_cache.
> What reasons?
> 

Because sometimes the images are dynamically banned (flagged) by users, and
in that case we need to render a special image. The DB query that we use
gives None for the image file path when it's banned. So when the image
status changes, we will need to dynamically purge the file path of the image
that is be cached.  

> Graham