[mod_python] How to cache memcache client connection/thread-safe

2009/8/31 Denzel Zhang <denzel.zhang at facilitatedigital.com>:
> Hi there,
>
>
>
> I am currently working on a project to work out an Ads delivery web page
> which needs to get the cache data from backend Memcached Server. The web
> server is Apache + mod_python, the performance I tested after I put
> memcached client lib (_pylibmc) inside the page was not quiet good, the main
> problem I think is that every time network sockets create/destroy will be
> very costly, so I changed the way of data retrieving like following:
>
>
>
> from mod_python import apache, util
>
> import _pylibmc as memcache
>
>
>
> global mc = None
>
> global bConnectionOpen = False
>
>
>
> def handler(req):
>
>
>
>          global mc, bConnectionOpen
>
>
>
>          if bConnectionOpen == False:
>
>                    mc = memcache.client(["127.0.0.1:11211"])
>
>                    bConnectionOpen = True
>
>
>
> ClientID = mc.get("ClientID")
>
> ….
>
>
>
> The performance got much better (700 requests/sec, was just no more than 300
> requests/sec) after I changed to above but I still worry about the data
> thread safe issue. Can I say that all data would be safe under the threads
> of mod_python as its thread-safe? And is there a proper way to cache or
> share the sock connection between multiple mod_python interpreters?

If handle returned by memcache.client() is not thread safe, then this
will fail in a multithread MPM such as worker MPM on UNIX and winnt
MPM on Windows.

If a single handle is multithread safe, then you are better off
initialising the connection at global scope in module as current way
you initialise it isn't thread safe.

In other words, move initialise from inside handler to outside of handler.

Graham