sean at monkeysnatchbanana.com
Fri Apr 11 09:00:24 EDT 2008
On Apr 10, 2008, at 2:53 PM, Culley Harrelson wrote: >> >> You might also want to look at caching solutions, especially if you >> have plenty of RAM :) You could probably roll your own with >> Memcached >> without too much trouble, and this could potentially ease the load on >> the CPU quite a bit. > > The database load doesn't seem to be really a big problem. There > are hundred > of thousands of pages, each translated into 6 languages. I tinkered > around > with mod_cache a bit but it couldn't handle it. > As a better way to tackle that. We have a mod_perl based application ( that is being transfered to mod_python ) and during high load period we found with profiling that there was no area that we could improve it was just the entire process was taking too long and to many reuqests were backing up and eventually either two many apache processes were launched and swapping and thrashing started or we had timeouts. this on code that took far less than 1 second from start to end. the code in question was mostly dynamically generating html, we spend about 60 minutes rewriting the code to store the final output for each different dynamic request in memcache and configuring out front end nginx server to do the following: check for file check in memcache check with application the 2nd step was the new part and we havent had a peep since then. server load ( debian linux ) hasnt crossed much beyond .5 since then and is usually below .3 as a bonus, our client side response time went down because the front end static server is competing less with the dynamic server for cpu time.