Thomas Moyer
tmmoyer at cse.psu.edu
Thu Sep 11 08:18:21 EDT 2008
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I am having some problems with performance when using mod_python to serve some dynamic content. The request handler gathers information being generated by background processes on the system, and then creates a simple XML document that is sent to the client. Here is the code I am using with some explanation. requestedFilePath = "/" + req.filename.split("/")[-1] [:-4].replace("_", "/").replace("-",".") sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) sock.connect(ATTService) Attest = message.recvMsg(sock)[0] sock.close() fmt = "%ds" % len(requestedFilePath) msg = struct.pack(fmt, requestedFilePath) sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) sock.connect(MHTService) message.sendMsg(sock, msg, fmt) MHTList = message.recvMsg(sock)[0] sock.close() req.content_type = 'text/xml' req.write("<ACA>%s%s</ACA>" % (Attest,MHTList)) return apache.OK There are 3 different pieces here. The first block connects to a daemon running on the system that is monitoring the system. I connect to a Unix domain socket and receive the most recent information it has. I then do the same thing with another daemon that is gathering information about files that are available on the system. Finally I combine these two blocks of information and send it to the client. My problem is, when I start using benchmarking tools like JMeter, and I put the system under any sort of load (100 clients for example), the first recvMsg call (Attest = message.recvMsg(sock)[0]) takes about 12 seconds to complete per client. I watch top during the run and see that the apache processes are completely occupied (usually showing 120+ % CPU usage). The system running apache is an 8-core machine with 16GB of RAM so I'm pretty sure there isn't a resource problem for either memory or processor. I'm not really sure where to look next to understand why a simple socket recv() call is taking 12 seconds to complete. The amount of data being recv()'d is ~80K of text if that makes a difference. ~tom -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAkjJDI0ACgkQa7Yypxfw9TeOPwCfb29vCZlna6WZL9X/SASxT4Ms cxYAnRyDlcMF6gWuSgdT8HHPnq1Urs/H =NPk2 -----END PGP SIGNATURE-----
|