[mod_python] Performance Question

Thomas Moyer tmmoyer at cse.psu.edu
Thu Sep 11 08:18:21 EDT 2008


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I am having some problems with performance when using mod_python to  
serve some dynamic content.  The request handler gathers information  
being generated by background processes on the system, and then  
creates a simple XML document that is sent to the client. Here is the  
code I am using with some explanation.

     requestedFilePath = "/" + req.filename.split("/")[-1] 
[:-4].replace("_", "/").replace("-",".")
     sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
     sock.connect(ATTService)
     Attest = message.recvMsg(sock)[0]
     sock.close()

     fmt = "%ds" % len(requestedFilePath)
     msg = struct.pack(fmt, requestedFilePath)
     sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
     sock.connect(MHTService)
     message.sendMsg(sock, msg, fmt)
     MHTList = message.recvMsg(sock)[0]
     sock.close()

     req.content_type = 'text/xml'
     req.write("<ACA>%s%s</ACA>" % (Attest,MHTList))
     return apache.OK

There are 3 different pieces here.  The first block connects to a  
daemon running on the system that is monitoring the system. I connect  
to a Unix domain socket and receive the most recent information it  
has.  I then do the same thing with another daemon that is gathering  
information about files that are available on the system. Finally I  
combine these two blocks of information and send it to the client.

My problem is, when I start using benchmarking tools like JMeter, and  
I put the system under any sort of load (100 clients for example), the  
first recvMsg call (Attest = message.recvMsg(sock)[0]) takes about 12  
seconds to complete per client.  I watch top during the run and see  
that the apache processes are completely occupied (usually showing 120+ 
% CPU usage).  The system running apache is an 8-core machine with  
16GB of RAM so I'm pretty sure there isn't a resource problem for  
either memory or processor.

I'm not really sure where to look next to understand why a simple  
socket recv() call is taking 12 seconds to complete.  The amount of  
data being recv()'d is ~80K of text if that makes a difference.

~tom
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iEYEARECAAYFAkjJDI0ACgkQa7Yypxfw9TeOPwCfb29vCZlna6WZL9X/SASxT4Ms
cxYAnRyDlcMF6gWuSgdT8HHPnq1Urs/H
=NPk2
-----END PGP SIGNATURE-----


More information about the Mod_python mailing list