|
M Chaudhary
igp-dev at infogridpacific.com
Tue Aug 23 02:12:56 EDT 2005
Hello Orr, Steve ,
As you told i tested my application using "ab" tool for three
cases (each for single request and 5 concurrent requests ):-
and the results were as follows (the following log was generated from my
mod-python handler itself using time.time() method ) ----
Attached is the test module . Please check it and view the following
results.
#---------------------------------------------------------------------------------------------------
# Test 1 -- dbxml :- get a document and write it to browser
#---------------------------------------------------------------------------------------------------
0.00605797767639 # single request
0.0058479309082
0.00573110580444
0.148849010468
0.00549292564392
0.00417685508728 # 5 concurrent requests
0.00559282302856
0.00551891326904
0.00600981712341
0.00549983978271
#---------------------------------------------------------------------------------------------------
# Test 2 -- pxtl :- pxtl process a File and write it to browser
#---------------------------------------------------------------------------------------------------
1.65334200859 # single request
0.25177192688
0.261835098267
0.294239997864
0.000832080841064
0.428733825684 # 5 concurrent requests
0.440675020218
0.269226074219
0.237013816833
0.13219499588
#---------------------------------------------------------------------------------------------------
# Test 3 -- pxtl and dbxml :- get a document from dbxml container ,pxtl
process it and write it to browser
#---------------------------------------------------------------------------------------------------
3.01918792725 # single request
13.4880371094
14.6321718693
15.9680738449 # 5 concurrent requests
16.1954669952
0.010479927063
17.400357008
12.5239679813
11.8724710941
10.8379950523
#---------------------------------------------------------------------------------------------------
As you can see the first problem is that when i test it for 5 concurrent
request i get 9 entries in my log file
each in each case.
And the second but most annoying problem is for Test 3 ( for both pxtl
and dbxml ) it takes more than 10 seconds
to satisfy each request .
Even i tested my application on Upgraded Kernel (2.6.9-11.EL).
Should i use any other Linux distribution.
Can you tell me what the problem may be ? And what should i do ?
Regards,
Ajinkya N
Orr, Steve wrote:
>I can't see that mod_python would be the problem. For a given request,
>what percentage of the overall response time is involved with database
>access versus code execution? You need to know that. I'd first try
>tuning the code before doing any web stuff. Especially tune the database
>queries and the network access to the database before going through the
>web server. Then setup database connection pooling so each page request
>reuses a connection instead of establishing a new connection with each
>page turn. Can you do some kind of session management to avoid repop'ing
>"the container" with each page turn?
>
>Using ab I did some bench marking a year ago and with an old, cheap Dell
>2GHz PC with 512M RAM as the web server I was able to get 750 page turns
>per second. The database was Oracle and I used the connection pooling
>feature that comes with cx_Oracle on the OCI. (A persistent connection
>can be established for each Apache process so this isn't really
>necessary.) I also reused cursors to avoid the parsing overhead.
>(Besides doing web development I'm a DBA dweeb.) Without mod_python and
>connection pooling it got as slow as 11 requests per second. Of course
>my benchmarking was doing simple stuff but I was satisfied that
>Apache/mod_python would perform well and I wouldn't have to go down a
>"Twisted" path. Don't know anything about dbxml but you should address
>the architecture/code first. If performance is really a big issue then
>consider load balanced web servers and a good disk array setup on the db
>server.
>
>
>Steve Orr
>
>
>-----Original Message-----
>From: mod_python-bounces at modpython.org
>[mailto:mod_python-bounces at modpython.org] On Behalf Of IGP
>Sent: Wednesday, August 17, 2005 11:01 PM
>To: xml at sleepycat.com; mod_python at modpython.org
>Subject: [mod_python] Performance Problem
>
>I am accessing dbxml via my mod-python handler.
>The dbxml manager instance is created , the container is opened and the
>
>required operation
>is performed(querying , geting a document ) for each client request .
>This is affecting the
>performance . Is there any way to enhance the performance ? I mean that
>the XmlManager instance
>will be created only once , the container will be opened and the stored
>in memory , so that the
>response time will decrease .
>I am using following versions of software
>1) dbxml - 2.1.8
>2) python - 2.3.4
>3) mod-python - 1.3.4
>4) apache 2.0.52
>5) Kernel version-2.6.9-5.0.5-EL
>
>Please do reply if you know the solution .
>Thanks in Advance,
>Regards,
>Ajinkya
>
>_______________________________________________
>Mod_python mailing list
>Mod_python at modpython.org
>http://mailman.modpython.org/mailman/listinfo/mod_python
>
>
>_______________________________________________
>Mod_python mailing list
>Mod_python at modpython.org
>http://mailman.modpython.org/mailman/listinfo/mod_python
>
>
>
>
>
>
-------------- next part --------------
from mod_python import apache,util
import time
import pxtl # PXTL module
import dbxml_access # module for dbxml operations
reload(dbxml_access)
import config # configuration file
reload(config)
db_path = config.db_path # take database path location
connect = dbxml_access.DBXML_ACCESS() # create database access object
#--------------------------------------------------------------------------------------------------------
def test1(req):
"""
This function returns the document from the dbxml container to the browser
"""
try:
stime = time.time()
document_id = 'none'
The_Form = util.FieldStorage(req)
for name in The_Form.keys():
if name == 'document_id':
document_id = The_Form[name].value # get the document_id requested from the database
else:
pass
content = connect.get_document(document_id,db_path) # get the document from the dbxml container
req.content_type = "text/xml"
req.write(content)
etime = time.time()
fp = open(config.log_path + 'test1.txt','a')
fp.write('\n' + str(etime-stime))
fp.close()
except:
req.write('Error Occured ')
#----------------------------------------------------------------------------------------------------
def test2(req):
"""
This function just pxtl processes a file and writes it to browser
The file should already exist
"""
try:
stime = time.time()
document_id = 'none'
The_Form = util.FieldStorage(req)
for name in The_Form.keys():
if name == 'document_id':
document_id = The_Form[name].value # get the document_id requested from the database
else:
pass
document_file_name = config.file_path + document_id + ".px"
req.content_type = "text/xml"
stime = time.time()
pxtl.processFile(document_file_name,writer=req) # PXTL processing
etime = time.time()
fp = open(config.log_path + 'test2.txt','a')
fp.write('\n' + str(etime-stime))
fp.close()
except:
req.write('Error Occured ')
#----------------------------------------------------------------------------------------------------
def test3(req):
"""This function gets the document from the dbxml container , pxtl processes it and writes it to browser
"""
try:
stime = time.time()
document_id = 'none'
The_Form = util.FieldStorage(req)
for name in The_Form.keys():
if name == 'document_id':
document_id = The_Form[name].value # get the document_id requested from the database
else:
pass
content = connect.get_document(document_id,db_path)# get the document from the dbxml container
document_file_name = config.file_path + document_id + ".px"
fp = open(document_file_name,"w")
fp.write(content)
fp.close()
req.content_type = "text/xml"
pxtl.processFile(document_file_name,writer=req)# PXTL processing
fp.close()
etime = time.time()
fp = open(config.log_path + 'test3.txt','a')
fp.write('\n' + str(etime-stime))
fp.close()
except:
req.write('Error Occured ')
#----------------------------------------------------------------------------------------------------
|