|
Graham Dumpleton
grahamd at dscpl.com.au
Thu Aug 25 02:19:28 EDT 2005
Why are you doing:
reload(pxtl)
What is it about "import" that isn't enough?
M Chaudhary wrote ..
>
> Hello friends,
>
> 1) When I perform just file io it takes 0.00026 seconds.
> import time
> fp = open("xyz.html","r")
> s = fp.read()
> fp.close()
> stime = time.time()
> fp = open('xyz.px','w')
> fp.write(s)
> fp.close()
> etime = time.time()
> print etime-stime
>
>
> 2) When I perform just pxtl processing it takes 0.064 seconds.
>
> import time
> stime = time.time()
> import pxtl
> reload(pxtl)
> pxtl.processFile('xyz.px')
> etime = time.time()
> print etime-stime
>
> 3) When I perform pxtl processing along with file-io it takes 1.33 seconds.
> import time
> fp = open("xyz.html","r")
> s = fp.read()
> fp.close()
> stime = time.time()
> fp = open('xyz.px','w')
> fp.write(s)
> fp.close()
> import pxtl
> reload(pxtl)
> pxtl.processFile('xyz.px')
> etime = time.time()
> print etime-stime
>
> As you can see there is a major difference in 1+2 and 3 .
> Can you please tell me what the problem might be and what should i do ?
> I am using PXTL for developing a website along with file io as PXTL does
> not support string processing .
> Thanks in Advance.
> Regards,
> Ajinkya
>
> Orr, Steve wrote:
>
> >Well obviously mod_python is performing well and I doubt anything
> >connected with Linux versions would account for that much difference so
> >the thing to focus on would be tuning dbxml or the code associated with
> >it. Don't you have a way to run/test chunks of code outside of a web
> >environment?
> >
> >-----Original Message-----
> >From: M Chaudhary [mailto:igp-dev at infogridpacific.com]
> >Sent: Tuesday, August 23, 2005 12:13 AM
> >To: mod_python at modpython.org
> >Cc: Orr, Steve; IGP; and at doxdesk.com; xml at sleepycat.com
> >Subject: Re: [mod_python] Performance Problem
> >
> >Hello Orr, Steve ,
> > As you told i tested my application using "ab" tool for three
> >cases (each for single request and 5 concurrent requests ):-
> >and the results were as follows (the following log was generated from
> my
> >
> >mod-python handler itself using time.time() method ) ----
> >Attached is the test module . Please check it and view the following
> >results.
> >#-----------------------------------------------------------------------
> >----------------------------
> ># Test 1 -- dbxml :- get a document and write it to browser
> >#-----------------------------------------------------------------------
> >----------------------------
> >0.00605797767639 # single request
> >0.0058479309082
> >0.00573110580444
> >0.148849010468
> >0.00549292564392
> >0.00417685508728 # 5 concurrent requests
> >0.00559282302856
> >0.00551891326904
> >0.00600981712341
> >0.00549983978271
> >#-----------------------------------------------------------------------
> >----------------------------
> ># Test 2 -- pxtl :- pxtl process a File and write it to browser
> >#-----------------------------------------------------------------------
> >----------------------------
> >1.65334200859 # single request
> >0.25177192688
> >0.261835098267
> >0.294239997864
> >0.000832080841064
> >0.428733825684 # 5 concurrent requests
> >0.440675020218
> >0.269226074219
> >0.237013816833
> >0.13219499588
> >#-----------------------------------------------------------------------
> >----------------------------
> ># Test 3 -- pxtl and dbxml :- get a document from dbxml container ,pxtl
> >
> >process it and write it to browser
> >#-----------------------------------------------------------------------
> >----------------------------
> >3.01918792725 # single request
> >13.4880371094
> >14.6321718693
> >15.9680738449 # 5 concurrent requests
> >16.1954669952
> >0.010479927063
> >17.400357008
> >12.5239679813
> >11.8724710941
> >10.8379950523
> >#-----------------------------------------------------------------------
> >----------------------------
> >
> >As you can see the first problem is that when i test it for 5 concurrent
> >
> >request i get 9 entries in my log file
> >each in each case.
> >And the second but most annoying problem is for Test 3 ( for both pxtl
> >and dbxml ) it takes more than 10 seconds
> >to satisfy each request .
> >Even i tested my application on Upgraded Kernel (2.6.9-11.EL).
> >Should i use any other Linux distribution.
> >Can you tell me what the problem may be ? And what should i do ?
> >
> >Regards,
> >Ajinkya N
> >
> >
> >Orr, Steve wrote:
> >
> >
> >
> >>I can't see that mod_python would be the problem. For a given request,
> >>what percentage of the overall response time is involved with database
> >>access versus code execution? You need to know that. I'd first try
> >>tuning the code before doing any web stuff. Especially tune the
> >>
> >>
> >database
> >
> >
> >>queries and the network access to the database before going through the
> >>web server. Then setup database connection pooling so each page request
> >>reuses a connection instead of establishing a new connection with each
> >>page turn. Can you do some kind of session management to avoid
> >>
> >>
> >repop'ing
> >
> >
> >>"the container" with each page turn?
> >>
> >>Using ab I did some bench marking a year ago and with an old, cheap
> >>
> >>
> >Dell
> >
> >
> >>2GHz PC with 512M RAM as the web server I was able to get 750 page
> >>
> >>
> >turns
> >
> >
> >>per second. The database was Oracle and I used the connection pooling
> >>feature that comes with cx_Oracle on the OCI. (A persistent connection
> >>can be established for each Apache process so this isn't really
> >>necessary.) I also reused cursors to avoid the parsing overhead.
> >>(Besides doing web development I'm a DBA dweeb.) Without mod_python and
> >>connection pooling it got as slow as 11 requests per second. Of course
> >>my benchmarking was doing simple stuff but I was satisfied that
> >>Apache/mod_python would perform well and I wouldn't have to go down a
> >>"Twisted" path. Don't know anything about dbxml but you should address
> >>the architecture/code first. If performance is really a big issue then
> >>consider load balanced web servers and a good disk array setup on the
> >>
> >>
> >db
> >
> >
> >>server.
> >>
> >>
> >>Steve Orr
> >>
> >>
> >>-----Original Message-----
> >>From: mod_python-bounces at modpython.org
> >>[mailto:mod_python-bounces at modpython.org] On Behalf Of IGP
> >>Sent: Wednesday, August 17, 2005 11:01 PM
> >>To: xml at sleepycat.com; mod_python at modpython.org
> >>Subject: [mod_python] Performance Problem
> >>
> >>I am accessing dbxml via my mod-python handler.
> >>The dbxml manager instance is created , the container is opened and
> >>
> >>
> >the
> >
> >
> >>required operation
> >>is performed(querying , geting a document ) for each client request .
> >>This is affecting the
> >>performance . Is there any way to enhance the performance ? I mean that
> >>
> >>
> >
> >
> >
> >>the XmlManager instance
> >>will be created only once , the container will be opened and the stored
> >>
> >>
> >
> >
> >
> >>in memory , so that the
> >>response time will decrease .
> >>I am using following versions of software
> >>1) dbxml - 2.1.8
> >>2) python - 2.3.4
> >>3) mod-python - 1.3.4
> >>4) apache 2.0.52
> >>5) Kernel version-2.6.9-5.0.5-EL
> >>
> >>Please do reply if you know the solution .
> >>Thanks in Advance,
> >>Regards,
> >>Ajinkya
> >>
> >>_______________________________________________
> >>Mod_python mailing list
> >>Mod_python at modpython.org
> >>http://mailman.modpython.org/mailman/listinfo/mod_python
> >>
> >>
> >>_______________________________________________
> >>Mod_python mailing list
> >>Mod_python at modpython.org
> >>http://mailman.modpython.org/mailman/listinfo/mod_python
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >
> >_______________________________________________
> >Mod_python mailing list
> >Mod_python at modpython.org
> >http://mailman.modpython.org/mailman/listinfo/mod_python
> >
> >
> >
> >
> >
> >
> _______________________________________________
> Mod_python mailing list
> Mod_python at modpython.org
> http://mailman.modpython.org/mailman/listinfo/mod_python
|