[mod_python] Urllib2 called from mod_pyton is 10x slower then called from python

Doug Gray dgray at coldstorage.com
Mon Apr 26 15:03:22 EST 2004

Thanks for the answer to my previous question, I was able to work with
urllib2 for logging into a site and grabbing webpages on that site. I found
ClientCookies 3rd party package that handles my cookie tracking and setting
in each webpage header seamlessly!


NOW, I have one other problem. Im still getting a "timeout" issue when
running my test program through my webserver.


I can run the little program which is essentially this...


Import urllib2

request =

resp = urllib2.urlopen(request)

print resp.info()



the above simple program if run in python2.3 runs in about 2-3 seconds but
the same program called from a mod python program takes 20 seconds and fails
90% of the time giving a very long trace...


Mod_python error: "PythonHandler mod_python.publisher"
Traceback (most recent call last):
  File "/usr/lib/python2.3/site-packages/mod_python/apache.py", line 299, in
    result = object(req)
  File "/usr/lib/python2.3/site-packages/mod_python/publisher.py", line 136,
in handler
    result = util.apply_fs_data(object, req.form, req=req)
  File "/usr/lib/python2.3/site-packages/mod_python/util.py", line 361, in
    return object(**args)
  File "/usr/local/apach2/fantasy/TSN/htmlRead.py", line 31, in getwebpage
    response = ClientCookie.urlopen(request)
  File "/usr/lib/python2.3/site-packages/ClientCookie/_urllib2_support.py",
line 829, in urlopen
    return _opener.open(url, data)
  File "/usr/lib/python2.3/site-packages/ClientCookie/_urllib2_support.py",
line 520, in open
    response = urllib2.OpenerDirector.open(self, req, data)
  File "/var/tmp/python2.3-2.3.3-root/usr/lib/python2.3/urllib2.py", line
326, in open
    '_open', req)
  File "/var/tmp/python2.3-2.3.3-root/usr/lib/python2.3/urllib2.py", line
306, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.3/site-packages/ClientCookie/_urllib2_support.py",
line 754, in http_open
    return self.do_open(httplib.HTTP, req)
  File "/usr/lib/python2.3/site-packages/ClientCookie/_urllib2_support.py",
line 612, in do_open
    raise URLError(err)


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.modpython.org/pipermail/mod_python/attachments/20040426/c6d8f462/attachment.html

More information about the Mod_python mailing list