[mod_python] strange behaviour with compressed response

Graham Dumpleton grahamd at dscpl.com.au
Tue Jan 2 22:07:55 EST 2007


Apologies if this comes through twice on the list, having problems with my
site running reaaaaaalllllllllyyyyyyyy slow and don't think if got through last
time. Graham

m.banaouas wrote ..
> hi,
> what is the difference between serving a file directly or by calling a
> script function?
> case 1: http://localhost/myfile.xml
> case 2: http://localhost/mydir/boo/getfile
> Must I do special call to apache api when compression is activated?
> With Firefox, every thing works fine in two cases.
> With a http client component I'm using in my developpements, both case
> 1 
> and case 2 works fine if I don't ask for compressed data. If I do 
> (Accept-Encoding: gzip), case 1 works fine but case 2 fails.

Your Apache configuration only enables the DEFLATE output filter for static
files in your htdocs directory, it doesn't enable it for anything under mydir
so unless DEFLATE is being enable somehow elsewhere in the Apache configuration
the output of your published functions would never be compressed.

See other notes further done mixed in with your code and configuration.

> I noticed that returning apache.OK adds unexpected extra "0" on the output.

Which would be what one expects as functions published using
mod_python.publisher are not the same as basic mod_python handlers. When using
mod_python.publisher the result of the published function is converted to a
string and written back as response content. In a basic mod_python handler the
response is a status indicating success of otherwise of the handler. Look at
the documentation again to see the difference.

> thanks.
> 
> DocumentRoot "C:/Apache/htdocs"
> <Directory />
>     Options FollowSymLinks
>     AllowOverride None
>     Order deny,allow
>     Deny from all
>     Satisfy all
>     SetOutputFilter DEFLATE
> </Directory>
> 
> Alias /mydir "C:/MYDIR"
> <Directory C:/MYDIR>
>     Allow from All
>     SetHandler mod_python
>     PythonHandler mod_python.publisher

Because this is a separate physical directory with distinct Directory
directive, you would also need here:

      SetOutputFilter DEFLATE

to have response from published functions be compressed.

Do take heed of the Apache documentation I pointed you to originally in
regard to DEFLATE module and a possible need to ensure you don't enable
it for some clients because of broken compressed data implementations
in those clients. Also, you shouldn't enable DEFLATE for all file types as files
such as images are usually already compressed and compressing them
again is a pointless exercise.

Another alternative with mod_python 3.3 is to actually make the judgement
within the published function as to whether for a specific response the output
should be compressed or not. To do this, before writing back any content
from the function, call:

    req.add_output_filter('DEFLATE')

This is the same as having using 'SetOutputFilter' in the Apache configuration.

> </Directory>
> 
> here is the simple boo.py script:
> #---------------------------
> # -*- coding: ISO-8859-1 -*-
> #
> # boo.py
> #
> import os, sys
> from mod_python import apache
> #
> def getfile(req):
>   req.content_type = 'text/xml'
>   req.send_http_header()
>   #
>   f = file('c:/apache/htdocs/myfile.xml','r')
>   xmldata = f.read()
>   f.close()
>   req.write(xmldata)

This is inefficient. Read the documentation and instead simply try:

  req.sendfile('c:/apache/htdocs/myfile.xml')

What I don't know off hand though is if the file contents still go through
the output filters. I have a feeling they don't as it is actually dropping
down and writing it direct out on the socket for efficiency.

>   #return apache.OK

Should not return apache.OK in published functions.

> #
> def index(req):
>   return "We are in index()"
> #---------------------------
> 
> and here three traces from a tcp viwer:
> 
> #--MyHttpClient/func_access/without_gzip
> Resolving Remote Host
> Remote Host resolved to 127.0.0.1
> Local Port (8080) opened
> Waiting for connections
> 400: Client connected; 127.0.0.1:4657
> 400: Connecting to Server
> 400: Client to Server (188 bytes)
> GET /mydir/boo/getfile HTTP/1.0
> Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
> Connection: keep-alive
> User-Agent: Mozilla/4.0 (compatible; ICS)
> Host: localhost:80
> 
> 400: Connected to Server
> 400: Server to Client (255 bytes)
> HTTP/1.1 200 OK
> Date: Wed, 03 Jan 2007 00:26:55 GMT
> Server: Apache/2.2.3 (Win32) mod_python/3.3.0b Python/2.4.4
> Vary: Accept-Encoding
> Connection: close
> Content-Type: text/xml
> 
> <?xml version="1.0" encoding="ISO-8859-1"?>
> <document>Hello</document>
> 
> 400: Server disconnected
> 400: Disconnected from Client
> 
> #-MyHttpClient/func_access/with_gzip
> Resolving Remote Host
> Remote Host resolved to 127.0.0.1
> Local Port (8080) opened
> Waiting for connections
> 488: Client connected; 127.0.0.1:4665
> 488: Connecting to Server
> 488: Client to Server (211 bytes)
> GET /mydir/boo/getfile HTTP/1.0
> Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
> Connection: keep-alive
> Accept-Encoding: gzip
> User-Agent: Mozilla/4.0 (compatible; ICS)
> Host: localhost:80
> 
> 488: Connected to Server
> 488: Server to Client (217 bytes)
> HTTP/1.1 200 OK
> Date: Wed, 03 Jan 2007 00:29:42 GMT
> Server: Apache/2.2.3 (Win32) mod_python/3.3.0b Python/2.4.4
> Vary: Accept-Encoding
> Content-Encoding: gzip
> Connection: close
> Content-Type: text/xml
> 
> 488: Server to Client (74 bytes)
> 0000  B3 B1 AF C8 CD 51 28 4B  2D 2A CE CC CF B3 55 32  .....Q(K-*....U2
> 0010  D4 33 50 52 48 CD 4B CE  4F C9 CC 4B B7 55 F2 0C  .3PRH.K.O..K.U..
> 0020  F6 D7 B5 B0 30 B5 D4 35  54 B2 B7 E3 B2 49 C9 4F  ....0..5T....I.O
> 0030  2E CD 4D CD 2B B1 F3 48  CD C9 C9 B7 D1 87 F3 B9  ..M.+..H........
> 0040  B8 00 86 D4 95 DA 48 00  00 00                    ......H...    
> 488: Server disconnected
> 488: Disconnected from Client
> 
> #--MyHttpClient/file_access/with_gzip
> Resolving Remote Host
> Remote Host resolved to 127.0.0.1
> Local Port (8080) opened
> Waiting for connections
> 428: Client connected; 127.0.0.1:4674
> 428: Connecting to Server
> 428: Connected to Server
> 428: Client to Server (204 bytes)
> GET /myfile.xml HTTP/1.0
> Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
> Connection: keep-alive
> Accept-Encoding: gzip
> User-Agent: Mozilla/4.0 (compatible; ICS)
> Host: localhost:80
> 
> 428: Server to Client (452 bytes)
> HTTP/1.1 200 OK
> Date: Wed, 03 Jan 2007 00:37:41 GMT
> Server: Apache/2.2.3 (Win32) mod_python/3.3.0b Python/2.4.4
> Last-Modified: Tue, 02 Jan 2007 23:30:43 GMT
> ETag: "f832-4b-1e6eace9"
> Accept-Ranges: bytes
> Vary: Accept-Encoding
> Content-Encoding: gzip
> Content-Length: 87
> Keep-Alive: timeout=5, max=100
> Connection: Keep-Alive
> Content-Type: application/xml
> 
> 0187  D4 33 50 52 48 CD 4B CE  4F C9 CC 4B B7 55 F2 0C  .3PRH.K.O..K.U..
> 0197  F6 D7 B5 B0 30 B5 D4 35  54 B2 B7 E3 E5 B2 49 C9  ....0..5T.....I.
> 01A7  4F 2E CD 4D CD 2B B1 F3  48 CD C9 C9 B7 D1 87 F3  O..M.+..H.......
> 01B7  79 B9 78 B9 00 CB E5 71  9D 4B 00 00 00           y.x....q.K... 
> 428: Server disconnected
> 428: Disconnected from Client
> 
> #--Firefox--------------------------
> Resolving Remote Host
> Remote Host resolved to 127.0.0.1
> Local Port (8080) opened
> Waiting for connections
> 512: Client connected; 127.0.0.1:4669
> 512: Connecting to Server
> 512: Connected to Server
> 512: Client to Server (439 bytes)
> GET /mydir/boo/getfile HTTP/1.1
> Host: localhost:80
> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.1) 
> Gecko/20061204 Firefox/2.0.0.1
> Accept: 
> text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
> Accept-Language: fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3
> Accept-Encoding: gzip,deflate
> Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
> Keep-Alive: 300
> Connection: keep-alive
> 
> 512: Server to Client (287 bytes)
> HTTP/1.1 200 OK
> Date: Wed, 03 Jan 2007 00:32:06 GMT
> Server: Apache/2.2.3 (Win32) mod_python/3.3.0b Python/2.4.4
> Vary: Accept-Encoding
> Content-Encoding: gzip
> Keep-Alive: timeout=5, max=100
> Connection: Keep-Alive
> Transfer-Encoding: chunked
> Content-Type: text/xml
> 
> a
> 512: Server to Client (85 bytes)
> 0000  34 61 0D 0A B3 B1 AF C8  CD 51 28 4B 2D 2A CE CC  4a.......Q(K-*..
> 0010  CF B3 55 32 D4 33 50 52  48 CD 4B CE 4F C9 CC 4B  ..U2.3PRH.K.O..K
> 0020  B7 55 F2 0C F6 D7 B5 B0  30 B5 D4 35 54 B2 B7 E3  .U......0..5T...
> 0030  B2 49 C9 4F 2E CD 4D CD  2B B1 F3 48 CD C9 C9 B7  .I.O..M.+..H....
> 0040  D1 87 F3 B9 B8 00 86 D4  95 DA 48 00 00 00 0D 0A  ..........H.....
> 0050  30 0D 0A 0D 0A                                    0....         
> 512: Server disconnected
> 512: Disconnected from Client
> 
> 
> _______________________________________________
> Mod_python mailing list
> Mod_python at modpython.org
> http://mailman.modpython.org/mailman/listinfo/mod_python


More information about the Mod_python mailing list