[mod_python] streaming zipped data with publisher]

Mike Looijmans nlv11281 at natlab.research.philips.com
Mon Jan 9 04:51:53 EST 2006

Daniel Nogradi wrote:
> Is it possible to "stream" zipped data using the publisher handler?

Sure. I'm streaming multi-gigabyte TAR archives to/from tape and other
storage this way.

The ZIPfile and TARfile units are quite alike.

In my case, using a temporary file is not an option - the archives may
span multiple 400GB (no, that's not a typo) tapes.

When streaming out the data, set the 'Transfer-Encoding' header to
"chunked". This is neccesary because you don't know forehand how much
data you'll be sending. Output the data in chunks, by writing the (hex)
size of the chunk, followed by \r\n and the data, followed by \r\n.

This code snippet copies data from the 'stream' to 'wfile' using chunked

             while 1:
                 data = stream.read()
                 if not data:
                 # chunk header is <hex-size>\r\n
                 l = len(data)
                 wfile.write("%x\r\n" % l)
                 # output data
                 # chunk footer
             # write chunk terminator

All that's left to do is grab the output from the Zipfile writer and
send it to the chunk encoder. You can probably directly replace


Mike Looijmans
Philips Natlab / Topic Automation

More information about the Mod_python mailing list