[mod_python] streaming zipped data with publisher]

Daniel Nogradi wrote:
> Is it possible to "stream" zipped data using the publisher handler?

Sure. I'm streaming multi-gigabyte TAR archives to/from tape and other
storage this way.

The ZIPfile and TARfile units are quite alike.

In my case, using a temporary file is not an option - the archives may
span multiple 400GB (no, that's not a typo) tapes.

When streaming out the data, set the 'Transfer-Encoding' header to
"chunked". This is neccesary because you don't know forehand how much
data you'll be sending. Output the data in chunks, by writing the (hex)
size of the chunk, followed by \r\n and the data, followed by \r\n.

This code snippet copies data from the 'stream' to 'wfile' using chunked
  encoding:

             while 1:
                 data = stream.read()
                 if not data:
                     break
                 # chunk header is <hex-size>\r\n
                 l = len(data)
                 wfile.write("%x\r\n" % l)
                 # output data
                 wfile.write(data)
                 # chunk footer
                 wfile.write("\r\n")
             # write chunk terminator
             wfile.write("0\r\n\r\n")

All that's left to do is grab the output from the Zipfile writer and
send it to the chunk encoder. You can probably directly replace
zipfile.write().

Mike.


-- 
Mike Looijmans
Philips Natlab / Topic Automation