nlv11281 at natlab.research.philips.com
Tue Feb 14 01:24:05 EST 2006
Mike Looijmans Philips Natlab / Topic Automation Daniel Nogradi wrote: > The unpacking itself doesn't take that long, but until the unpacked files reach their final destination (directories, databases, etc) will take quite some time as they have to be processed in some way. But I agree, uploading takes much longer, but I would like to make the waiting time for the users as short as possible, and in principle the unpacking/processing/etc can be done after they receive an acknowledgement page. > And like all users in the world, they'll go to some status page and hit the "refresh" button 5 times a second to see if it's there yet. >>You might consider using the latest mod_python 3.2.7 modification that >>supports file "streaming". You can unpack the ZIP while it's being >>uploaded. This also prevents using TEMP space. And when the upload is >>ready, the files are there as well. I tested this method for uploading >>multi-gigabyte TAR files. ... > Thanks a lot for the code, I didn't even think about such an approach, this would be the best of course. It would probably also be the fastest method performance wise (with my big tar files, untar-as-you-go was about 2x to 3x faster than upload and untar in separate steps) in the end, your users will be waiting much shorter, and once they get the acknowledgement page, the work has really been done. Forking off processes from a HTTP deamon just does not sound "right" to me. It will also prevent your program from ever working on a non-unix platform. But that's more a religious thing I guess...