[mod_python] Due to design of code util.FieldStorage causes MemoryError on V.Large files

Just to through some information out there...

This is the function that reads the "uploaded" file and writes to disk.

     def read_to_boundary(self, req, boundary, file):
         delim = ""
         line = req.readline()
         sline = line.strip()
         last_bound = boundary + "--"
         while line and sline != boundary and sline != last_bound:
             odelim = delim
             if line[-2:] == "\r\n":
                 delim = "\r\n"
                 line = line[:-2]
             elif line[-1:] == "\n":
                 delim = "\n"
                 line = line[:-1]
             file.write(odelim + line)
             line = req.readline()
             sline = line.strip()

What about if there are no "\n" delimiters in the file and the file  
is 512MB?

I know that someone already suggested instead of readline() to do  
sized read(MAX_SIZE) ... Is this already implemented in mod_python 3.2?


/amn

On Nov 3, 2005, at 11:34 AM, Alexis Marrero wrote:

> Barry,
>
> I'm also having a MemoryError on line:
>
>   File "/Library/Frameworks/Python.framework/Versions/2.4/lib/ 
> python2.4/site-packages/mod_python/util.py", line 175, in __init__
>     self.read_to_boundary(req, boundary, file)
>
>   File "/Library/Frameworks/Python.framework/Versions/2.4/lib/ 
> python2.4/site-packages/mod_python/util.py", line 206, in  
> read_to_boundary
>     sline = line.strip()
>
> MemoryError
>
>
> The file that is uploaded is 550MB.
>
> BTW, the Content-Disposition header has a filename attribute.
>
> What is the status with the changes that you mention on doing?
>
> My web application must be able to support attachments of arbitrary  
> size.
>
> Any comments?
>
> /amn
> _______________________________________________
> Mod_python mailing list
> Mod_python at modpython.org
> http://mailman.modpython.org/mailman/listinfo/mod_python
>