[mod_python] problem with input filter and POST data (fileupload)

2009/2/15 Joerg <jerch at rockborn.de>:
> Hi,
>
> I ve encountered a problem with a POST request while a mod_python input filter
> is running (on ubuntu 8.04 with apache 2.2.8 prefork and mod_python 3.3.1).
>
> the situation:
> Imagine the simplest input filter as stated in the docs and try to upload a
> file (some big POST data) thru this filter. This file will be trunceated to
> the first 64k.
>
> i have tracked down the problem to fragmented data handling in the file
> util.py. If i change line 363 in that file from
>>>> req.readline(readBlockSize)
> to
>>>> req.readline()
> it works again. so i dumped the stream data here to see wots going on
> with the data with 4 different setups:
>
> 1) req.readline() without filter
> 2) req.readline() with filter
> 3) req.readline(readBlockSize) without filter
> 4) req.readline(readBlockSize) with filter
>
> 1-3 seem to be exactly the same, while in the 4th case the req-stream is empty
> after some readings (then req.readline(readBlockSize) becomes None and the
> method returns as expected).
> I dont know why suddenly the stream is empty long before the boundery (or
> readline() thinks so), could this be some delay effect caused by the filter?
> or is this a special behavior by req.readline() itself with a given length?
>
> If this is not easy to patch, will there be any neg. effect of using
> req.readline() instead of req.readline(readBlockSize) aside from the big file
> problem?

Behaviour explained through failing to check for 'None' on data read
by input filter before closing it.

  from mod_python import apache

  def outputfilter(filter):

      s = filter.read()
      while s:
          filter.write(s.upper())
          s = filter.read()

      if s is None:
          filter.close()

In other words, forget the 'if s is None' and you will see that occur
as input filter has been prematurely closed.

Graham