[mod_python] problem with input filter and POST data (fileupload)

Graham Dumpleton graham.dumpleton at gmail.com
Sun Feb 15 22:31:06 EST 2009

2009/2/16 Joerg <jerch at rockborn.de>:
> Am Monday 16 February 2009 02:42:32 schrieben Sie:
>> 2009/2/16 Joerg <jerch at rockborn.de>:
>> > hello,
>> >
>> > ive dumped a fileupload (original size = 231KB) with the following
>> > results:
>> >
>> > without filter:
>> >        - req.read() --> 231KB
>> >        - req.readline() --> 231KB
>> >        - req.readline(64000) --> 231KB
>> >        - via util.py --> 231KB
>> >
>> > with filter:
>> >        - req.read() --> 231KB
>> >        - req.readline() --> 231KB
>> >        - req.readline(64000) --> 64KB
>> What about:
>>   s = req.readline(64000)
>>   while not s:
>>     ...
>>     # log some debug ....
>>     s = req.readline(64000)
>> Does that result in all data being read, or is that what your test was
>> doing?
> hm, dont know wot u mean with the while not clause, the handler will loop here
> for ever.

Yeah, me just getting my logic around the wrong way.

> ive inspected the data stream of a indexed textfile to get the break point.
> wot ive found out:
> if u set the size of req.readline(size) pretty low the stream breaks after
> 103888 Bytes. til there the readline works correct, it "resumes" the old line
> data and so on. all data after 103888 is lost.
> if u set the size greater than this border, only the first line read will
> succeed. the other data is lost.
> btw, this break point is in the middle of a filter call, not at the end (was
> the 6th here with every call working on 1,4KB of stream data)
> since this weird behavior shows up only with a running filter (dont know, if
> non python input filter are affected too), this is probably caused by some
> code in the c part, which is only active with a filter before. also it might
> be some buffer issue there, since the 103888 Byte border is reproducible.

Frankly I would not be surprised if how mod_python implements the
filters is broken.

My question at this point is, what are you trying to implement that
uses filters anyway. Writing filters in mod_python isn't actually a
wise thing as far as I am concerned as it is not going to be
particularly efficient. Much more sensible to write a proper Apache C
module to implement the filter.

The only serious use of mod_python filters I have seen is Tramline
(upload/download accelerator). They actually complain the mod_python
filters don't work properly as well and say a fix has to be made to
mod_python Python code. The fix itself they propose doesn't make any
sense, unless it is acting as a workaround for borked C code
underneath. The issue for that is at:


Anyway, if you are still having problems, not sure there is much more
I can do to help. I continue to only answer questions here, but I
don't actually use mod_python myself anymore nor even start it up to
try things out when people have issues and just rely on my memory of
what happens.

If you can track the problem down to a specific area of code with
small reproducable recipe to trigger, then log a bug report in the
issue tracker. It can go with all the other bugs that haven't been
fixed. :-)


More information about the Mod_python mailing list