[mod_python] OS X / Apache 2 / mod_python bug

On Monday, March 14, 2005, at 08:07  PM, Jamie Kirkpatrick wrote:
> That said, have you setup use of Apache in the main configuration file
>
> or in a .htaccess file? Can you post what configuration you are using
>
> so we can see if it is pretty normal or something strange? Post it if
>
> you can.
>
> I can post but its rather long - I am pretty darn confident that there 
> is nothing wrong with the config tho.  I havent changed barely 
> anything from the stock install, in-fact, I can reproduce the error by 
> leaving everything as is from when the install is performed and only 
> adding a python handler to the default htdocs directory.  The only 
> thing I have consistently changed is the port the server is listening 
> on to port 8080.  I have tested with the minimum of changes: added a 
> handler and dropped in a small test script.

The whole Apache config file isn't required, but just the small section 
where
your define SetHandler/AddHandler and PythonHandler. Ie., the 
appropriate
<Directory> construct covering your htdocs directory. This is just to 
get the
whole picture and clarify that it is being done within a Directory 
directive
as opposed to being at global scope in the config.

> Do you get these 500 errors if the only mod_python application 
> configured
>
> is a really basic "mptest" type test handler, or does it only happen
>
> when you go to set it up for this more complicated application?
>
>
> I used the hello.py example from the site and consistently got the 
> same results.  This was how I finally pinned it down to mod_python.  
> See the last answer for further details.

Sending a copy of the hello.py wouldn't hurt as well.

Based on your other answers, is very strange and nothing obvious stands 
out.
The next thing I would try is to localise where the 500 error actually 
gets
raised. There are four possible places it could be occurring. These are:

1. In Python code executed within the handler itself.

2. In the Python code part of mod_python. Ie., somewhere within the file
lib/python/mod_python/apache.py.

3. In the C code part of mod_python. Ie., somewhere within the file
src/mod_python.c.

4. In Apache itself somewhere prior to, but possibly after mod_python is
invoked. Lets hope not. :-)

Elimination of 1 is pretty straight forward by using a test handler 
which
contains:

     from mod_python import apache
     apache.log_error("start import")

     def handler(req):
         apache.log_error("start handler")
         req.content_type = "text/plain"
         req.send_http_header()
         req.write("Hello World!")
         apache.log_error("end handler")
         return apache.OK

     apache.log_error("end import")

If the 500 error is not generated from within the import of the handler
module or the handler itself when called, you should see in the Apache
error log all of the messages above.

BTW, as an extra precaution, if Apache is able to write to your document
directories, you may want to manually remove any ".pyc" files that may
have been generated as the result of a prior test, before running the 
test.

To test out 2, one would need to modify lib/python/mod_python/apache.py
and reinstall it. You should add calls to apache.log_error() into 
apache.py
to track where it may get to within its execution.

I think the appropriate place to place log statements in that file is at
the start of the class method:

   def HandlerDispatch(self, req):

     log_error("start dispatch")

Prior to the final return of the method, add:

     log_error("end dispatch %s"%str(result))

There is a return part way through the method which returns a 500 error,
but it is preceded by logging of a message anyway.

If it is found that 500 is being returned from the method, this would
localise the cause of the problem to this method. At this point we would
have to start analysing further the method and the potential sources of
the 500 error by adding more logging and checking where result is being
set to 500.

If this method always returns a 200 status response, yet the browser
still sees 500, we would have to consider number 3 above.

At this point things get a bit trickier as one is dealing with the C 
code.
It is worth noting though that most instances of where a 500 response 
are
generated in the mod_python.c file are preceded by message logging. The
only ones that aren't are all of the form:

     /* get/create interpreter */
     idata = get_interpreter(interp_name, req->server);

     if (!idata)
         return HTTP_INTERNAL_SERVER_ERROR;

Not being able to get/create the interpreter is pretty severe and if 
that
was happening then no Python code at all would be getting executed. I am
not sure why this situation couldn't be logged as well. Thus instead of
that above, have:

     if (!idata) {
         ap_log_rerror(APLOG_MARK, APLOG_NOERRNO|APLOG_ERR, 0, req,
                       "python_handler: Can't get/create interpreter.");

         return HTTP_INTERNAL_SERVER_ERROR;
     }

Anyway, that is the steps I would personally pursue to try and narrow 
down
where the 500 error is being generated from. Once one can reliably 
pinpoint
it down to a specific area of code in this way, have a better chance of
determining why it is happening.

Graham