[mod_python] invalid pointer, child process dies, broken connections

Graham Dumpleton grahamd at dscpl.com.au
Thu Nov 23 16:08:57 EST 2006


Only suggestions at this point are:

1. Use:

  PythonInterpreter main_interpreter

directive for the mod_python application in case some third party Python
module is doing strange stuff with threads but wasn't written to be able
to work properly where there are multiple Python interpreter instances.
By using that named interpreter it will force application to run within the
context of the very first interpreter created. This can help where a module
uses only the simplified GIL thread locking API.

Note that this only applies for mod_python 3.2.10 and later.

2. Read through:

  http://www.dscpl.com.au/wiki/ModPython/Articles/ExpatCausingApacheCrash

and run some of the checks talked about there to make sure this isn't being
caused by some expat library mismatch.

3. Run Apache in single process mode and attach a debugger process to the
process serving the requests (you might see 2 process, you will have to work
out which is the correct one). Then when it crashes you might be able to get
a stack trace back which might help to identify in which component it is
crashing.

  /usr/local/apache-2.0/bin/httpd -X -DONE_PROCESS

In a separate window:

graham-dumpletons-powerbook-g4-15:~ grahamd$ ps ax | grep httpd
  267  p1  S+     0:00.10 /usr/local/apache-2.2/bin/httpd -X -DONE_PROCESS
  268  p1  S+     0:00.00 /usr/local/apache-2.2/bin/httpd -X -DONE_PROCESS
  270  p2  R+     0:00.00 grep httpd

Run debugger. Usually on lowest number pid.

graham-dumpletons-powerbook-g4-15:~ grahamd$ gdb /usr/local/apache-2.2/bin/httpd -p 267
GNU gdb 6.3.50-20050815 (Apple version gdb-477) (Sun Apr 30 20:06:22 GMT 2006)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "powerpc-apple-darwin"...Reading symbols for shared libraries ....... done

/Users/grahamd/267: No such file or directory.
Attaching to program: `/usr/local/apache-2.2/bin/httpd', process 267.
Reading symbols for shared libraries .+++...................... done
0x900c7060 in sigwait ()
(gdb) 

Should see sigwait() from memory.

Type 'cont' command to gdb and then start sending requests until it crashes.
Then use 'where' to get a stack trace.

4. It would also be a good idea to simply start paring back the code of your
application bit by bit until you can come up with the most minimal bit of code
which causes the crash. In doing this, don't treat modules as black boxes, you
need to delve done into them and look at what they do, going deeper and deeper
until you get to the bottom.

5. Try the dev snapshot of mod_python 3.3. There were various crashes fixed
in that version, but these were generally odd things which probably only I
would ever think to try.

  http://mail-archives.apache.org/mod_mbox/httpd-python-dev/200611.mbox/%[email protected]%3e

6. Post just the part of your Apache configuration which relates to how you
trigger mod_python and in what context. From that I might see if you are
managing to hit any of the issues referred to in 5.

Graham

Richard Lewis wrote ..
> Hi there,
> 
> Debian unstable; apache: 2.2.3; modpython: 3.2.10; python: 2.4.4; lxml
> 1.1.1: 
> libxml: 2.6.16; libxslt: 1.1.8.
> 
> I posted on 2nd Nov about some invalid pointer errors in my Apache error
> log. 
> At the time I thought they were coming from MySQL and received some 
> interesting replies regarding the incompatibility of the mysql client 
> libraries compiled into various apache modules.
> 
> However, since then I've decided to cut MySQL out of my web application
> and, 
> instead of dynamically converting database records into XML, use 
> filesystem-cached XML files which are generated periodically from the 
> database by a separate process.
> 
> While there are a couple of other advantages to this approach, it hasn't
> solved my initial problem! I still have numerous errors like this in my
> Apache error log:
> 
> [Mon Nov 20 10:51:58 2006] [notice] child pid 7087 exit signal Aborted
> (6)
> *** glibc detected *** free(): invalid pointer: 0x0835ab26 ***
> *** glibc detected *** free(): invalid pointer: 0x0835ab26 ***
> *** glibc detected *** free(): invalid pointer: 0x0835ab26 ***
> *** glibc detected *** free(): invalid pointer: 0x0835ab26 ***
> [Mon Nov 20 10:54:28 2006] [notice] child pid 7088 exit signal Aborted
> (6)
> [Mon Nov 20 10:54:28 2006] [notice] child pid 7089 exit signal Aborted
> (6)
> [Mon Nov 20 10:54:28 2006] [notice] child pid 7098 exit signal Aborted
> (6)
> 
> It seems to handle several requests and then you get an error message from
> the 
> client (browser) saying that "the connection is broken". You can then retry
> the request and it works after one or two attempts.
> 
> I /think/ there is no direct correlation between the invalid pointer errors
> messages and the dropped connections; the times of the messages seem to
> be 
> pretty evenly distributed and don't all result in a broken connection.
> 
> Also, the memory addresses of the invalid pointers are often (but not always)
> the same. While the child process ids are always different. What is the
> child 
> process? Is it python?
> 
> I've checked to make sure that I'm always returning the content length
> and 
> Apache status code; I'm pretty sure I am! My application is using a frameset
> where the frame document and its four frames are all generated using 
> mod_python. Is there anything special I should do to help mod_python with
> this? (Like returning special Apache status codes?)
> 
> Apart from getting these errors, my code is pretty much ready to go live.
> So 
> it would be really great if anyone could help me sort out this problem!
> 
> Cheers,
> Richard
> -- 
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> Richard Lewis
> Sonic Arts Research Archive
> http://www.sara.uea.ac.uk/
> JID: ironchicken at jabber.earth.li
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> _______________________________________________
> Mod_python mailing list
> Mod_python at modpython.org
> http://mailman.modpython.org/mailman/listinfo/mod_python


More information about the Mod_python mailing list