[mod_python] Re: Upgrading expat for apache question

wayne collier Wayne.Collier at noaa.gov
Mon Mar 22 14:20:58 EDT 2010


Okay Graham.  I ran the "thread apply all bt" that dicuss in the link 
and got the following:

(gdb) thread apply all bt

Thread 1 (Thread 0x135700 (LWP 28732)):
#0  0x001ef402 in __kernel_vsyscall ()
#1  0x00c735bb in semop () from /lib/libc.so.6
#2  0x0012420a in proc_mutex_sysv_acquire (mutex=0x134934)
    at locks/unix/proc_mutex.c:226
#3  0x00123242 in apr_proc_mutex_lock (mutex=0x9123098)
    at locks/unix/proc_mutex.c:878
---Type <return> to continue, or q <return> to quit---
#4  0x081ac7a0 in child_main (child_num_arg=<value optimized out>)
    at prefork.c:205
#5  0x081acb77 in make_child (s=0x909d9a8, slot=12) at prefork.c:746
#6  0x081ad4d0 in ap_mpm_run (_pconf=0x90960a8, plog=0x90d41a0, s=0x909d9a8)
    at prefork.c:881
#7  0x08088945 in main (argc=151601312, argv=0x91d5338) at main.c:740
(gdb)                                                    


*The line 740 and 741 of  main.c is as follows:
*
740      if (ap_mpm_run(pconf, plog, server_conf))
741            break;

*Here are the lines from prefolk.c:*

881                make_child(ap_server_conf, free_slots[i]);
746        child_main(slot);
205    apr_status_t rv = apr_proc_mutex_lock(accept_mutex);


* Here are the lines from locks/unix/proc_mutex.c :*

876  APR_DECLARE(apr_status_t) apr_proc_mutex_lock(apr_proc_mutex_t *mutex)
877  {
*878      return mutex->meth->acquire(mutex);    #<=====line 878*
879  }


static apr_status_t proc_mutex_sysv_acquire(apr_proc_mutex_t *mutex)
{
    int rc;

    do {
        rc = semop(mutex->interproc->filedes, &proc_mutex_op_on, 
1);       *#<=====line 226*
    } while (rc < 0 && errno == EINTR);
    if (rc < 0) {
        return errno;
    }
    mutex->curr_locked = 1;
    return APR_SUCCESS;
}


Can you detect what might be an issue from?

Wayne

Graham Dumpleton wrote:
> On 19 March 2010 07:48, wayne collier <Wayne.Collier at noaa.gov> wrote:
>   
>> Graham -my understanding of RH apache  is  modules have to   explicitly
>> loaded  in httpd.con file.  I check all the snippet  files  and  no  php
>> shared object file is being loaded.  I noticed before that did not seem to
>> have any mod_authz_ldap shared object loaded in my configuration file. Do I
>> need to set things up like i am doing authorization and authentication thru
>> apache?  Is possible that needs to be included?
>>
>> Also, I went thru some of the troubleshooting steps. I ran my web code to
>> reproduce the seg fault and then gbc to attach the process id.  I got the
>> following before getting really lost here. Any suggestions here would
>> greatly appreciated. Thanks.
>>
>> Wayne
>>
>> GBD WORK .....
>>
>> Detaching from program: /usr/local/httpd-2.2.11/bin/httpd, process 18421
>> [root at mingus psp]# gdb /usr/local/httpd-2.2.11/bin/httpd 18421
>> GNU gdb Fedora (6.8-37.el5)
>> Copyright (C) 2008 Free Software Foundation, Inc.
>> License GPLv3+: GNU GPL version 3 or later
>> <http://gnu.org/licenses/gpl.html>
>> This is free software: you are free to change and redistribute it.
>> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
>> and "show warranty" for details.
>> This GDB was configured as "i386-redhat-linux-gnu"...
>> Attaching to program: /usr/local/httpd-2.2.11/bin/httpd, process 18421
>>
>> warning: .dynamic section for "/usr/lib/libgssapi_krb5.so.2" is not at the
>> expected address
>>
>> warning: difference appears to be caused by prelink, adjusting expectations
>>
>> warning: .dynamic section for "/usr/lib/libkrb5.so.3" is not at the expected
>> address
>>
>> warning: difference appears to be caused by prelink, adjusting expectations
>>
>> warning: .dynamic section for "/lib/libcom_err.so.2" is not at the expected
>> address
>>
>> warning: difference appears to be caused by prelink, adjusting expectations
>>
>> warning: .dynamic section for "/usr/lib/libk5crypto.so.3" is not at the
>> expected address
>>
>> warning: difference appears to be caused by prelink, adjusting expectations
>>
>> warning: .dynamic section for "/lib/libresolv.so.2" is not at the expected
>> address
>>
>> warning: difference appears to be caused by prelink, adjusting expectations
>>
>> warning: .dynamic section for "/usr/lib/libkrb5support.so.0" is not at the
>> expected address
>>
>> warning: difference appears to be caused by prelink, adjusting expectations
>>
>> warning: .dynamic section for "/lib/libkeyutils.so.1" is not at the expected
>> address
>>
>> warning: difference appears to be caused by prelink, adjusting expectations
>>
>> warning: .dynamic section for "/lib/libselinux.so.1" is not at the expected
>> address
>>
>> warning: difference appears to be caused by prelink, adjusting expectations
>>
>> warning: .dynamic section for "/lib/libsepol.so.1" is not at the expected
>> address
>>
>> warning: difference appears to be caused by prelink, adjusting expectations
>>
>> warning: .dynamic section for "/usr/lib/liblber-2.3.so.0" is not at the
>> expected address
>>
>> warning: difference appears to be caused by prelink, adjusting expectations
>> Reading symbols from /lib/libm.so.6...done.
>> Loaded symbols for /lib/libm.so.6
>> Reading symbols from /usr/local/httpd-2.2.11/lib/libaprutil-1.so.0...done.
>> Loaded symbols for /usr/local/httpd-2.2.11/lib/libaprutil-1.so.0
>> Reading symbols from /lib/libexpat.so.0...done.
>> Loaded symbols for /lib/libexpat.so.0
>> Reading symbols from /usr/local/httpd-2.2.11/lib/libapr-1.so.0...done.
>> Loaded symbols for /usr/local/httpd-2.2.11/lib/libapr-1.so.0
>> Reading symbols from /lib/libuuid.so.1...done.
>> Loaded symbols for /lib/libuuid.so.1
>> Reading symbols from /lib/librt.so.1...done.
>> Loaded symbols for /lib/librt.so.1
>> Reading symbols from /lib/libcrypt.so.1...done.
>> Loaded symbols for /lib/libcrypt.so.1
>> Reading symbols from /lib/libpthread.so.0...done.
>> [Thread debugging using libthread_db enabled]
>> [New Thread 0x135700 (LWP 18421)]
>> Loaded symbols for /lib/libpthread.so.0
>> Reading symbols from /lib/libdl.so.2...done.
>> Loaded symbols for /lib/libdl.so.2
>> Reading symbols from /lib/libc.so.6...done.
>> Loaded symbols for /lib/libc.so.6
>> Reading symbols from /lib/ld-linux.so.2...done.
>> Loaded symbols for /lib/ld-linux.so.2
>> Reading symbols from /lib/libnss_files.so.2...done.
>> Loaded symbols for /lib/libnss_files.so.2
>>
>>
>> (gdb) print PyRun_SimpleString("import traceback; traceback.print_stact()")
>>
>> Program received signal SIGSEGV, Segmentation fault.
>> PyImport_AddModule (name=0x5596b3 "__main__") at Python/import.c:366
>> 366             PyInterpreterState *interp = PyThreadState_GET()->interp;
>> The program being debugged was signaled while in a function called from GDB.
>> GDB remains in the frame where the signal was received.
>> To change this behavior use "set unwindonsignal on"
>> Evaluation of the expression containing the function (PyRun_SimpleString)
>> will be abandoned.
>> (gdb)
>>     
>
> I don't want a Python stack trace, I want a C stack trace as
> documented in that page I sent you to. Use:
>
>   (gdb) thread apply all bt
>
> Graham
>
>   
>> Graham Dumpleton wrote:
>>
>> On 13 March 2010 11:28, wayne collier <Wayne.Collier at noaa.gov> wrote:
>>
>>
>> Okay I am running RH linux and I don't seem to have mod_ldap or
>> mod_authnz_ldap shared objects loaded in my httpd.conf file.
>>
>>
>> They may not be loaded from httpd.conf but from a configuration file
>> snippet loaded into Apache. I don't know RH Apache configuration
>> layout, but check all snippet files in subdirectories to the
>> httpd.conf directory.
>>
>> The conflict may also be because you are loading mod_php.
>>
>> Ultimately you may have to use gdb to work out exactly where it
>> crashes. Hopefully not too much symbol information has been stripped
>> and gdb will be able to tell you from which library LDAP code that is
>> crashing is coming from.
>>
>> Use first documented procedure for running Apache in single process
>> under gdb in:
>>
>>
>> http://code.google.com/p/modwsgi/wiki/DebuggingTechniques#Debugging_Crashes_With_GDB
>>
>> This is for mod_wsgi, but same can be done for mod_python.
>>
>> Graham
>>
>>
>>
>> Also I ran ldd
>> and got the following results:
>>
>>
>>
>>
>> ldd /usr/local/httpd-2.2.11/bin/httpd
>>
>>
>>     linux-gate.so.1 =>  (0x00157000)
>>     libm.so.6 => /lib/libm.so.6 (0x00ce8000)
>>     libaprutil-1.so.0 => /usr/local/httpd-2.2.11/lib/libaprutil-1.so.0
>> (0x00315000)
>>     libexpat.so.0 => /lib/libexpat.so.0 (0x00d70000)
>>     libapr-1.so.0 => /usr/local/httpd-2.2.11/lib/libapr-1.so.0 (0x00844000)
>>     libuuid.so.1 => /lib/libuuid.so.1 (0x002f2000)
>>     librt.so.1 => /lib/librt.so.1 (0x00d65000)
>>     libcrypt.so.1 => /lib/libcrypt.so.1 (0x07e07000)
>>     libpthread.so.0 => /lib/libpthread.so.0 (0x00d17000)
>>     libdl.so.2 => /lib/libdl.so.2 (0x00d11000)
>>     libc.so.6 => /lib/libc.so.6 (0x00ba0000)
>>     /lib/ld-linux.so.2 (0x00b82000)
>>
>>
>>
>> ldd /usr/local/httpd-2.2.11/modules/mod_python.so
>>
>>
>>     linux-gate.so.1 =>  (0x00aec000)
>>     libpthread.so.0 => /lib/libpthread.so.0 (0x00730000)
>>     libdl.so.2 => /lib/libdl.so.2 (0x0027f000)
>>     libutil.so.1 => /lib/libutil.so.1 (0x00f70000)
>>     libm.so.6 => /lib/libm.so.6 (0x009b1000)
>>     libc.so.6 => /lib/libc.so.6 (0x00110000)
>>     /lib/ld-linux.so.2 (0x00b82000)
>>
>> I don't seem to have anything with ldap listed. Am I on the right track.?
>>
>>
>> Wayne
>>
>> Graham Dumpleton wrote:
>>
>> A segmentation fault can occur for other reasons. If it is happening
>> when you use LDAP, it could be a conflict in versions of LDAP
>> libraries being used by Apache or PHP and Python LDAP module.
>>
>> You should use the 'ldd' command to determine what LDAP shared library
>> is used by mod_authnz_ldap and compare that to the LDAP library being
>> used by the C extension module component of the Python module you are
>> using to access LDAP. Similarly find any PHP extension modules which
>> use LDAP and look at those.
>>
>> If mod_authnz_ldap is loaded into Apache and you don't need it, then
>> don't load it and see if problem goes away. Also try not loading PHP
>> if it is being loaded. Narrow down which is in conflict.
>>
>> As far as I remember, Apache itself doesn't link to LDAP library and
>> even possible that mod_authnz_ldap uses an internal LDAP library
>> rather than linking external library. All the same even use ldd on
>> Apache binary itself to see which LDAP library may be linked.
>>
>> Graham
>>
>> On 10 March 2010 04:18, wayne collier <Wayne.Collier at noaa.gov> wrote:
>>
>>
>> Okay I guess I will look else where. However, I keep getting segmentation
>> faults when try an execute the ldap command
>>
>> t=l.simple_bind_s(dn,pw). It is only line of code that crashes. If the have
>> the classic expat problem, would that mean I would get a segmentation
>> fault problem from the start.  Also, ldap script works okay as a stand alone
>> script. Thanks.
>>
>> Wayne
>>
>>
>> Graham Dumpleton wrote:
>>
>> That is not your problem, quoting:
>>
>> http://code.google.com/p/modwsgi/wiki/IssuesWithExpatLibrary
>>
>> which is more up to date in respect of this issue, even if the
>> mod_wsgi variant of that document:
>>
>> """Note that this only applies to Python versions prior to Python 2.5.
>> >From Python 2.5 onwards, the copy of the "expat" library bundled in
>> with Python is name space prefixed, thereby avoid name clashes with an
>> "expat" library which has previously been loaded."""
>>
>> You are using Python 2.6, so you cannot be affected by this problem.
>>
>> Graham
>>
>> On 9 March 2010 05:54, wayne collier <Wayne.Collier at noaa.gov> wrote:
>>
>>
>> Hello all. I need some help solving the dreaded segmentation fault that has
>> cropped several time on this board. I have read the link
>> http://www.dscpl.com.au/wiki/ModPython/Articles/ExpatCausingApacheCrash and
>> followed the step and have uncovered the following.
>>
>>
>>
>>
>> ldd    /usr/local/httpd-2.2.11/bin/httpd | grep expat
>>
>>
>>     libexpat.so.0 => /lib/libexpat.so.0 (0x00d70000)
>>
>>
>>
>> strings /lib/libexpat.so.0  | grep expat_
>>
>>
>> expat_1.95.8
>>
>>
>>
>>   ps aux | grep http | head -3
>>
>>
>> webuser  14986  0.0  2.1  54236 19048 ?        S    Mar04   0:01
>> /usr/local/httpd-2.2.11/bin/httpd -d /usr/local/httpd-2.2.11 -f
>> /usr/local/httpd-2.2.11/conf/httpd.conf
>> webuser  14987  0.0  3.2 144128 29540 ?        S    Mar04   0:01
>> /usr/local/httpd-2.2.11/bin/httpd -d /usr/local/httpd-2.2.11 -f
>> /usr/local/httpd-2.2.11/conf/httpd.conf
>> webuser  14998  0.0  3.2 144212 29660 ?        S    Mar04   0:01
>> /usr/local/httpd-2.2.11/bin/httpd -d /usr/local/httpd-2.2.11 -f
>> /usr/local/httpd-2.2.11/conf/httpd.conf
>>
>>
>>
>>   /usr/sbin/lsof -p 14986 | grep expat
>>
>>
>> httpd   14986 webuser  mem    REG     3,5   572313  972637
>> /opt/Python/lib/python2.6/lib-dynload/pyexpat.so
>> httpd   14986 webuser  mem    REG     3,5   133120 1883791
>> /lib/libexpat.so.0.5.0
>>
>> I am not sure why I get 2 shared libaries here.
>>
>>
>>
>>   strings  /lib/libexpat.so.0.5.0 | grep expat_
>>
>>
>> expat_1.95.8
>>
>>
>>
>>
>> strings /opt/Python/lib/python2.6/lib-dynload/pyexpat.so | grep expat_
>>
>>
>> pyexpat.expat_CAPI 1.0
>> expat_CAPI
>> expat_2.0.0
>>
>>
>>
>>
>> import pyexpat
>> pyexpat.version_info
>>
>>
>> (2, 0, 0)
>>
>>
>> So it seems like I need get expat_1.95.8 apache is using up to expat_2.0.0.
>> Can I simply try replacing the 1.95.8 library with 2.0.0. Or do I have
>> recompile with Apache? If I do recompile I need to recompile apache with
>> mod_python as well?
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> Mod_python mailing list
>> Mod_python at modpython.org
>> http://mailman.modpython.org/mailman/listinfo/mod_python
>>
>>
>>
>>
>>
>> _______________________________________________
>> Mod_python mailing list
>> Mod_python at modpython.org
>> http://mailman.modpython.org/mailman/listinfo/mod_python
>>
>>
>>
>>
>>
>> _______________________________________________
>> Mod_python mailing list
>> Mod_python at modpython.org
>> http://mailman.modpython.org/mailman/listinfo/mod_python
>>
>>
>>
>>     
>
> _______________________________________________
> Mod_python mailing list
> Mod_python at modpython.org
> http://mailman.modpython.org/mailman/listinfo/mod_python
>
>   

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mm_cfg_has_not_been_edited_to_set_host_domains/pipermail/mod_python/attachments/20100322/55c34695/attachment-0001.html


More information about the Mod_python mailing list