[mod_python] Global Interpreter Lock problem with Berkeley DB XML and mod_python

David Schachter ds at bittorrent.com
Tue Apr 10 19:10:32 EDT 2007


Folks--

I'm having a problem embedding Berkeley DB XML into Apache with 
mod_python. The first attempt to create an XmlManager causes a lockup in 
Python--the Global Interpreter Lock is deadlocked, I think. (Stack trace 
below.) Any ideas of how to debug? Is there a BDB XML Python expert? I 
checked Google without success.

The code to demonstrate the problem is simple and shown below. When the 
line "xmlManager = dbxml.XmlManager()" is commented out, the code runs 
ok. Commented in, Apache locks up.

The most recent mod_python has this comment at 
http://www.modpython.org/live/mod_python-3.3.1/doc-html/app-changes.html:

    Third party C modules that use the simplified API for the Global
    Interpreter Lock (GIL), as described in PEP 311, can now be used.
    The only requirement is that such modules can only be used in the
    context of the "main_interpreter". /(More info at
    http://issues.apache.org/jira/browse/MODPYTHON-77.)/

The documentation for mod_python at 
http://www.modpython.org/live/current/doc-html/pyapi-interps.html says:

    Note that if any third party module is being used which has a C code
    component that uses the simplified API for access to the Global
    Interpreter Lock (GIL) for Python extension modules, then the
    interpreter name must be forcibly set to be "main_interpreter". This
    is necessary as such a module will only work correctly if run within
    the context of the first Python interpreter created by the process.

Conforming to this, I set the interpreter name in the httpd.conf file 
using the directive "PythonInterpreter main_interpreter" and I verified 
this by printing the interpreter name from my Python code.

In dbxml-2.3.10/dbxml/src/python, there's some code relating to the GIL 
in dbxml_python_wrap.cpp. If someone else has already beaten this 
problem into submission, I'd rather not duplicate the effort.

                         -- David Schachter

#! /usr/bin/python

import dbxml
from mod_python import apache
import os


def handler(req):
  id = "pid %d, interpreter '%s'" % (os.getpid(), req.interpreter)

  apache.log_error("About to start BDB XML from %s." % id)
  xmlManager = dbxml.XmlManager()
  apache.log_error("Started BDB XML.")

  req.write("Hi from %s." % id)
  return apache.OK


STACK TRACE IS:

(gdb) bt
#0  0xffffe410 in __kernel_vsyscall ()
#1  0xb7e97af0 in sem_wait at GLIBC_2.0 () from 
/lib/tls/i686/cmov/libpthread.so.0
#2  0x00000004 in ?? ()
#3  0xb7ce7a00 in PyThread_acquire_lock (lock=0x0, waitflag=1) at 
thread_pthread.h:298
#4  0xb7cb5689 in PyEval_RestoreThread (tstate=0x810f888) at 
Python/ceval.c:305
#5  0xb7ce16e7 in PyGILState_Ensure () at Python/pystate.c:512
#6  0xb78282f9 in _wrap_new_XmlManager (self=0x0, args=0xb7b9102c) at 
dbxml_python_wrap.cpp:898
     (The reference to line 898 is correct but surprising, the result of 
a macro expansion at line 4195. --D.S.)
#7  0xb7c65967 in PyObject_Call (func=0x0, arg=0xfffffffc, 
kw=0xfffffffc) at Objects/abstract.c:1795
#8  0xb7cb9e1d in PyEval_EvalFrame (f=0x822cdbc) at Python/ceval.c:3845
#9  0xb7cbd719 in PyEval_EvalCodeEx (co=0xb6b12e20, globals=0xfffffffc, 
locals=0x0, args=0x0, argcount=1,
    kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at 
Python/ceval.c:2741
#10 0xb7d0d6be in function_call (func=0xb6b0902c, arg=0xb6b892ac, 
kw=0x0) at Objects/funcobject.c:548
#11 0xb7c65967 in PyObject_Call (func=0x0, arg=0xfffffffc, 
kw=0xfffffffc) at Objects/abstract.c:1795
#12 0xb7c6e389 in instancemethod_call (func=0xfffffffc, arg=0xb6b892ac, 
kw=0xfffffffc)
    at Objects/classobject.c:2532
#13 0xb7c65967 in PyObject_Call (func=0x0, arg=0xfffffffc, 
kw=0xfffffffc) at Objects/abstract.c:1795
#14 0xb7c99728 in slot_tp_init (self=0xb6af1fac, args=0xfffffffc, 
kwds=0xfffffffc)
    at Objects/typeobject.c:4774
#15 0xb7c91ad4 in type_call (type=0xb7c99675, args=0xb7b9102c, kwds=0x0) 
at Objects/typeobject.c:435
#16 0xb7c65967 in PyObject_Call (func=0x0, arg=0xfffffffc, 
kw=0xfffffffc) at Objects/abstract.c:1795
#17 0xb7cba239 in PyEval_EvalFrame (f=0x820f224) at Python/ceval.c:3776
#18 0xb7cbd0eb in PyEval_EvalFrame (f=0x8195854) at Python/ceval.c:3651
#19 0xb7cbd0eb in PyEval_EvalFrame (f=0x81c9ac4) at Python/ceval.c:3651
#20 0xb7cbd719 in PyEval_EvalCodeEx (co=0xb6b87e20, globals=0xfffffffc, 
locals=0x0, args=0x7, argcount=0,
    kws=0x8282440, kwcount=7, defs=0x0, defcount=0, closure=0x0) at 
Python/ceval.c:2741
#21 0xb7cbb46a in PyEval_EvalFrame (f=0x82822a4) at Python/ceval.c:3660
#22 0xb7cbd719 in PyEval_EvalCodeEx (co=0xb6b87ee0, globals=0xfffffffc, 
locals=0x0, args=0x0, argcount=2,
    kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at 
Python/ceval.c:2741
#23 0xb7d0d6be in function_call (func=0xb6af3a74, arg=0xb6b7c78c, 
kw=0x0) at Objects/funcobject.c:548
#24 0xb7c65967 in PyObject_Call (func=0x0, arg=0xfffffffc, 
kw=0xfffffffc) at Objects/abstract.c:1795
#25 0xb7c6e389 in instancemethod_call (func=0xfffffffc, arg=0xb6b7c78c, 
kw=0xfffffffc)
    at Objects/classobject.c:2532
#26 0xb7c65bf6 in PyObject_CallMethod (o=0x0, name=0xb6b891ec "\001", 
format=0xb6b891ec "\001")
    at Objects/abstract.c:1795
#27 0xb7c5f505 in python_handler (req=0x828a7b8, phase=0xb7d16e59 
"PythonHandler") at mod_python.c:1652
#28 0x08073daf in ap_run_handler (r=0x828a7b8) at config.c:157
#29 0x080741b1 in ap_invoke_handler (r=0x828a7b8) at config.c:372
---Type <return> to continue, or q <return> to quit---
#30 0x0808ba77 in ap_process_request (r=0x828a7b8) at http_request.c:258
#31 0x0808929b in ap_process_http_connection (c=0x8185638) at 
http_core.c:184
#32 0x0807a599 in ap_run_process_connection (c=0x8185638) at connection.c:43
#33 0x0809e55a in child_main (child_num_arg=-4) at prefork.c:640
#34 0x0809e7a8 in make_child (s=0x80c9c80, slot=0) at prefork.c:680
#35 0x0809ee53 in ap_mpm_run (_pconf=0xbf86d6f0, plog=0x81031a0, 
s=0xbf86d6f4) at prefork.c:956
#36 0x080622c8 in main (argc=2, argv=0xbf86d8b4) at main.c:717
(gdb)



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mm_cfg_has_not_been_edited_to_set_host_domains/pipermail/mod_python/attachments/20070410/530f317c/attachment.html


More information about the Mod_python mailing list