[mod_python] OS X / Apache 2 / mod_python bug

Jamie Kirkpatrick jkp at kirkconsulting.co.uk
Mon Mar 14 04:16:57 EST 2005

On 13 Mar 2005, at 17:13, John Raines wrote:

> I'm running Apache2.0.53,python 2.3.3(from fink), osX10.3.8 and 
> mod_python 3.1.4.  Apachectl that comes in the sys path is targeted 
> at the Apache1.3 that came with my powerMacG4. the apachectl that 
> relates to apache2 on my machine is at 
> /usr/local/apache2/bin/apachectl and can be invoked with the full 
> path; I renamed the std apachectl and put a link to the apache2 
> version into /usr/sbin (where apachectl for 1.3 sits)

Sure, but I'm definitely using the right one!!!! ;)

> Another issue has been the dynamic shared object thing which has 
> prevented me from moving up to python 2.3.4 or python2.4.
> Graham Dumpleton set me straight on that issue in these posts:
> If you really want to use Python 2.4, you are going to have to somehow
> force a reference to the Python library in mod_python.so. Since you 
> aren't
> using a framework install, not sure what the library is you have to 
> force
> link with. You might want to look for Python related libraries in such
> directories as:
>   /usr/local/lib
>   /usr/local/lib/python2.4/config

That's pretty easy to do - just point the ./configure script at the 
correct python install when you build and make sure it's components are 
first in your path.  that way it will be preffered over the stock 
install.  Darwinports is already configured to used  it's own 2.4 
install and I had no trouble linking to my own.  You can tell which is 
being using by the footer (or looking in the error_log) shown by Apache.

> BTW, is there anything you really want in Python 2.4? Especially if 
> you are
> a UNIX novice, trying to host two different versions of Python on 
> MacOSX
> can be a problem. I guess at least you didn't use a framework install 
> and
> do it on top of the system supplied version of Python. Doing that can
> really screw things up bad.

I don't really need 2.4, but I tried all versions in my quest to sort 
this mess out!

> The only catch on using standard MacOSX Python installation is that 
> you
> need to patch src/mod_python.c file in source to workaround a problem
> with it. Use of "worker" may also trigger some other mod_python bugs.
> Fixes for both are detailed in:
>   http://www.dscpl.com.au/projects/vampire/PATCHES
> Graham
> The problem is that the libpython2.4.a is a static library and there
> is not a dynamically loadable library. See following comment from the
> mod_python README file:
>   3. On Darwin, libpython cannot be linked statically with mod_python
>   using libtool. libpython has to be a dynamic shared object. The 
> Python
>   distribution does not provide a way of building libpython as a 
> shared
>   library, but the Fink Python distribution comes with one
>   (/sw/lib/python2.2/config/libpython2.2.dylib), so the easiest thing 
> is
>   to install Python from Fink (fink.sourceforge.net).
> I would assume the same comment applies if building your own version
> of Python on full MacOSX and where a framework install wasn't done.
> Normally with a framework install, the dynamically loadable version is
> installed under the framework package as:
>   /System/Library/Frameworks/Python.framework/Versions/2.3/Python
> If you run "otool -L" on your old mod_python.so which used standard
> Python version, that is what you should have seen.
> Your options thus seem to be:
> 1. Use standard Python 2.3 shipped with MacOSX.
> 2. Use Python 2.4 provided with "fink" packages.
> 3. Install Python 2.4 as a framework but make sure you install it
> such that you don't overwrite the standard Python installation.

I did do this at one point - it doesn't overwrite since it puts 
everything in /Library rather that /System/Library.

> The safest and quickest thing to do is use Python 2.3 shipped with
> the box. Using fink means a lot of crap get installed on your box
> that you might not want. Installing Python 2.4 runs the risk of
> trashing your system version of Python if you do the wrong thing.

I think he is right here, just that this bug was obscuring my vision.  
If I could iron out the bug I might go back to building everything from 
source and doing it this way.  I prefer the control i get of things 
that way.  Lets see what the responses from the list turn up!



PS - my errors seem to be of a different breed to yours since you got 
detail in the logs.  Even with PythonDebug on i get nothing.  :/

> Graham
> John Raines wrote ..
>> My website had been stable and running without error but I'd been
>> running on 2.3.3, 2.0.52 and 3.1.3; since everything was behind the
>> times I decided to finally reinstall; I made the mistake of doing it 
>> on
>> the drive that everything was being served off of instead of using my
>> backup drive. :-(
>> I installed python 2.4 and it seems to work when I call it from the
>> shell. I did not do a frameworkInstall, which seems to be recommended
>> by some people. I am too much of a unix novice to understand what the
>> implications of frameworkInstall are. I installed apache and 
>> mod_python
>> as follows:
>> cd ~/Desktop/httpd-2.0.53
>> ./configure --enable-so --with-mpm=worker
>> make
>> sudo make install
>> cd ~/Desktop/mod_python-3.1.4
>> ./configure --with apxs=/usr/local/apache2/bin/apxs
>> --with-python=/usr/local/bin/python2.4
>> make
>> sudo make install
>> I restored my old httpd.conf file and my website files. My index.html
>> file is straight html which links to .py files. As soon as I try to
>> access any of the .py stuff I now get an internal server error 
>> message
>> and the server log shows this:
>> [Tue Feb 15 16:25:25 2005] [notice] mod_python: Creating 32 session
>> mutexes based on 6 max processes and 25 max threads.
>> [Tue Feb 15 16:25:25 2005] [notice] Apache configured -- resuming
>> normal operations
>> [Tue Feb 15 16:25:36 2005] [error] make_obcallback: could not import
>> mod_python.apache.\n
>> Traceback (most recent call last):
>>    File 
>> "/usr/local/lib/python2.4/site-packages/mod_python/apache.py",
>> line 22, in ?
>>      import time
>> ImportError: Failure linking new module: : dyld:
>> /usr/local/apache2/bin/httpd Undefined symbols:
>> /usr/local/lib/python2.4/lib-dynload/time.so undefined reference to
>> _PyArg_Parse expected to be defined in the executable
>> /usr/local/lib/python2.4/lib-dynload/time.so undefined reference to
>> _PyArg_ParseTuple expected to be defined in the executable
>> /usr/local/lib/python2.4/lib-dynload/time.so undefined reference to
>> _PyDict_GetItemString expected to be defined in the executable
>> /usr/local/lib/python2.4/lib-dynload/time.so un
>>                                                               <--what
>> happened here??
>> [Tue Feb 15 16:25:36 2005] [error] make_obcallback: could not import
>> mod_python.apache.
>> for what its worth I can access various parts of the time module 
>> when I
>> run Python2.4 from the shell.
>> Going back through the the mailing list I saw that Graham Dumpleton
>> (back on Dec 23) suggested the output from this might be useful; 
>> what I
>> get is different from what he saw but I'm not sure what to make of my
>> result:
>> jraines-Computer:~/Desktop/Website jrraines$ otool otool -L
>> /usr/local/apache2/modules/mod_python.so
>> otool: can't open file: otool (No such file or directory)
>> /usr/local/apache2/modules/mod_python.so:
>>          /usr/lib/libSystem.B.dylib (compatibility version 1.0.0,
>> current version 71.1.1)
>> jraines-Computer:~/Desktop/Website jrraines$ ls
>> /usr/local/apache2/modules
>> httpd.exp       mod_python.so
>> My problem seemed like it might be similar to a thread titled Weird
>> ob_callback problems at the end of Jan. I tried the suggestions 
>> Grisha
>> made:Try defining DYLD_FORCE_FLAT_NAMESPACE=1 environment variable
>> before launching httpd.
>> That didn't help either.
> On Mar 13, 2005, at 5:39 AM, Jamie Kirkpatrick wrote:
>> Dear all.
>> This mail is really directed at the developers so I hope there are 
>> some of you listening.  I would have preferred to enter this in a 
>> bug tracker of some sort, but I couldn't find a relevant URL on the 
>> website.  Anyway.
>> I have just spent 5 days battling with what i probably the most 
>> annoying bug I have ever come across!  For the last week I have been 
>> trying to work out why the hell I couldnt' get a working install of 
>> mod_python on OS X and I have tried _everything_.    I had managed 
>> to get it up and running once or twice, but then I would attempt to 
>> tweak my Apache config to set things up for the sw i wanted to use 
>> (trac) and it would die again.  The logs showed absolutely nothing 
>> of use even with PythonDebug on.
>> The bug I have discovered basically boils down to a tiny thing that 
>> is not an easy thing to discover.  If you use `apachectl restart` to 
>> control the execution of Apache with mod_python installed on OS X 
>> (server at least, I didnt try the client version but I imagine it 
>> will be the same), you will find mod_python ceases to function, 
>> producing a 500 error from apache.
>> As I say , this is not an easy bug to discover, since you never 
>> expect there to be a difference in the way  a module behaves due to 
>> such an obscure thing.  I can confirm that this happens on OS X 
>> Server 10.3.6-8 with various different builds of apache2.  I used 
>> the stock build, a custom built version (from the latest sources) 
>> and one from dports with the same results.  I also tried different 
>> versions of python (The stock 2.3 Framework, 2.4 built as a 
>> framework, 2.4 normal, 2.3 normal....) as I had read on these lists 
>> that there are some oddities in this area on OS X.    Same results.  
>> Just as I was beginning to give up completely I worked out what was 
>> going on. :p
>> Bottom line is that i dont want anyone to have to go through the 
>> pain I went through this week trying to get to the bottom of this 
>> issue.  I feel 10 times better now I know what the problem is, and 
>> for the time being the solution is to simply issue a `stop` followed 
>> by a `start` when i need to change the configuration of the server.  
>> I hope this email at least finds it's way to the archives so that 
>> fellow OS X travellers can learn from my experience as there really 
>> is practially nothing relating to mod_python on os x on the web.  
>> /me wishes apple would have just bundled it in....
>> I really hope however that my findings can be investigated further 
>> so that the bug can be resolved and a fix can be rolled into the 
>> trunk.
>> I am happy to do any further testing if a patch is added by anyone.  
>> I would like to try myself, but I just don't have the time to get 
>> further into this stuff.
>> Many thanks
>> ps.  At this stage I have only reported the bug to you guys since 
>> the apache bug tracker asks you to specifically verify that the bug 
>> isnt related to a module - lets make sure that mod_python isnt at 
>> fault first and then pass it to the main apache guys if not.
>> --
>> jkp
>> Version: PGP 8.0.3
>> +zRkD1HnkaPVwylmNvI4RmceA0JuhCi9Y35Bkxa1yBLLB+uRC1Zf4+Ek/QCg/4Jh
>> 161Ewn5UOV2eAmxFFRmyCLsD/jCeNrU4w5i/8YLoVFy3vZYhNsPL299vNc/PmF+h
>> +NPbbWAgay2UVnyc0lkbtOr+ZiEjgf0ltr3rDrjaF6faxwo8sDE5Tuh1yUUJ5Ngv
>> VLgyDLn9ChnOxiKYGZeI173JXYYhk3V8eU8qtIdH+IBQeHJjeWJg4dKRNqqDzlaI
>> r3GmA/9S11g+wKWYwKUNbPqRp29KuiRkINV34EXjAdK20kwb3misSt1x86xUHslq
>> Egys6T7Ulw1bAFKAWnm4pddv0khVx95reQlk/pIJZym26QSdXORPIjJatEgkQuHX
>> p5hWVsN4mwak7gKFw6VhJMZGXtT3V/WJgrUUE9AMcMJPQSYneLQsSmFtaWUgS2ly
>> a3BhdHJpY2sgPGprcEBraXJrY29uc3VsdGluZy5jby51az6JAFcEEBECABcFAkAv
>> Dn+mGLNi1gCgtQE1xpBMxTY3wjFM7SCT4vUlwGa5Ag0EQC+sKRAIAPZCV7cIfwgX
>> cqK61qlC8wXo+VMROU+28W65Szgg2gGnVqMU6Y9AVfPQB8bLQ6mUrfdMZIZJ+AyD
>> vWXpF9Sh01D49Vlf3HZSTz09jdvOmeFXklnN/biudE/F/Ha8g8VHMGHOfMlm/xX5
>> u/2RXscBqtNbno2gpXI61Brwv0YAWCvl9Ij9WE5J280gtJ3kkQc2azNsOA1FHQ98
>> iLMcfFstjvbzySPAQ/ClWxiNjrtVjLhdONM0/XwXV0OjHRhs3jMhLLUq/zzhsSlA
>> GBGNfISnCnLWhsQDGcgHKXrKlQzZlp+r0ApQmwJG0wg9ZqRdQZ+cfL2JSyIZJrqr
>> ol7DVekyCzsAAgIH/0aGygku5RCrzJUp3lXlPuvc7BvmwRP0RRO9ktbP4abEZXYp
>> UcUeRbBcU6E3AukBHf5hgmTcNDNAyQeZA75XZhJTQoAkEM9oVUkV66+KoF307hrG
>> 4dNOlFSBYHe/hiT/khmOlXIfB+SvDWYSjsIrP+JU/F1nxrcAqru5xeUh6Y+qIt4b
>> JPYNYxwf3+OeaRhQFneN+7oxBgSEl74qHmT79jQKqfllZshrs01PpB1UvG77Rn/I
>> zadbj29gIFw1Gt2eh5aRIN0F2QxvqeMvZ6APqjutv4i+tm1L3U8JNW8fsCpCKkI6
>> AAAACgkQ4RXc3SiZNM/0VACeORa/zYtGyekOwxUXzW5BQqdlsjIAoPhHLTE5Kwcq
>> 2WgOOCUuqD0QoU8e
>> =/4Ed
>> _______________________________________________
>> Mod_python mailing list
>> Mod_python at modpython.org
>> http://mailman.modpython.org/mailman/listinfo/mod_python

More information about the Mod_python mailing list