[mod_python] Catalog of mod_python.publisher problems.

Graham Dumpleton wrote ..
> 
> Do we need to document properly first the perceived problems and some
> examples of errornous behaviour? This will help to ensure we fix
> everything and provide a basis for some tests of any new implementation.
> I know that such information can be noted against the bug tracker item,
> but is it easier to thrash it out here first.

FWIW, here is my personal list of problems in mod_python.publisher that
I know about. Note that I have ignored totally the basic issue that sys.path
makes which module you get unpredictable if import_module() is used
directly. This is just mod_python.publisher problems which should in
themselves be fixable whatever may eventually be done about the
import_module() method. Let me know if you want these put on bug
tracker as one item or three items.


Continual reloading of modules
------------------------------

Create a subdirectory called "publisher". In that directory create a
".htaccess" file containing:

  SetHandler python-program
  PythonHandler mod_python.publisher
  PythonDebug On

Now create two subdirectories "subdir-1" and "subdir-1". In both of these
subdirectories create an "index.py" file which contains:

  import os
  def index():
    return os.getpid(),__file__

Restart Apache to clear any cached modules and then cycle between the URLs
corresponding to the two subdirectories. In my case this is:

  /~grahamd/publisher/subdir-1
  /~grahamd/publisher/subdir-2
  /~grahamd/publisher/subdir-1
  /~grahamd/publisher/subdir-2
  ...

The output for each page was in turn:

  (462, '/Users/grahamd/Sites/publisher/subdir-1/index.py')
  (462, '/Users/grahamd/Sites/publisher/subdir-2/index.py')
  (462, '/Users/grahamd/Sites/publisher/subdir-1/index.pyc')
  (462, '/Users/grahamd/Sites/publisher/subdir-2/index.pyc')
  ...

If you look at the Apache error log file you will see something like:

  [Tue Jan 25 09:03:45 2005] [notice] mod_python: Creating 32 session mutexes
  based on 4 max processes and 25 max threads.
  [Tue Jan 25 09:03:45 2005] [notice] Apache/2.0.51 (Unix) mod_python/3.1.3
  Python/2.3 configured -- resuming normal operations

  [Tue Jan 25 09:04:02 2005] [notice] mod_python: (Re)importing module
  'mod_python.publisher'
  [Tue Jan 25 09:04:02 2005] [notice] mod_python: (Re)importing module 'index'
  with path set to '['/Users/grahamd/Sites/publisher/subdir-1']'
  [Tue Jan 25 09:04:16 2005] [notice] mod_python: (Re)importing module 'index'
  with path set to '['/Users/grahamd/Sites/publisher/subdir-2']'
  [Tue Jan 25 09:04:26 2005] [notice] mod_python: (Re)importing module 'index'
  with path set to '['/Users/grahamd/Sites/publisher/subdir-1']'
  [Tue Jan 25 09:04:40 2005] [notice] mod_python: (Re)importing module 'index'
  with path set to '['/Users/grahamd/Sites/publisher/subdir-2']'

You will see how as one cycles between the two URLs corresponding to the
modules, that the modules are reimported everytime.

Note that I ensured that only one Apache server process was initially
started so that all requests served by same process. One can also see this
is the response, which includes the process ID of the server process.

Overall what is returned is correct, but it isn't efficient because each
request is triggering a module import.

Cross contamination of modules
------------------------------

For the above case, change the code in "subdir-1/index.py" to:

  subdir1 = None
  def index():
    return "subdir-1",globals().keys()

and "subdir-2/index.py" to:

  subdir2 = None
  def index():
    return "subdir-2",globals().keys()

Accessing "subdir-1" the result is:

  ('subdir-1', ['index', '__mtime__', '__builtins__',
   '__file__', 'subdir1', '__name__', '__doc__'])

Now accessing "subdir-2" the result is:

  ('subdir-2', ['index', '__mtime__', '__builtins__',
   '__file__', 'subdir2', 'subdir1', '__name__', '__doc__'])

Back to "subdir-1" again:

  ('subdir-1', ['index', '__mtime__', '__builtins__',
   '__file__', 'subdir2', 'subdir1', '__name__', '__doc__'])

Because modules of the same name are reimported on top of the existing
module you can end up with cross contamination of modules in respect of
global variables, functions, class definitions, module imports etc.

The most obvious problem this causes with publisher, is that one can
suddenly have appear in a module a function from a different module. This
function then becomes accessible using an appropriate URL via the module
in which it doesn't belong.

Overall this is annoying and could be problematic for those unaware.

Sticky modules obscuring real module
------------------------------------

Create and "index.py", "subdir-1/index.py" and "subdir-2/index.py" all
containing:

  import os
  def index():
    return os.getpid(),__file__

Now cycle through accessing these in the order:

  index.py
  subdir-1/index.py
  index.py
  subdir-2/index.py
  index.py
  subdir-1/index.py
  index.py
  subdir-2/index.py
  ...

In my case this is:

  /~grahamd/publisher
  /~grahamd/publisher/subdir-1
  /~grahamd/publisher
  /~grahamd/publisher/subdir-2
  /~grahamd/publisher
  /~grahamd/publisher/subdir-1
  /~grahamd/publisher
  /~grahamd/publisher/subdir-2
  /~grahamd/publisher
  ...

The result for this is:

  (521, '/Users/grahamd/Sites/publisher/index.py')
  (521, '/Users/grahamd/Sites/publisher/subdir-1/index.py')
  (521, '/Users/grahamd/Sites/publisher/subdir-1/index.py')
  (521, '/Users/grahamd/Sites/publisher/subdir-2/index.py')
  (521, '/Users/grahamd/Sites/publisher/subdir-2/index.py')
  (521, '/Users/grahamd/Sites/publisher/subdir-1/index.pyc')
  (521, '/Users/grahamd/Sites/publisher/subdir-1/index.pyc')
  (521, '/Users/grahamd/Sites/publisher/subdir-2/index.pyc')
  (521, '/Users/grahamd/Sites/publisher/subdir-2/index.pyc')

One can see that once one of the "index.py" files in the subdirectories is
accessed, they become sticky and the latest one accessed from a subdirectory
is returned instead of the top level one when it is accessed.

This one has got to be a bug.

--
Graham Dumpleton (grahamd at dscpl.com.au)