[mod_python] mod_python apache config weirdness

I am reposting as I have not solved this properly and I am still having
trouble with it.  Please help.  The installs are all defaults from the
ports section.  I followed the instructions attached to them.  They are
usually pretty up to date.

<in Apache2 config>

AddHandler mod_python .py
       Gives a full rendering of the page (home, about, products) if I
manually add the .py extension


With:
SetHandler mod_python

will render all the text and links but no images or menu colors, image
links appears as broken image links.


Although if I an add handler and load the default page (nothing after
trailing slash) then change to SetHandler it will render and work fine.
But it is pulling images and such from the cache.  This is what tricked
me into thinking it was solved earlier.  As soon as I clear the cache or
it expires it falls back to no images and noo menu highlighting.

Ideas?

Once again the set up:
Apache2.0.49
mod_python3.1.3
Python2.3.3 patchlevel 4
on freebsd 4.9
4.6692016090