[mod_python] newbie question - using multiple handlers

I'm pretty new to both mod_python and apache. I have just written multiple handlers to deal with different webpages I have written. My problem is that if I use PythonHandler for each of my handlers in the conf. file, whenever I run a webpage, it tries to use ALL my handlers. Basically, I want only one handler to be used per webpage. The "action" in my forms only specify one handler, but based on the errors I am getting, it is also running the other handlers. Is there a way that I can set directives such as PythonHandler for particular html files? Or do I need to separate my files into different directories? I would hate to have to make a switch handler that determined what methods to call from my handler files so that the conf file only had one handler in it.
Thanks,
Tom