[mod_python] silly question

----- Original Message ----- 
From: "Daniele Cruciani" <crxop at tiscali.it>
To: "mod_python list" <mod_python at modpython.org>
Sent: Tuesday, November 09, 2004 6:54 AM
Subject: [mod_python] silly question


> Hello.
>
> I just want something to make /robots.txt available with get method. I
> already make a directory robots.txt and
>
> /.htaccess  : #this is in the main dir
>
> SetHandler mod_python
> PythonHandler mod_python.publisher
>
> /robots.txt/index.py
> def index(req):
>    req.content_type="text/plain"
>    return """
> User-agent: *
> Disallow:
> """
>
> This works for client but I think a spider will get a 404 (telnet
> does). Also this make great confution and apache replace the index
> function for / too, so request "GET /" return the same as "GET
> /robots.txt/", I guess this is a know problem.
>
> I've tried "RemoveHandler .txt" but it don't works (with SetHandler I
> suppose).
>
> So, there is a method for making robots.txt available?

The easiest way I can think of is to have a normal 'robots.txt' file 
available in the document root directory
and then in your handler simply test for the presence of "robots.txt" in the 
request URI.  If true,
exit the handler with the return code "apache.DECLINED"

You can use this trick any time you want a particular document to be 
processed in the 'normal' apache
way - that is, as if mod_python didn't exist on the server.