30 Aug
2009
30 Aug
'09
10:32 p.m.
Hello, This one is for webmasters or those who administer web servers. I'm looking for a robots.txt file or perhaps entries to add to an httpd.conf or .htaccess file to block or keep out bad spiders, crawlers that proliferate spam, or any type of marketing thingy, anything other than a search engine. If anyone has anything on this i'm interested. Thanks. Dave.