How to stop (most) search engine crawlers
Most search engines adhere to the Robots Exclusion Standards when crawling a site. If you want to stop it,you’ll need to write a proper robots.txt file. How to write one, you ask? Check out the Web Robots pages.
Comments are closed.















