Robots.txt File
From Lankyland
robots.txt
The robots.txt file is a simple text file that is placed in the root directory of your Website. Most Search Engines read these robots.txt files which can include instructions on what the Search Engine can and cannot do.
