A webapp that does the work of creating a robots.txt file for you. "The robots.txt is a very simple text file that is placed on your root directory. An example would bewww.yourdomain.com/robots.txt. This file tells search engine and other robots which areas of your site they are allowed to visit and index."