This is needed for security reasons - often people looking to exploit applications will search for something in a certain version of a web application and find many sites to exploit. This stops that from being so easy.
User-agent: * Disallow: /
