# All robots will spider the domain User-agent: * Disallow: MASTERTEMPLATE.shtml # Disallow directory / User-agent: * Disallow: /includes/ Disallow: /images/ Disallow: /development/ Disallow: /TEMPLATES/