User-agent: * # directed to all spiders, not just Scooter Disallow: /intranet Disallow: /scr_inc Disallow: /archive.jim Disallow: /assets/ Disallow: /include Disallow: /js Disallow: /scripts Disallow: /addtocart.asp