# Make changes for all web spiders User-agent: * Disallow: /site/Pages/TemplateSystem/ Disallow: /site/Pages/Shop/ShoppingCart.aspx Disallow: /*.axd$ Disallow: /*.axd Disallow: /ScriptResource.axd Disallow: /WebResource.axd Disallow: /scriptresource.axd Disallow: /webresource.axd User-agent: Mp3Bot Disallow: / User-agent: EasouSpider Disallow: / User-agent: Baiduspider Disallow: / User-agent: MJ12bot Disallow: / User-agent: Yandex Disallow: / User-Agent: AhrefsBot Disallow: / User-Agent: QuerySeekerSpider ( http://queryseeker.com/bot.html ) Disallow: / User-agent: XoviBot Disallow: /