User-agent: * # directed to all spiders, Disallow: /admin/ Disallow: /catalogue/ Disallow: /errordocs/ Disallow: /images/ Disallow: /shop_manager/ Disallow: /*.js