# All robots will spider the domain# User-agent: Mediapartners-Google* Disallow: User-agent: * Disallow: /tag/* Disallow: /widget.asp* Disallow: /en/* Disallow: /fr/* Disallow: /de/* Disallow: /es/* Disallow: /gr/* Disallow: /it/* Disallow: /ru/* Disallow: /tr/* Disallow: /pl/* Disallow: /ja/* Disallow: /sr/* Disallow: /nl/* Disallow: /pt/*