Author: marcus Date: Sun Aug 11 20:52:06 2013 New Revision: 1513002 URL: http://svn.apache.org/r1513002 Log: Updated 'robots.txt'
Modified: openoffice/ooo-site/trunk/content/robots.txt Modified: openoffice/ooo-site/trunk/content/robots.txt URL: http://svn.apache.org/viewvc/openoffice/ooo-site/trunk/content/robots.txt?rev=1513002&r1=1513001&r2=1513002&view=diff ============================================================================== --- openoffice/ooo-site/trunk/content/robots.txt (original) +++ openoffice/ooo-site/trunk/content/robots.txt Sun Aug 11 20:52:06 2013 @@ -1,14 +1,22 @@ User-agent: daumoa Disallow: / -User-Agent: Sosospider +User-agent: Sosospider Disallow: / -User-Agent: Snoopy +User-agent: Snoopy Disallow: / -User-Agent: * +User-agent: * Crawl-delay: 5 +Disallow: /download/cachedimages/ +Disallow: /download/legacy/ +Disallow: /download/next/ +Disallow: /download/sdk/ +Disallow: /download/source/ Disallow: /download/test/ +Disallow: /issues/ +Disallow: /search/ +Disallow: /servlets/ Disallow: /support/test/ -Disallow: /test +Disallow: /test/