11th April 2014 - crawler-commons 0.4 is released We are glad to announce the 0.4 release of Crawler Commons. Amongst other improvements, this release includes support for Googlebot-compatible regular expressions in URL specifications, further imprvements to robots.txt parsing and an upgrade of httpclient to v4.2.6.
See the CHANGES.txt<http://crawler-commons.googlecode.com/svn/tags/crawler-commons-0.4/CHANGES.txt> file included with the release for a full list of details. We suggest all users to upgrade to this version. Details of how to do so can be found on Maven Central<http://search.maven.org/#search%7Cgav%7C1%7Cg%3A%22com.google.code.crawler-commons%22%20AND%20a%3A%22crawler-commons%22> . Thanks Lewis (on behalf of CC team) -- *Lewis*
