[ https://issues.apache.org/jira/browse/NUTCH-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17541483#comment-17541483 ]
Hudson commented on NUTCH-2950: ------------------------------- SUCCESS: Integrated in Jenkins build Nutch ยป Nutch-trunk #76 (See [https://ci-builds.apache.org/job/Nutch/job/Nutch-trunk/76/]) NUTCH-2950 Improve performance of UpdateHostDb (snagel: [https://github.com/apache/nutch/commit/70b2d5e556d937a90018e156c6147e525bfee04b]) * (edit) src/java/org/apache/nutch/util/URLUtil.java * (edit) src/java/org/apache/nutch/hostdb/UpdateHostDbMapper.java NUTCH-2950 Improve performance of UpdateHostDb (snagel: [https://github.com/apache/nutch/commit/5a6ac3bb1ccbd49c556ae5c98e686948d0171634]) * (edit) src/java/org/apache/nutch/crawl/Generator.java * (edit) src/java/org/apache/nutch/hostdb/HostDatum.java * (edit) src/java/org/apache/nutch/hostdb/ReadHostDb.java * (edit) src/java/org/apache/nutch/hostdb/UpdateHostDbReducer.java NUTCH-2950 Improve performance of UpdateHostDb (snagel: [https://github.com/apache/nutch/commit/417dee6d1ef642586388ebf5b57f8b1bda18e2d9]) * (edit) src/java/org/apache/nutch/util/URLUtil.java * (edit) src/java/org/apache/nutch/hostdb/UpdateHostDbMapper.java NUTCH-2950 Improve performance of UpdateHostDb (snagel: [https://github.com/apache/nutch/commit/5086958be704022de2253d5311da1cc23fc19b37]) * (edit) src/java/org/apache/nutch/hostdb/UpdateHostDbMapper.java * (edit) src/java/org/apache/nutch/util/URLUtil.java NUTCH-2950 Improve performance of UpdateHostDb (snagel: [https://github.com/apache/nutch/commit/947e67bef075f7e3c87d6b55f5004b0cd6402385]) * (edit) src/java/org/apache/nutch/util/URLUtil.java > UpdateHostDb: performance improvements > -------------------------------------- > > Key: NUTCH-2950 > URL: https://issues.apache.org/jira/browse/NUTCH-2950 > Project: Nutch > Issue Type: Improvement > Components: hostdb > Affects Versions: 1.18 > Reporter: Sebastian Nagel > Assignee: Sebastian Nagel > Priority: Major > Fix For: 1.19 > > > This issue addresses a couple of performance improvements when creating the > HostDb: > - avoid needless conversions between hostname and URL > - improvements of HostDb serialization (write and read) > - parametrize logging and log less on level INFO > - do not create DNS resolver threads if DNS look-ups are not requested by > command-line options > A patch/PR is ready. Depending on the chosen command-line options, a 10-20% > speed-up should be visible if DNS look-ups, normalization and filtering are > off. -- This message was sent by Atlassian Jira (v8.20.7#820007)