[jira] [Commented] (NUTCH-1747) Use AtomicInteger as semaphore in Fetcher
[ https://issues.apache.org/jira/browse/NUTCH-1747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13961211#comment-13961211 ] Sebastian Nagel commented on NUTCH-1747: +1 Looks like inProgress was intended to hold more than the bare count of FetchItems in progress. In doubt, we can get the in-progress FetchItems and their associated queue from FetcherThreads (cf. NUTCH-1182). Use AtomicInteger as semaphore in Fetcher - Key: NUTCH-1747 URL: https://issues.apache.org/jira/browse/NUTCH-1747 Project: Nutch Issue Type: Improvement Components: fetcher Affects Versions: 1.8 Reporter: Julien Nioche Priority: Minor Attachments: NUTCH-1747-trunk.patch In Fetcher we currently use SetFetchItem inProgress = Collections.synchronizedSet(new HashSetFetchItem()); as semaphore within the FetchItemQueues to keep track of the URLs being fetched and prevent threads from pulling from them. It works fine but we could use AtomicIntegers instead as all we need is the counts, not the contents. This change would have little impact on the performance but would make the code a bit cleaner. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (NUTCH-1747) Use AtomicInteger as semaphore in Fetcher
[ https://issues.apache.org/jira/browse/NUTCH-1747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13961236#comment-13961236 ] Hudson commented on NUTCH-1747: --- SUCCESS: Integrated in Nutch-trunk #2592 (See [https://builds.apache.org/job/Nutch-trunk/2592/]) NUTCH-1747 Use AtomicInteger as semaphore in Fetcher (jnioche: http://svn.apache.org/viewvc/nutch/trunk/?view=revrev=1585196) * /nutch/trunk/CHANGES.txt * /nutch/trunk/src/java/org/apache/nutch/fetcher/Fetcher.java Use AtomicInteger as semaphore in Fetcher - Key: NUTCH-1747 URL: https://issues.apache.org/jira/browse/NUTCH-1747 Project: Nutch Issue Type: Improvement Components: fetcher Affects Versions: 1.8 Reporter: Julien Nioche Priority: Minor Attachments: NUTCH-1747-trunk.patch In Fetcher we currently use SetFetchItem inProgress = Collections.synchronizedSet(new HashSetFetchItem()); as semaphore within the FetchItemQueues to keep track of the URLs being fetched and prevent threads from pulling from them. It works fine but we could use AtomicIntegers instead as all we need is the counts, not the contents. This change would have little impact on the performance but would make the code a bit cleaner. -- This message was sent by Atlassian JIRA (v6.2#6252)
