[jira] [Commented] (HADOOP-11221) JAVA specification for hashcode does not enforce it to be non-negative, but IdentityHashStore assumes System.identityHashCode() is non-negative
[
https://issues.apache.org/jira/browse/HADOOP-11221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191919#comment-14191919
]
Hudson commented on HADOOP-11221:
-
FAILURE: Integrated in Hadoop-Mapreduce-trunk #1943 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1943/])
HADOOP-11221. IdentityHashStore assumes System.identityHashCode() is
non-negative. Contributed by Jinghui Wang (szetszwo: rev
a3dacc07e2c08bb4ecfce7b5e6d5602273989e9c)
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/IdentityHashStore.java
* hadoop-common-project/hadoop-common/CHANGES.txt
> JAVA specification for hashcode does not enforce it to be non-negative, but
> IdentityHashStore assumes System.identityHashCode() is non-negative
> ---
>
> Key: HADOOP-11221
> URL: https://issues.apache.org/jira/browse/HADOOP-11221
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 2.4.1
>Reporter: Jinghui Wang
>Assignee: Jinghui Wang
> Fix For: 2.6.0
>
> Attachments: HADOOP-11221.patch, HADOOP-11221.v1.patch,
> HADOOP-11221.v2.patch, HADOOP-11221.v3.patch
>
>
> The following code snippet shows that IdentityHashStore assumes the hashCode
> is always non-negative.
> {code:borderStyle=solid}
>private void putInternal(Object k, Object v) {
> int hash = System.identityHashCode(k);
> final int numEntries = buffer.length / 2;
> int index = hash % numEntries;
>...
>}
>
> private int getElementIndex(K k) {
> ...
> final int numEntries = buffer.length / 2;
> int hash = System.identityHashCode(k);
> int index = hash % numEntries;
> int firstIndex = index;
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11221) JAVA specification for hashcode does not enforce it to be non-negative, but IdentityHashStore assumes System.identityHashCode() is non-negative
[
https://issues.apache.org/jira/browse/HADOOP-11221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191788#comment-14191788
]
Hudson commented on HADOOP-11221:
-
FAILURE: Integrated in Hadoop-Hdfs-trunk #1918 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1918/])
HADOOP-11221. IdentityHashStore assumes System.identityHashCode() is
non-negative. Contributed by Jinghui Wang (szetszwo: rev
a3dacc07e2c08bb4ecfce7b5e6d5602273989e9c)
* hadoop-common-project/hadoop-common/CHANGES.txt
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/IdentityHashStore.java
> JAVA specification for hashcode does not enforce it to be non-negative, but
> IdentityHashStore assumes System.identityHashCode() is non-negative
> ---
>
> Key: HADOOP-11221
> URL: https://issues.apache.org/jira/browse/HADOOP-11221
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 2.4.1
>Reporter: Jinghui Wang
>Assignee: Jinghui Wang
> Fix For: 2.6.0
>
> Attachments: HADOOP-11221.patch, HADOOP-11221.v1.patch,
> HADOOP-11221.v2.patch, HADOOP-11221.v3.patch
>
>
> The following code snippet shows that IdentityHashStore assumes the hashCode
> is always non-negative.
> {code:borderStyle=solid}
>private void putInternal(Object k, Object v) {
> int hash = System.identityHashCode(k);
> final int numEntries = buffer.length / 2;
> int index = hash % numEntries;
>...
>}
>
> private int getElementIndex(K k) {
> ...
> final int numEntries = buffer.length / 2;
> int hash = System.identityHashCode(k);
> int index = hash % numEntries;
> int firstIndex = index;
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11221) JAVA specification for hashcode does not enforce it to be non-negative, but IdentityHashStore assumes System.identityHashCode() is non-negative
[
https://issues.apache.org/jira/browse/HADOOP-11221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191713#comment-14191713
]
Hudson commented on HADOOP-11221:
-
FAILURE: Integrated in Hadoop-Yarn-trunk #729 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/729/])
HADOOP-11221. IdentityHashStore assumes System.identityHashCode() is
non-negative. Contributed by Jinghui Wang (szetszwo: rev
a3dacc07e2c08bb4ecfce7b5e6d5602273989e9c)
* hadoop-common-project/hadoop-common/CHANGES.txt
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/IdentityHashStore.java
> JAVA specification for hashcode does not enforce it to be non-negative, but
> IdentityHashStore assumes System.identityHashCode() is non-negative
> ---
>
> Key: HADOOP-11221
> URL: https://issues.apache.org/jira/browse/HADOOP-11221
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 2.4.1
>Reporter: Jinghui Wang
>Assignee: Jinghui Wang
> Fix For: 2.6.0
>
> Attachments: HADOOP-11221.patch, HADOOP-11221.v1.patch,
> HADOOP-11221.v2.patch, HADOOP-11221.v3.patch
>
>
> The following code snippet shows that IdentityHashStore assumes the hashCode
> is always non-negative.
> {code:borderStyle=solid}
>private void putInternal(Object k, Object v) {
> int hash = System.identityHashCode(k);
> final int numEntries = buffer.length / 2;
> int index = hash % numEntries;
>...
>}
>
> private int getElementIndex(K k) {
> ...
> final int numEntries = buffer.length / 2;
> int hash = System.identityHashCode(k);
> int index = hash % numEntries;
> int firstIndex = index;
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11221) JAVA specification for hashcode does not enforce it to be non-negative, but IdentityHashStore assumes System.identityHashCode() is non-negative
[
https://issues.apache.org/jira/browse/HADOOP-11221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14190497#comment-14190497
]
Hudson commented on HADOOP-11221:
-
FAILURE: Integrated in Hadoop-trunk-Commit #6396 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/6396/])
HADOOP-11221. IdentityHashStore assumes System.identityHashCode() is
non-negative. Contributed by Jinghui Wang (szetszwo: rev
a3dacc07e2c08bb4ecfce7b5e6d5602273989e9c)
* hadoop-common-project/hadoop-common/CHANGES.txt
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/IdentityHashStore.java
> JAVA specification for hashcode does not enforce it to be non-negative, but
> IdentityHashStore assumes System.identityHashCode() is non-negative
> ---
>
> Key: HADOOP-11221
> URL: https://issues.apache.org/jira/browse/HADOOP-11221
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 2.4.1
>Reporter: Jinghui Wang
>Assignee: Jinghui Wang
> Fix For: 2.6.0
>
> Attachments: HADOOP-11221.patch, HADOOP-11221.v1.patch,
> HADOOP-11221.v2.patch, HADOOP-11221.v3.patch
>
>
> The following code snippet shows that IdentityHashStore assumes the hashCode
> is always non-negative.
> {code:borderStyle=solid}
>private void putInternal(Object k, Object v) {
> int hash = System.identityHashCode(k);
> final int numEntries = buffer.length / 2;
> int index = hash % numEntries;
>...
>}
>
> private int getElementIndex(K k) {
> ...
> final int numEntries = buffer.length / 2;
> int hash = System.identityHashCode(k);
> int index = hash % numEntries;
> int firstIndex = index;
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11221) JAVA specification for hashcode does not enforce it to be non-negative, but IdentityHashStore assumes System.identityHashCode() is non-negative
[
https://issues.apache.org/jira/browse/HADOOP-11221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14185516#comment-14185516
]
Hadoop QA commented on HADOOP-11221:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12677335/HADOOP-11221.v3.patch
against trunk revision 463aec1.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/4960//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/4960//console
This message is automatically generated.
> JAVA specification for hashcode does not enforce it to be non-negative, but
> IdentityHashStore assumes System.identityHashCode() is non-negative
> ---
>
> Key: HADOOP-11221
> URL: https://issues.apache.org/jira/browse/HADOOP-11221
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 2.4.1
>Reporter: Jinghui Wang
>Assignee: Jinghui Wang
> Attachments: HADOOP-11221.patch, HADOOP-11221.v1.patch,
> HADOOP-11221.v2.patch, HADOOP-11221.v3.patch
>
>
> The following code snippet shows that IdentityHashStore assumes the hashCode
> is always non-negative.
> {code:borderStyle=solid}
>private void putInternal(Object k, Object v) {
> int hash = System.identityHashCode(k);
> final int numEntries = buffer.length / 2;
> int index = hash % numEntries;
>...
>}
>
> private int getElementIndex(K k) {
> ...
> final int numEntries = buffer.length / 2;
> int hash = System.identityHashCode(k);
> int index = hash % numEntries;
> int firstIndex = index;
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11221) JAVA specification for hashcode does not enforce it to be non-negative, but IdentityHashStore assumes System.identityHashCode() is non-negative
[
https://issues.apache.org/jira/browse/HADOOP-11221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14185491#comment-14185491
]
Colin Patrick McCabe commented on HADOOP-11221:
---
Great. +1 pending jenkins
> JAVA specification for hashcode does not enforce it to be non-negative, but
> IdentityHashStore assumes System.identityHashCode() is non-negative
> ---
>
> Key: HADOOP-11221
> URL: https://issues.apache.org/jira/browse/HADOOP-11221
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 2.4.1
>Reporter: Jinghui Wang
>Assignee: Jinghui Wang
> Attachments: HADOOP-11221.patch, HADOOP-11221.v1.patch,
> HADOOP-11221.v2.patch, HADOOP-11221.v3.patch
>
>
> The following code snippet shows that IdentityHashStore assumes the hashCode
> is always non-negative.
> {code:borderStyle=solid}
>private void putInternal(Object k, Object v) {
> int hash = System.identityHashCode(k);
> final int numEntries = buffer.length / 2;
> int index = hash % numEntries;
>...
>}
>
> private int getElementIndex(K k) {
> ...
> final int numEntries = buffer.length / 2;
> int hash = System.identityHashCode(k);
> int index = hash % numEntries;
> int firstIndex = index;
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11221) JAVA specification for hashcode does not enforce it to be non-negative, but IdentityHashStore assumes System.identityHashCode() is non-negative
[
https://issues.apache.org/jira/browse/HADOOP-11221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14185406#comment-14185406
]
Jinghui Wang commented on HADOOP-11221:
---
Patch updated with comments.
> JAVA specification for hashcode does not enforce it to be non-negative, but
> IdentityHashStore assumes System.identityHashCode() is non-negative
> ---
>
> Key: HADOOP-11221
> URL: https://issues.apache.org/jira/browse/HADOOP-11221
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 2.4.1
>Reporter: Jinghui Wang
>Assignee: Jinghui Wang
> Attachments: HADOOP-11221.patch, HADOOP-11221.v1.patch,
> HADOOP-11221.v2.patch, HADOOP-11221.v3.patch
>
>
> The following code snippet shows that IdentityHashStore assumes the hashCode
> is always non-negative.
> {code:borderStyle=solid}
>private void putInternal(Object k, Object v) {
> int hash = System.identityHashCode(k);
> final int numEntries = buffer.length / 2;
> int index = hash % numEntries;
>...
>}
>
> private int getElementIndex(K k) {
> ...
> final int numEntries = buffer.length / 2;
> int hash = System.identityHashCode(k);
> int index = hash % numEntries;
> int firstIndex = index;
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11221) JAVA specification for hashcode does not enforce it to be non-negative, but IdentityHashStore assumes System.identityHashCode() is non-negative
[
https://issues.apache.org/jira/browse/HADOOP-11221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14185346#comment-14185346
]
Colin Patrick McCabe commented on HADOOP-11221:
---
Thanks, Jinghui. +1
Can you add a comment to the code stating that numEntries must be a power of 2
for this hashing code to work?
> JAVA specification for hashcode does not enforce it to be non-negative, but
> IdentityHashStore assumes System.identityHashCode() is non-negative
> ---
>
> Key: HADOOP-11221
> URL: https://issues.apache.org/jira/browse/HADOOP-11221
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 2.4.1
>Reporter: Jinghui Wang
>Assignee: Jinghui Wang
> Attachments: HADOOP-11221.patch, HADOOP-11221.v1.patch,
> HADOOP-11221.v2.patch
>
>
> The following code snippet shows that IdentityHashStore assumes the hashCode
> is always non-negative.
> {code:borderStyle=solid}
>private void putInternal(Object k, Object v) {
> int hash = System.identityHashCode(k);
> final int numEntries = buffer.length / 2;
> int index = hash % numEntries;
>...
>}
>
> private int getElementIndex(K k) {
> ...
> final int numEntries = buffer.length / 2;
> int hash = System.identityHashCode(k);
> int index = hash % numEntries;
> int firstIndex = index;
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11221) JAVA specification for hashcode does not enforce it to be non-negative, but IdentityHashStore assumes System.identityHashCode() is non-negative
[
https://issues.apache.org/jira/browse/HADOOP-11221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183621#comment-14183621
]
Hadoop QA commented on HADOOP-11221:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12677008/HADOOP-11221.v2.patch
against trunk revision dc6e819.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/4952//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/4952//console
This message is automatically generated.
> JAVA specification for hashcode does not enforce it to be non-negative, but
> IdentityHashStore assumes System.identityHashCode() is non-negative
> ---
>
> Key: HADOOP-11221
> URL: https://issues.apache.org/jira/browse/HADOOP-11221
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 2.4.1
>Reporter: Jinghui Wang
>Assignee: Jinghui Wang
> Attachments: HADOOP-11221.patch, HADOOP-11221.v1.patch,
> HADOOP-11221.v2.patch
>
>
> The following code snippet shows that IdentityHashStore assumes the hashCode
> is always non-negative.
> {code:borderStyle=solid}
>private void putInternal(Object k, Object v) {
> int hash = System.identityHashCode(k);
> final int numEntries = buffer.length / 2;
> int index = hash % numEntries;
>...
>}
>
> private int getElementIndex(K k) {
> ...
> final int numEntries = buffer.length / 2;
> int hash = System.identityHashCode(k);
> int index = hash % numEntries;
> int firstIndex = index;
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11221) JAVA specification for hashcode does not enforce it to be non-negative, but IdentityHashStore assumes System.identityHashCode() is non-negative
[
https://issues.apache.org/jira/browse/HADOOP-11221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183503#comment-14183503
]
Jinghui Wang commented on HADOOP-11221:
---
Thanks for the clarification. I missed the orignal divide by 2. Patch updated.
> JAVA specification for hashcode does not enforce it to be non-negative, but
> IdentityHashStore assumes System.identityHashCode() is non-negative
> ---
>
> Key: HADOOP-11221
> URL: https://issues.apache.org/jira/browse/HADOOP-11221
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 2.4.1
>Reporter: Jinghui Wang
>Assignee: Jinghui Wang
> Attachments: HADOOP-11221.patch, HADOOP-11221.v1.patch,
> HADOOP-11221.v2.patch
>
>
> The following code snippet shows that IdentityHashStore assumes the hashCode
> is always non-negative.
> {code:borderStyle=solid}
>private void putInternal(Object k, Object v) {
> int hash = System.identityHashCode(k);
> final int numEntries = buffer.length / 2;
> int index = hash % numEntries;
>...
>}
>
> private int getElementIndex(K k) {
> ...
> final int numEntries = buffer.length / 2;
> int hash = System.identityHashCode(k);
> int index = hash % numEntries;
> int firstIndex = index;
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11221) JAVA specification for hashcode does not enforce it to be non-negative, but IdentityHashStore assumes System.identityHashCode() is non-negative
[
https://issues.apache.org/jira/browse/HADOOP-11221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183267#comment-14183267
]
Tsz Wo Nicholas Sze commented on HADOOP-11221:
--
The original code has
{code}
final int numEntries = buffer.length / 2;
{code}
I just change " / 2" to " >> 1" for better performance.
> JAVA specification for hashcode does not enforce it to be non-negative, but
> IdentityHashStore assumes System.identityHashCode() is non-negative
> ---
>
> Key: HADOOP-11221
> URL: https://issues.apache.org/jira/browse/HADOOP-11221
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 2.4.1
>Reporter: Jinghui Wang
>Assignee: Jinghui Wang
> Attachments: HADOOP-11221.patch, HADOOP-11221.v1.patch
>
>
> The following code snippet shows that IdentityHashStore assumes the hashCode
> is always non-negative.
> {code:borderStyle=solid}
>private void putInternal(Object k, Object v) {
> int hash = System.identityHashCode(k);
> final int numEntries = buffer.length / 2;
> int index = hash % numEntries;
>...
>}
>
> private int getElementIndex(K k) {
> ...
> final int numEntries = buffer.length / 2;
> int hash = System.identityHashCode(k);
> int index = hash % numEntries;
> int firstIndex = index;
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11221) JAVA specification for hashcode does not enforce it to be non-negative, but IdentityHashStore assumes System.identityHashCode() is non-negative
[
https://issues.apache.org/jira/browse/HADOOP-11221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183248#comment-14183248
]
Jinghui Wang commented on HADOOP-11221:
---
Hi Nicholas,
A follow up question, why the right shift by 1 on the second line. Doing that
will make the index < numEntries/2 rather than index < numEntries. i.e.
numEntries is 8, by shifting right 1 bit, index will be between 0 and 3.
> JAVA specification for hashcode does not enforce it to be non-negative, but
> IdentityHashStore assumes System.identityHashCode() is non-negative
> ---
>
> Key: HADOOP-11221
> URL: https://issues.apache.org/jira/browse/HADOOP-11221
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 2.4.1
>Reporter: Jinghui Wang
>Assignee: Jinghui Wang
> Attachments: HADOOP-11221.patch, HADOOP-11221.v1.patch
>
>
> The following code snippet shows that IdentityHashStore assumes the hashCode
> is always non-negative.
> {code:borderStyle=solid}
>private void putInternal(Object k, Object v) {
> int hash = System.identityHashCode(k);
> final int numEntries = buffer.length / 2;
> int index = hash % numEntries;
>...
>}
>
> private int getElementIndex(K k) {
> ...
> final int numEntries = buffer.length / 2;
> int hash = System.identityHashCode(k);
> int index = hash % numEntries;
> int firstIndex = index;
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11221) JAVA specification for hashcode does not enforce it to be non-negative, but IdentityHashStore assumes System.identityHashCode() is non-negative
[
https://issues.apache.org/jira/browse/HADOOP-11221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183230#comment-14183230
]
Jinghui Wang commented on HADOOP-11221:
---
Hi Nicholas,
Thanks for the suggestion, I double checked the code looks like the capacity of
buffer.length will always be power of 2. Will update the patch.
{code:borderStyle=solid}
public IdentityHashStore(int capacity) {
Preconditions.checkArgument(capacity >= 0);
if (capacity == 0) {
this.capacity = 0;
this.buffer = null;
} else {
// Round the capacity we need up to a power of 2.
realloc((int)Math.pow(2,
Math.ceil(Math.log(capacity) / Math.log(2;
}
}
{code}
> JAVA specification for hashcode does not enforce it to be non-negative, but
> IdentityHashStore assumes System.identityHashCode() is non-negative
> ---
>
> Key: HADOOP-11221
> URL: https://issues.apache.org/jira/browse/HADOOP-11221
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 2.4.1
>Reporter: Jinghui Wang
>Assignee: Jinghui Wang
> Attachments: HADOOP-11221.patch, HADOOP-11221.v1.patch
>
>
> The following code snippet shows that IdentityHashStore assumes the hashCode
> is always non-negative.
> {code:borderStyle=solid}
>private void putInternal(Object k, Object v) {
> int hash = System.identityHashCode(k);
> final int numEntries = buffer.length / 2;
> int index = hash % numEntries;
>...
>}
>
> private int getElementIndex(K k) {
> ...
> final int numEntries = buffer.length / 2;
> int hash = System.identityHashCode(k);
> int index = hash % numEntries;
> int firstIndex = index;
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11221) JAVA specification for hashcode does not enforce it to be non-negative, but IdentityHashStore assumes System.identityHashCode() is non-negative
[
https://issues.apache.org/jira/browse/HADOOP-11221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14182327#comment-14182327
]
Tsz Wo Nicholas Sze commented on HADOOP-11221:
--
The capacity of an IdentityHashStore and buffer.length (=4*capacity) are powers
of 2. Therefore, we may use & to calculate positive mod, i.e.
{code}
final int hash = System.identityHashCode(k);
final int numEntries = buffer.length >> 1;
final int index = hash & (numEntries - 1);
{code}
> JAVA specification for hashcode does not enforce it to be non-negative, but
> IdentityHashStore assumes System.identityHashCode() is non-negative
> ---
>
> Key: HADOOP-11221
> URL: https://issues.apache.org/jira/browse/HADOOP-11221
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 2.4.1
>Reporter: Jinghui Wang
>Assignee: Jinghui Wang
> Attachments: HADOOP-11221.patch, HADOOP-11221.v1.patch
>
>
> The following code snippet shows that IdentityHashStore assumes the hashCode
> is always non-negative.
> {code:borderStyle=solid}
>private void putInternal(Object k, Object v) {
> int hash = System.identityHashCode(k);
> final int numEntries = buffer.length / 2;
> int index = hash % numEntries;
>...
>}
>
> private int getElementIndex(K k) {
> ...
> final int numEntries = buffer.length / 2;
> int hash = System.identityHashCode(k);
> int index = hash % numEntries;
> int firstIndex = index;
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11221) JAVA specification for hashcode does not enforce it to be non-negative, but IdentityHashStore assumes System.identityHashCode() is non-negative
[
https://issues.apache.org/jira/browse/HADOOP-11221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14182305#comment-14182305
]
Hadoop QA commented on HADOOP-11221:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12676803/HADOOP-11221.v1.patch
against trunk revision db45f04.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/4943//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/4943//console
This message is automatically generated.
> JAVA specification for hashcode does not enforce it to be non-negative, but
> IdentityHashStore assumes System.identityHashCode() is non-negative
> ---
>
> Key: HADOOP-11221
> URL: https://issues.apache.org/jira/browse/HADOOP-11221
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 2.4.1
>Reporter: Jinghui Wang
>Assignee: Jinghui Wang
> Attachments: HADOOP-11221.patch, HADOOP-11221.v1.patch
>
>
> The following code snippet shows that IdentityHashStore assumes the hashCode
> is always non-negative.
> {code:borderStyle=solid}
>private void putInternal(Object k, Object v) {
> int hash = System.identityHashCode(k);
> final int numEntries = buffer.length / 2;
> int index = hash % numEntries;
>...
>}
>
> private int getElementIndex(K k) {
> ...
> final int numEntries = buffer.length / 2;
> int hash = System.identityHashCode(k);
> int index = hash % numEntries;
> int firstIndex = index;
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11221) JAVA specification for hashcode does not enforce it to be non-negative, but IdentityHashStore assumes System.identityHashCode() is non-negative
[
https://issues.apache.org/jira/browse/HADOOP-11221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14182281#comment-14182281
]
Chris Douglas commented on HADOOP-11221:
How about {{hash = System.identityHashCode(k) & Integer.MAX_VALUE;}} ?
> JAVA specification for hashcode does not enforce it to be non-negative, but
> IdentityHashStore assumes System.identityHashCode() is non-negative
> ---
>
> Key: HADOOP-11221
> URL: https://issues.apache.org/jira/browse/HADOOP-11221
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 2.4.1
>Reporter: Jinghui Wang
>Assignee: Jinghui Wang
> Attachments: HADOOP-11221.patch, HADOOP-11221.v1.patch
>
>
> The following code snippet shows that IdentityHashStore assumes the hashCode
> is always non-negative.
> {code:borderStyle=solid}
>private void putInternal(Object k, Object v) {
> int hash = System.identityHashCode(k);
> final int numEntries = buffer.length / 2;
> int index = hash % numEntries;
>...
>}
>
> private int getElementIndex(K k) {
> ...
> final int numEntries = buffer.length / 2;
> int hash = System.identityHashCode(k);
> int index = hash % numEntries;
> int firstIndex = index;
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11221) JAVA specification for hashcode does not enforce it to be non-negative, but IdentityHashStore assumes System.identityHashCode() is non-negative
[
https://issues.apache.org/jira/browse/HADOOP-11221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14182279#comment-14182279
]
Jinghui Wang commented on HADOOP-11221:
---
Hi Chris,
Thanks for pointing out the edge case there. I have updated the patch.
> JAVA specification for hashcode does not enforce it to be non-negative, but
> IdentityHashStore assumes System.identityHashCode() is non-negative
> ---
>
> Key: HADOOP-11221
> URL: https://issues.apache.org/jira/browse/HADOOP-11221
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 2.4.1
>Reporter: Jinghui Wang
>Assignee: Jinghui Wang
> Attachments: HADOOP-11221.patch, HADOOP-11221.v1.patch
>
>
> The following code snippet shows that IdentityHashStore assumes the hashCode
> is always non-negative.
> {code:borderStyle=solid}
>private void putInternal(Object k, Object v) {
> int hash = System.identityHashCode(k);
> final int numEntries = buffer.length / 2;
> int index = hash % numEntries;
>...
>}
>
> private int getElementIndex(K k) {
> ...
> final int numEntries = buffer.length / 2;
> int hash = System.identityHashCode(k);
> int index = hash % numEntries;
> int firstIndex = index;
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11221) JAVA specification for hashcode does not enforce it to be non-negative, but IdentityHashStore assumes System.identityHashCode() is non-negative
[
https://issues.apache.org/jira/browse/HADOOP-11221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14181962#comment-14181962
]
Chris Douglas commented on HADOOP-11221:
{{Math.abs(Integer.MIN_VALUE)}} is still negative.
> JAVA specification for hashcode does not enforce it to be non-negative, but
> IdentityHashStore assumes System.identityHashCode() is non-negative
> ---
>
> Key: HADOOP-11221
> URL: https://issues.apache.org/jira/browse/HADOOP-11221
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 2.4.1
>Reporter: Jinghui Wang
>Assignee: Jinghui Wang
> Attachments: HADOOP-11221.patch
>
>
> The following code snippet shows that IdentityHashStore assumes the hashCode
> is always non-negative.
> {code:borderStyle=solid}
>private void putInternal(Object k, Object v) {
> int hash = System.identityHashCode(k);
> final int numEntries = buffer.length / 2;
> int index = hash % numEntries;
>...
>}
>
> private int getElementIndex(K k) {
> ...
> final int numEntries = buffer.length / 2;
> int hash = System.identityHashCode(k);
> int index = hash % numEntries;
> int firstIndex = index;
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11221) JAVA specification for hashcode does not enforce it to be non-negative, but IdentityHashStore assumes System.identityHashCode() is non-negative
[
https://issues.apache.org/jira/browse/HADOOP-11221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14181811#comment-14181811
]
Hadoop QA commented on HADOOP-11221:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12676663/HADOOP-11221.patch
against trunk revision d71d40a.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 3 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common:
org.apache.hadoop.metrics2.impl.TestMetricsSystemImpl
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/4938//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HADOOP-Build/4938//artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/4938//console
This message is automatically generated.
> JAVA specification for hashcode does not enforce it to be non-negative, but
> IdentityHashStore assumes System.identityHashCode() is non-negative
> ---
>
> Key: HADOOP-11221
> URL: https://issues.apache.org/jira/browse/HADOOP-11221
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 2.4.1
>Reporter: Jinghui Wang
>Assignee: Jinghui Wang
> Attachments: HADOOP-11221.patch
>
>
> The following code snippet shows that IdentityHashStore assumes the hashCode
> is always non-negative.
> {code:borderStyle=solid}
>private void putInternal(Object k, Object v) {
> int hash = System.identityHashCode(k);
> final int numEntries = buffer.length / 2;
> int index = hash % numEntries;
>...
>}
>
> private int getElementIndex(K k) {
> ...
> final int numEntries = buffer.length / 2;
> int hash = System.identityHashCode(k);
> int index = hash % numEntries;
> int firstIndex = index;
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11221) JAVA specification for hashcode does not enforce it to be non-negative, but IdentityHashStore assumes System.identityHashCode() is non-negative
[
https://issues.apache.org/jira/browse/HADOOP-11221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14181706#comment-14181706
]
Jinghui Wang commented on HADOOP-11221:
---
This causes failures with IBM JAVA where System.identityHashCode can return
negative values. Attached patch fixes the problem.
> JAVA specification for hashcode does not enforce it to be non-negative, but
> IdentityHashStore assumes System.identityHashCode() is non-negative
> ---
>
> Key: HADOOP-11221
> URL: https://issues.apache.org/jira/browse/HADOOP-11221
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
>Affects Versions: 2.4.1
>Reporter: Jinghui Wang
>Assignee: Jinghui Wang
>
> The following code snippet shows that IdentityHashStore assumes the hashCode
> is always non-negative.
> {code:borderStyle=solid}
>private void putInternal(Object k, Object v) {
> int hash = System.identityHashCode(k);
> final int numEntries = buffer.length / 2;
> int index = hash % numEntries;
>...
>}
>
> private int getElementIndex(K k) {
> ...
> final int numEntries = buffer.length / 2;
> int hash = System.identityHashCode(k);
> int index = hash % numEntries;
> int firstIndex = index;
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
