[jira] [Commented] (HBASE-12981) FSHLog: UNEXPECTED java.lang.ArrayIndexOutOfBoundsException: -4
[ https://issues.apache.org/jira/browse/HBASE-12981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14309957#comment-14309957 ] Lars Hofhansl commented on HBASE-12981: --- {code} + idx = Math.abs(idx); {code} Whoa, that can make things much worse, no? > FSHLog: UNEXPECTED java.lang.ArrayIndexOutOfBoundsException: -4 > --- > > Key: HBASE-12981 > URL: https://issues.apache.org/jira/browse/HBASE-12981 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.98.10 >Reporter: stack >Assignee: stack > Fix For: 0.98.11 > > Attachments: 12981.0.98.txt > > > A user reported the below. It happens after the RS has been running a while. > 015-01-20 22:33:23,031 ERROR org.apache.hadoop.hbase.regionserver.wal.FSHLog: > UNEXPECTED > java.lang.ArrayIndexOutOfBoundsException: -4 > at > org.apache.hadoop.hbase.regionserver.wal.FSHLog$AsyncWriter.run(FSHLog.java:1149) > at java.lang.Thread.run(Thread.java:745) > 2015-01-20 22:33:23,035 INFO org.apache.hadoop.hbase.regionserver.wal.FSHLog: > regionserver60020-WAL.AsyncWriter exiting > ## > Similarly on Node 23 - on 12-20-2014 05:13: > 2014-12-20 05:13:40,715 ERROR > org.apache.hadoop.hbase.regionserver.wal.FSHLog: UNEXPECTED > java.lang.ArrayIndexOutOfBoundsException: -3 > at > org.apache.hadoop.hbase.regionserver.wal.FSHLog$AsyncWriter.run(FSHLog.java:1149) > at java.lang.Thread.run(Thread.java:745) > ### > Looking in code, I can't see how this could come about other than our write > seqid ran over the top of a long (unlikely). I think this a 0.98 issue since > 1.0+ is different here. It does: > int index = Math.abs(this.syncRunnerIndex++) % this.syncRunners.length; > I'm going to add logging of the circumstance that produces a negative index > and then defense against our using negative indices; there could be more > going on in here, more than I can see. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12981) FSHLog: UNEXPECTED java.lang.ArrayIndexOutOfBoundsException: -4
[ https://issues.apache.org/jira/browse/HBASE-12981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310062#comment-14310062 ] Enis Soztutar commented on HBASE-12981: --- Could this be HBASE-11200 ? We have seen this in action using earlier 0.98 versions. > FSHLog: UNEXPECTED java.lang.ArrayIndexOutOfBoundsException: -4 > --- > > Key: HBASE-12981 > URL: https://issues.apache.org/jira/browse/HBASE-12981 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.98.10 >Reporter: stack >Assignee: stack > Fix For: 0.98.11 > > Attachments: 12981.0.98.txt > > > A user reported the below. It happens after the RS has been running a while. > 015-01-20 22:33:23,031 ERROR org.apache.hadoop.hbase.regionserver.wal.FSHLog: > UNEXPECTED > java.lang.ArrayIndexOutOfBoundsException: -4 > at > org.apache.hadoop.hbase.regionserver.wal.FSHLog$AsyncWriter.run(FSHLog.java:1149) > at java.lang.Thread.run(Thread.java:745) > 2015-01-20 22:33:23,035 INFO org.apache.hadoop.hbase.regionserver.wal.FSHLog: > regionserver60020-WAL.AsyncWriter exiting > ## > Similarly on Node 23 - on 12-20-2014 05:13: > 2014-12-20 05:13:40,715 ERROR > org.apache.hadoop.hbase.regionserver.wal.FSHLog: UNEXPECTED > java.lang.ArrayIndexOutOfBoundsException: -3 > at > org.apache.hadoop.hbase.regionserver.wal.FSHLog$AsyncWriter.run(FSHLog.java:1149) > at java.lang.Thread.run(Thread.java:745) > ### > Looking in code, I can't see how this could come about other than our write > seqid ran over the top of a long (unlikely). I think this a 0.98 issue since > 1.0+ is different here. It does: > int index = Math.abs(this.syncRunnerIndex++) % this.syncRunners.length; > I'm going to add logging of the circumstance that produces a negative index > and then defense against our using negative indices; there could be more > going on in here, more than I can see. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12981) FSHLog: UNEXPECTED java.lang.ArrayIndexOutOfBoundsException: -4
[ https://issues.apache.org/jira/browse/HBASE-12981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310096#comment-14310096 ] Sean Busbey commented on HBASE-12981: - {quote} {code} + idx = Math.abs(idx); {code} Whoa, that can make things much worse, no? {quote} At this point idx should be in the range (-asyncSyncers.length, asyncSyncers.length) so taking the absolute value should be okay. Usually I'd do a < 0 check and add the devisor, but I don't think it makes a difference unless this is a very hot code path. > FSHLog: UNEXPECTED java.lang.ArrayIndexOutOfBoundsException: -4 > --- > > Key: HBASE-12981 > URL: https://issues.apache.org/jira/browse/HBASE-12981 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.98.10 >Reporter: stack >Assignee: stack > Fix For: 0.98.11 > > Attachments: 12981.0.98.txt > > > A user reported the below. It happens after the RS has been running a while. > 015-01-20 22:33:23,031 ERROR org.apache.hadoop.hbase.regionserver.wal.FSHLog: > UNEXPECTED > java.lang.ArrayIndexOutOfBoundsException: -4 > at > org.apache.hadoop.hbase.regionserver.wal.FSHLog$AsyncWriter.run(FSHLog.java:1149) > at java.lang.Thread.run(Thread.java:745) > 2015-01-20 22:33:23,035 INFO org.apache.hadoop.hbase.regionserver.wal.FSHLog: > regionserver60020-WAL.AsyncWriter exiting > ## > Similarly on Node 23 - on 12-20-2014 05:13: > 2014-12-20 05:13:40,715 ERROR > org.apache.hadoop.hbase.regionserver.wal.FSHLog: UNEXPECTED > java.lang.ArrayIndexOutOfBoundsException: -3 > at > org.apache.hadoop.hbase.regionserver.wal.FSHLog$AsyncWriter.run(FSHLog.java:1149) > at java.lang.Thread.run(Thread.java:745) > ### > Looking in code, I can't see how this could come about other than our write > seqid ran over the top of a long (unlikely). I think this a 0.98 issue since > 1.0+ is different here. It does: > int index = Math.abs(this.syncRunnerIndex++) % this.syncRunners.length; > I'm going to add logging of the circumstance that produces a negative index > and then defense against our using negative indices; there could be more > going on in here, more than I can see. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12981) FSHLog: UNEXPECTED java.lang.ArrayIndexOutOfBoundsException: -4
[ https://issues.apache.org/jira/browse/HBASE-12981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310116#comment-14310116 ] stack commented on HBASE-12981: --- Thanks [~enis] I think you are right. I was looking at wrong 0.98 version. This is a patched 0.98.1 (CDH5.1.3). It doesn't have HBASE-11200. Let me close this as a duplicate (Thanks for reviews [~lhofhansl] and [~busbey] -- agree w/ Sean). > FSHLog: UNEXPECTED java.lang.ArrayIndexOutOfBoundsException: -4 > --- > > Key: HBASE-12981 > URL: https://issues.apache.org/jira/browse/HBASE-12981 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.98.10 >Reporter: stack >Assignee: stack > Fix For: 0.98.11 > > Attachments: 12981.0.98.txt > > > A user reported the below. It happens after the RS has been running a while. > 015-01-20 22:33:23,031 ERROR org.apache.hadoop.hbase.regionserver.wal.FSHLog: > UNEXPECTED > java.lang.ArrayIndexOutOfBoundsException: -4 > at > org.apache.hadoop.hbase.regionserver.wal.FSHLog$AsyncWriter.run(FSHLog.java:1149) > at java.lang.Thread.run(Thread.java:745) > 2015-01-20 22:33:23,035 INFO org.apache.hadoop.hbase.regionserver.wal.FSHLog: > regionserver60020-WAL.AsyncWriter exiting > ## > Similarly on Node 23 - on 12-20-2014 05:13: > 2014-12-20 05:13:40,715 ERROR > org.apache.hadoop.hbase.regionserver.wal.FSHLog: UNEXPECTED > java.lang.ArrayIndexOutOfBoundsException: -3 > at > org.apache.hadoop.hbase.regionserver.wal.FSHLog$AsyncWriter.run(FSHLog.java:1149) > at java.lang.Thread.run(Thread.java:745) > ### > Looking in code, I can't see how this could come about other than our write > seqid ran over the top of a long (unlikely). I think this a 0.98 issue since > 1.0+ is different here. It does: > int index = Math.abs(this.syncRunnerIndex++) % this.syncRunners.length; > I'm going to add logging of the circumstance that produces a negative index > and then defense against our using negative indices; there could be more > going on in here, more than I can see. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12981) FSHLog: UNEXPECTED java.lang.ArrayIndexOutOfBoundsException: -4
[ https://issues.apache.org/jira/browse/HBASE-12981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310226#comment-14310226 ] Hadoop QA commented on HBASE-12981: --- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12697132/12981.0.98.txt against 0.98 branch at commit 75148385ee5b2065992aea19a810436196576f20. ATTACHMENT ID: 12697132 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 23 warning messages. {color:green}+1 checkstyle{color}. The applied patch does not increase the total number of checkstyle errors {color:red}-1 findbugs{color}. The patch appears to introduce 5 new Findbugs (version 2.0.3) warnings. {color:red}-1 release audit{color}. The applied patch generated 42 release audit warnings (more than the master's current 0 warnings). {color:green}+1 lineLengths{color}. The patch does not introduce lines longer than 100 {color:green}+1 site{color}. The mvn site goal succeeds with this patch. {color:red}-1 core tests{color}. The patch failed these unit tests: {color:red}-1 core zombie tests{color}. There are 1 zombie test(s): at org.apache.hadoop.hbase.TestChoreService.testCorePoolIncrease(TestChoreService.java:395) Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/12722//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12722//artifact/patchprocess/patchReleaseAuditWarnings.txt Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12722//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12722//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12722//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12722//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12722//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12722//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12722//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12722//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12722//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12722//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12722//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/12722//artifact/patchprocess/checkstyle-aggregate.html Javadoc warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12722//artifact/patchprocess/patchJavadocWarnings.txt Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/12722//console This message is automatically generated. > FSHLog: UNEXPECTED java.lang.ArrayIndexOutOfBoundsException: -4 > --- > > Key: HBASE-12981 > URL: https://issues.apache.org/jira/browse/HBASE-12981 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.98.1 >Reporter: stack >Assignee: stack > Attachments: 12981.0.98.txt > > > A user reported the below. It happens after the RS has been running a while. > 015-01-20 22:33:23,031 ERROR org.apache.hadoop.hbase.regionserver.wal.FSHLog: > UNEXPECTED > java.lang.ArrayIndexOutOfBoundsException: -4 > at > org.apache.hadoop.hbase.regionserver.wal.FSHLog$AsyncWriter.run(FSHLog.java:1149) > at java.lang.