[jira] [Updated] (HBASE-14952) hbase-assembly source artifact has some incorrect modules
[
https://issues.apache.org/jira/browse/HBASE-14952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-14952:
Resolution: Fixed
Fix Version/s: 1.1.3
Status: Resolved (was: Patch Available)
> hbase-assembly source artifact has some incorrect modules
> -
>
> Key: HBASE-14952
> URL: https://issues.apache.org/jira/browse/HBASE-14952
> Project: HBase
> Issue Type: Bug
> Components: build, dependencies
>Affects Versions: 2.0.0, 1.2.0
>Reporter: Esteban Gutierrez
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3
>
> Attachments: HBASE-14952-branch-1-addendum.1.patch,
> HBASE-14952.1.patch, HBASE-14952.2.patch
>
>
> After generating a tarball we noticed:
> * that hbase-external-blockcache was missing.
> * that hbase-spark is missing
> * that there are duplicate hbase-shaded-{client,server} modules
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14994) Clean up some broken links and references to old APIs
[ https://issues.apache.org/jira/browse/HBASE-14994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-14994: Status: Patch Available (was: Open) > Clean up some broken links and references to old APIs > - > > Key: HBASE-14994 > URL: https://issues.apache.org/jira/browse/HBASE-14994 > Project: HBase > Issue Type: Task > Components: documentation >Affects Versions: 2.0.0 >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-14994.patch > > > Clean up broken links and references to old APIs found in > https://builds.apache.org/job/HBase%20Website%20Link%20Ckecker/14/artifact/link_report/errorX.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14952) hbase-assembly source artifact has some incorrect modules
[
https://issues.apache.org/jira/browse/HBASE-14952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-14952:
Description:
After generating a tarball we noticed:
* that hbase-external-blockcache was missing.
* that hbase-spark is missing
* that there are duplicate hbase-shaded-{client,server} modules
was:After generating a tarball we noticed that hbase-external-blockcache was
missing.
> hbase-assembly source artifact has some incorrect modules
> -
>
> Key: HBASE-14952
> URL: https://issues.apache.org/jira/browse/HBASE-14952
> Project: HBase
> Issue Type: Bug
> Components: build, dependencies
>Affects Versions: 2.0.0, 1.2.0
>Reporter: Esteban Gutierrez
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3
>
> Attachments: HBASE-14952-branch-1-addendum.1.patch,
> HBASE-14952.1.patch, HBASE-14952.2.patch
>
>
> After generating a tarball we noticed:
> * that hbase-external-blockcache was missing.
> * that hbase-spark is missing
> * that there are duplicate hbase-shaded-{client,server} modules
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14977) ChoreService.shutdown may result in ConcurrentModificationException
[ https://issues.apache.org/jira/browse/HBASE-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060666#comment-15060666 ] Matteo Bertozzi commented on HBASE-14977: - patch looks ok to me, but can't we just remove the synchronized from getName(), getPeriod(), getTimeUnit() and getInitialDelay()? they are private final anyway > ChoreService.shutdown may result in ConcurrentModificationException > --- > > Key: HBASE-14977 > URL: https://issues.apache.org/jira/browse/HBASE-14977 > Project: HBase > Issue Type: Bug >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4 > > Attachments: HBASE-14977-add.patch, HBASE-14977-v1.patch > > > As seen in this test: > https://builds.apache.org/job/HBase-1.3/jdk=latest1.8,label=Hadoop/425/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy-output.txt > We need to make shutdown method synchronized to avoid this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14952) hbase-assembly source artifact has some incorrect modules
[ https://issues.apache.org/jira/browse/HBASE-14952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14952: Summary: hbase-assembly source artifact has some incorrect modules (was: hbase-assembly has hbase-external-blockcache missing) > hbase-assembly source artifact has some incorrect modules > - > > Key: HBASE-14952 > URL: https://issues.apache.org/jira/browse/HBASE-14952 > Project: HBase > Issue Type: Bug > Components: build, dependencies >Affects Versions: 2.0.0, 1.2.0 >Reporter: Esteban Gutierrez >Assignee: Sean Busbey >Priority: Blocker > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3 > > Attachments: HBASE-14952-branch-1-addendum.1.patch, > HBASE-14952.1.patch, HBASE-14952.2.patch > > > After generating a tarball we noticed that hbase-external-blockcache was > missing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14964) Backport HBASE-14901 to brach-1 - There is duplicated code to create/manage encryption keys
[ https://issues.apache.org/jira/browse/HBASE-14964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060682#comment-15060682 ] Sean Busbey commented on HBASE-14964: - I'm testing this locally since hte build bot exploded. > Backport HBASE-14901 to brach-1 - There is duplicated code to create/manage > encryption keys > --- > > Key: HBASE-14964 > URL: https://issues.apache.org/jira/browse/HBASE-14964 > Project: HBase > Issue Type: Improvement > Components: encryption >Reporter: Nate Edel >Assignee: Nate Edel >Priority: Minor > Fix For: 1.2.0, 1.3.0 > > Attachments: HBASE-14964-branch-1.1.patch, HBASE-14964.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > There is duplicated code from MobUtils.createEncryptionContext in HStore, and > there is a subset of that code in HFileReaderImpl. > Refactored key selection > Moved both to EncryptionUtil.java > Can't figure out how to write a unit test for this, but there's no new code > just refactoring. > A lot of the Mob stuff hasn't been backported, so this is a very small patch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14984) Allow memcached block cache to set optimze to false
[ https://issues.apache.org/jira/browse/HBASE-14984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060702#comment-15060702 ] Sean Busbey commented on HBASE-14984: - is this worth a release note? > Allow memcached block cache to set optimze to false > --- > > Key: HBASE-14984 > URL: https://issues.apache.org/jira/browse/HBASE-14984 > Project: HBase > Issue Type: Improvement > Components: BlockCache >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14984.patch > > > In order to keep latency consistent it might not be good to allow the spy > memcached client to optimize. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14994) Clean up some broken links and references to old APIs
[ https://issues.apache.org/jira/browse/HBASE-14994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060701#comment-15060701 ] Sean Busbey commented on HBASE-14994: - +1 > Clean up some broken links and references to old APIs > - > > Key: HBASE-14994 > URL: https://issues.apache.org/jira/browse/HBASE-14994 > Project: HBase > Issue Type: Task > Components: documentation >Affects Versions: 2.0.0 >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-14994.patch > > > Clean up broken links and references to old APIs found in > https://builds.apache.org/job/HBase%20Website%20Link%20Ckecker/14/artifact/link_report/errorX.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14964) Backport HBASE-14901 to brach-1 - There is duplicated code to create/manage encryption keys
[ https://issues.apache.org/jira/browse/HBASE-14964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060731#comment-15060731 ] Sean Busbey commented on HBASE-14964: - (note that the buildbot should have said -1 because the build failed due to a flakey that I'm pretty sure is unrelated) > Backport HBASE-14901 to brach-1 - There is duplicated code to create/manage > encryption keys > --- > > Key: HBASE-14964 > URL: https://issues.apache.org/jira/browse/HBASE-14964 > Project: HBase > Issue Type: Improvement > Components: encryption >Reporter: Nate Edel >Assignee: Nate Edel >Priority: Minor > Fix For: 1.2.0, 1.3.0 > > Attachments: HBASE-14964-branch-1.1.patch, HBASE-14964.patch > > Original Estimate: 2h > Remaining Estimate: 2h > > There is duplicated code from MobUtils.createEncryptionContext in HStore, and > there is a subset of that code in HFileReaderImpl. > Refactored key selection > Moved both to EncryptionUtil.java > Can't figure out how to write a unit test for this, but there's no new code > just refactoring. > A lot of the Mob stuff hasn't been backported, so this is a very small patch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14974) Total number of Regions in Transition number on UI incorrect
[ https://issues.apache.org/jira/browse/HBASE-14974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-14974: Attachment: HBASE-14974.patch 2 liner > Total number of Regions in Transition number on UI incorrect > > > Key: HBASE-14974 > URL: https://issues.apache.org/jira/browse/HBASE-14974 > Project: HBase > Issue Type: Bug > Components: UI >Reporter: Elliott Clark >Priority: Trivial > Attachments: HBASE-14974.patch, Screen Shot 2015-12-14 at 11.34.14 > AM.png > > > Total number of Regions in Transition shows 100 when there are 100 or more > regions in transition. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14952) hbase-assembly source artifact has some incorrect modules
[
https://issues.apache.org/jira/browse/HBASE-14952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060742#comment-15060742
]
Hudson commented on HBASE-14952:
FAILURE: Integrated in HBase-1.2-IT #344 (See
[https://builds.apache.org/job/HBase-1.2-IT/344/])
HBASE-14952 ensure correct components in source assembly. (busbey: rev
58ddbe863f3023248f2f2d65bf6a13feaf169f26)
* hbase-assembly/src/main/assembly/src.xml
> hbase-assembly source artifact has some incorrect modules
> -
>
> Key: HBASE-14952
> URL: https://issues.apache.org/jira/browse/HBASE-14952
> Project: HBase
> Issue Type: Bug
> Components: build, dependencies
>Affects Versions: 2.0.0, 1.2.0
>Reporter: Esteban Gutierrez
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3
>
> Attachments: HBASE-14952-branch-1-addendum.1.patch,
> HBASE-14952.1.patch, HBASE-14952.2.patch
>
>
> After generating a tarball we noticed:
> * that hbase-external-blockcache was missing.
> * that hbase-spark is missing
> * that there are duplicate hbase-shaded-{client,server} modules
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14984) Allow memcached block cache to set optimze to false
[ https://issues.apache.org/jira/browse/HBASE-14984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060743#comment-15060743 ] Hudson commented on HBASE-14984: FAILURE: Integrated in HBase-1.2-IT #344 (See [https://builds.apache.org/job/HBase-1.2-IT/344/]) HBASE-14984 Allow memcached block cache to set optimze to false (eclark: rev 0e52565d640b1fe3057722a3b0d9b8431c4564bb) * hbase-external-blockcache/src/main/java/org/apache/hadoop/hbase/io/hfile/MemcachedBlockCache.java > Allow memcached block cache to set optimze to false > --- > > Key: HBASE-14984 > URL: https://issues.apache.org/jira/browse/HBASE-14984 > Project: HBase > Issue Type: Improvement > Components: BlockCache >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14984.patch > > > In order to keep latency consistent it might not be good to allow the spy > memcached client to optimize. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14988) AssertionError: results.length when doing batch with large number of increment
[
https://issues.apache.org/jira/browse/HBASE-14988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-14988:
-
Summary: AssertionError: results.length when doing batch with large number
of increment (was: java.lang.reflect.InvocationTargetException when doing
batch with large number of increment)
> AssertionError: results.length when doing batch with large number of increment
> --
>
> Key: HBASE-14988
> URL: https://issues.apache.org/jira/browse/HBASE-14988
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Keith Lui
>
> Tried to do a 10K increment with batch. When using
> public void batch(List actions, Object[] results)
> got
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at com.simontuffs.onejar.Boot.run(Boot.java:340)
> at com.simontuffs.onejar.Boot.main(Boot.java:166)
> Caused by: java.lang.AssertionError: results.length
> at
> org.apache.hadoop.hbase.client.AsyncProcess$AsyncRequestFutureImpl.(AsyncProcess.java:763)
> at
> org.apache.hadoop.hbase.client.AsyncProcess.createAsyncRequestFuture(AsyncProcess.java:1578)
> at
> org.apache.hadoop.hbase.client.AsyncProcess.submitAll(AsyncProcess.java:554)
> at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:1000)
> Interestingly no exception is thrown when using the deprecated method
> public Object[] batch(List actions)
> This is a sample code in Scala:
> val table = connection.getTable(TableName.valueOf("test_table"))
> val increments = for (i <- 0 until 1) yield {
> val increment = new Increment(Random.nextDouble().toString.getBytes)
> increment.addColumn(family, qualifier, Random.nextLong())
> increment
> }
> table.batch(increments, Array.empty[Object])
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Reopened] (HBASE-14988) java.lang.reflect.InvocationTargetException when doing batch with large number of increment
[
https://issues.apache.org/jira/browse/HBASE-14988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk reopened HBASE-14988:
--
I read too quickly. This is apparently not related to the onejar reflection.
> java.lang.reflect.InvocationTargetException when doing batch with large
> number of increment
> ---
>
> Key: HBASE-14988
> URL: https://issues.apache.org/jira/browse/HBASE-14988
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Keith Lui
>
> Tried to do a 10K increment with batch. When using
> public void batch(List actions, Object[] results)
> got
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at com.simontuffs.onejar.Boot.run(Boot.java:340)
> at com.simontuffs.onejar.Boot.main(Boot.java:166)
> Caused by: java.lang.AssertionError: results.length
> at
> org.apache.hadoop.hbase.client.AsyncProcess$AsyncRequestFutureImpl.(AsyncProcess.java:763)
> at
> org.apache.hadoop.hbase.client.AsyncProcess.createAsyncRequestFuture(AsyncProcess.java:1578)
> at
> org.apache.hadoop.hbase.client.AsyncProcess.submitAll(AsyncProcess.java:554)
> at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:1000)
> Interestingly no exception is thrown when using the deprecated method
> public Object[] batch(List actions)
> This is a sample code in Scala:
> val table = connection.getTable(TableName.valueOf("test_table"))
> val increments = for (i <- 0 until 1) yield {
> val increment = new Increment(Random.nextDouble().toString.getBytes)
> increment.addColumn(family, qualifier, Random.nextLong())
> increment
> }
> table.batch(increments, Array.empty[Object])
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14977) ChoreService.shutdown may result in ConcurrentModificationException
[ https://issues.apache.org/jira/browse/HBASE-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060703#comment-15060703 ] Vladimir Rodionov commented on HBASE-14977: --- [~enis] can you commit the addendum? Thanks. > ChoreService.shutdown may result in ConcurrentModificationException > --- > > Key: HBASE-14977 > URL: https://issues.apache.org/jira/browse/HBASE-14977 > Project: HBase > Issue Type: Bug >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4 > > Attachments: HBASE-14977-add.patch, HBASE-14977-v1.patch > > > As seen in this test: > https://builds.apache.org/job/HBase-1.3/jdk=latest1.8,label=Hadoop/425/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy-output.txt > We need to make shutdown method synchronized to avoid this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14988) java.lang.reflect.InvocationTargetException when doing batch with large number of increment
[
https://issues.apache.org/jira/browse/HBASE-14988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060728#comment-15060728
]
Keith Lui commented on HBASE-14988:
---
It's not related to the onejar for sure. Same issue happens when running in
Apache Spark Streaming which tries to consume data from Kafka and writes to
HBase.
java.lang.AssertionError: results.length
at
org.apache.hadoop.hbase.client.AsyncProcess$AsyncRequestFutureImpl.(AsyncProcess.java:763)
at
org.apache.hadoop.hbase.client.AsyncProcess.createAsyncRequestFuture(AsyncProcess.java:1578)
at
org.apache.hadoop.hbase.client.AsyncProcess.submitAll(AsyncProcess.java:554)
at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:1000)
This error will not show up and run smoothly when using the deprecated batch
method. In addition, when the number of increments decrease to 1K, it will not
run into the same assertion error. So the issue should be related to the new
batch method when the numbers of increment is huge.
> java.lang.reflect.InvocationTargetException when doing batch with large
> number of increment
> ---
>
> Key: HBASE-14988
> URL: https://issues.apache.org/jira/browse/HBASE-14988
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Keith Lui
>
> Tried to do a 10K increment with batch. When using
> public void batch(List actions, Object[] results)
> got
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at com.simontuffs.onejar.Boot.run(Boot.java:340)
> at com.simontuffs.onejar.Boot.main(Boot.java:166)
> Caused by: java.lang.AssertionError: results.length
> at
> org.apache.hadoop.hbase.client.AsyncProcess$AsyncRequestFutureImpl.(AsyncProcess.java:763)
> at
> org.apache.hadoop.hbase.client.AsyncProcess.createAsyncRequestFuture(AsyncProcess.java:1578)
> at
> org.apache.hadoop.hbase.client.AsyncProcess.submitAll(AsyncProcess.java:554)
> at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:1000)
> Interestingly no exception is thrown when using the deprecated method
> public Object[] batch(List actions)
> This is a sample code in Scala:
> val table = connection.getTable(TableName.valueOf("test_table"))
> val increments = for (i <- 0 until 1) yield {
> val increment = new Increment(Random.nextDouble().toString.getBytes)
> increment.addColumn(family, qualifier, Random.nextLong())
> increment
> }
> table.batch(increments, Array.empty[Object])
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14977) ChoreService.shutdown may result in ConcurrentModificationException
[
https://issues.apache.org/jira/browse/HBASE-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060727#comment-15060727
]
Matteo Bertozzi commented on HBASE-14977:
-
https://docs.oracle.com/javase/specs/jls/se7/html/jls-13.html
{quote}Adding or deleting a synchronized modifier of a method does not break
compatibility with pre-existing binaries.{quote}
> ChoreService.shutdown may result in ConcurrentModificationException
> ---
>
> Key: HBASE-14977
> URL: https://issues.apache.org/jira/browse/HBASE-14977
> Project: HBase
> Issue Type: Bug
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Minor
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4
>
> Attachments: HBASE-14977-add.patch, HBASE-14977-v1.patch
>
>
> As seen in this test:
> https://builds.apache.org/job/HBase-1.3/jdk=latest1.8,label=Hadoop/425/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy-output.txt
> We need to make shutdown method synchronized to avoid this issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13605) RegionStates should not keep its list of dead servers
[
https://issues.apache.org/jira/browse/HBASE-13605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060733#comment-15060733
]
Enis Soztutar commented on HBASE-13605:
---
I don't think I will have the time to work on it given that this need a
redesign for the cluster membership and assignment protocol in branch-1. It
turned out that the fix in this area uncovers some previously hidden bugs. It
is hard to cover all the bases with todays code base.
I am inclined to wait for the proc based AM for the long term. I have not
checked the patches yet there to see whether cluster membership is correctly
handled there.
> RegionStates should not keep its list of dead servers
> -
>
> Key: HBASE-13605
> URL: https://issues.apache.org/jira/browse/HBASE-13605
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
>Priority: Critical
> Fix For: 2.0.0, 1.3.0
>
> Attachments: hbase-13605_v1.patch, hbase-13605_v3-branch-1.1.patch,
> hbase-13605_v4-branch-1.1.patch, hbase-13605_v4-master.patch
>
>
> As mentioned in
> https://issues.apache.org/jira/browse/HBASE-9514?focusedCommentId=13769761=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13769761
> and HBASE-12844 we should have only 1 source of cluster membership.
> The list of dead server and RegionStates doing it's own liveliness check
> (ServerManager.isServerReachable()) has caused an assignment problem again in
> a test cluster where the region states "thinks" that the server is dead and
> SSH will handle the region assignment. However the RS is not dead at all,
> living happily, and never gets zk expiry or YouAreDeadException or anything.
> This leaves the list of regions unassigned in OFFLINE state.
> master assigning the region:
> {code}
> 15-04-20 09:02:25,780 DEBUG [AM.ZK.Worker-pool3-t330] master.RegionStates:
> Onlined 77dddcd50c22e56bfff133c0e1f9165b on
> os-amb-r6-us-1429512014-hbase4-6.novalocal,16020,1429520535268 {ENCODED =>
> 77dddcd50c
> {code}
> Master then disabled the table, and unassigned the region:
> {code}
> 2015-04-20 09:02:27,158 WARN [ProcedureExecutorThread-1]
> zookeeper.ZKTableStateManager: Moving table loadtest_d1 state from DISABLING
> to DISABLING
> Starting unassign of
> loadtest_d1,,1429520544378.77dddcd50c22e56bfff133c0e1f9165b. (offlining),
> current state: {77dddcd50c22e56bfff133c0e1f9165b state=OPEN,
> ts=1429520545780,
> server=os-amb-r6-us-1429512014-hbase4-6.novalocal,16020,1429520535268}
> bleProcedure$BulkDisabler-0] master.AssignmentManager: Sent CLOSE to
> os-amb-r6-us-1429512014-hbase4-6.novalocal,16020,1429520535268 for region
> loadtest_d1,,1429520544378.77dddcd50c22e56bfff133c0e1f9165b.
> 2015-04-20 09:02:27,414 INFO [AM.ZK.Worker-pool3-t316] master.RegionStates:
> Offlined 77dddcd50c22e56bfff133c0e1f9165b from
> os-amb-r6-us-1429512014-hbase4-6.novalocal,16020,1429520535268
> {code}
> On table re-enable, AM does not assign the region:
> {code}
> 2015-04-20 09:02:30,415 INFO [ProcedureExecutorThread-3]
> balancer.BaseLoadBalancer: Reassigned 25 regions. 25 retained the pre-restart
> assignment.ยท
> 2015-04-20 09:02:30,415 INFO [ProcedureExecutorThread-3]
> procedure.EnableTableProcedure: Bulk assigning 25 region(s) across 5
> server(s), retainAssignment=true
> l,16000,1429515659726-GeneralBulkAssigner-4] master.RegionStates: Couldn't
> reach online server
> os-amb-r6-us-1429512014-hbase4-6.novalocal,16020,1429520535268
> l,16000,1429515659726-GeneralBulkAssigner-4] master.AssignmentManager:
> Updating the state to OFFLINE to allow to be reassigned by SSH
> nmentManager: Skip assigning
> loadtest_d1,,1429520544378.77dddcd50c22e56bfff133c0e1f9165b., it is on a dead
> but not processed yet server:
> os-amb-r6-us-1429512014-hbase4-6.novalocal,16020,1429520535268
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14952) hbase-assembly has hbase-external-blockcache missing
[ https://issues.apache.org/jira/browse/HBASE-14952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060310#comment-15060310 ] Sean Busbey commented on HBASE-14952: - (the above looks an awful like a precommit check we could add.) > hbase-assembly has hbase-external-blockcache missing > > > Key: HBASE-14952 > URL: https://issues.apache.org/jira/browse/HBASE-14952 > Project: HBase > Issue Type: Bug > Components: build, dependencies >Affects Versions: 2.0.0, 1.2.0 >Reporter: Esteban Gutierrez >Assignee: Sean Busbey >Priority: Blocker > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14952.1.patch, HBASE-14952.2.patch > > > After generating a tarball we noticed that hbase-external-blockcache was > missing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14803) Add some debug logs to StoreFileScanner
[ https://issues.apache.org/jira/browse/HBASE-14803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14803: Fix Version/s: (was: 1.2.0) 1.3.0 2.0.0 Status: Open (was: Patch Available) moving out of patch available pending update. > Add some debug logs to StoreFileScanner > --- > > Key: HBASE-14803 > URL: https://issues.apache.org/jira/browse/HBASE-14803 > Project: HBase > Issue Type: Bug >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari >Priority: Minor > Labels: beginner > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-14803.v0-trunk.patch, HBASE-14803.v1-trunk.patch, > HBASE-14803.v2-trunk.patch, HBASE-14803.v3-trunk.patch, > HBASE-14803.v4-trunk.patch, HBASE-14803.v4-trunk.patch, > HBASE-14803.v5-trunk.patch > > > To validate some behaviors I had to add some logs into StoreFileScanner. > I think it can be interesting for other people looking for debuging. So > sharing the modifications here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-14943) Procedure-V2: Deadlock in WALProcedureStore
[ https://issues.apache.org/jira/browse/HBASE-14943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Yuan Jiang resolved HBASE-14943. Resolution: Duplicate > Procedure-V2: Deadlock in WALProcedureStore > --- > > Key: HBASE-14943 > URL: https://issues.apache.org/jira/browse/HBASE-14943 > Project: HBase > Issue Type: Bug > Components: proc-v2 >Affects Versions: 2.0.0, 1.2.0, 1.1.3 >Reporter: Ashu Pachauri >Assignee: Matteo Bertozzi >Priority: Blocker > Fix For: 2.0.0, 1.2.0, 1.1.3 > > Attachments: HBASE-14943-v0.patch, deadlock-jstack.out > > > Found deadlock in WALProcedureStore between the WALProcedureSyncThread and > WALProcedureStore#update/delete/insert API which is used by the > ProcedureExecutor. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (HBASE-14943) Procedure-V2: Deadlock in WALProcedureStore
[ https://issues.apache.org/jira/browse/HBASE-14943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen Yuan Jiang reopened HBASE-14943: > Procedure-V2: Deadlock in WALProcedureStore > --- > > Key: HBASE-14943 > URL: https://issues.apache.org/jira/browse/HBASE-14943 > Project: HBase > Issue Type: Bug > Components: proc-v2 >Affects Versions: 2.0.0, 1.2.0, 1.1.3 >Reporter: Ashu Pachauri >Assignee: Matteo Bertozzi >Priority: Blocker > Fix For: 2.0.0, 1.2.0, 1.1.3 > > Attachments: HBASE-14943-v0.patch, deadlock-jstack.out > > > Found deadlock in WALProcedureStore between the WALProcedureSyncThread and > WALProcedureStore#update/delete/insert API which is used by the > ProcedureExecutor. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060385#comment-15060385 ] Francis Liu commented on HBASE-6721: I pretty much did that with TestShellGroups class, I just used the 'shell.test' property to define what test to be run intead of shell.test.include/shell.test.exclude. The reason the TestShell was failing was because you copied the TestShell class and did not exclude group_shell_test.rb similar to how replication_admin_test.rb is set. So we will need to update TestShell once it's restored so it doesn't fail because it tries to run group_shell_test.rb. > RegionServer Group based Assignment > --- > > Key: HBASE-6721 > URL: https://issues.apache.org/jira/browse/HBASE-6721 > Project: HBase > Issue Type: New Feature >Reporter: Francis Liu >Assignee: Francis Liu > Labels: hbase-6721 > Attachments: 6721-master-webUI.patch, HBASE-6721 > GroupBasedLoadBalancer Sequence Diagram.xml, HBASE-6721-DesigDoc.pdf, > HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, > HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, HBASE-6721_11.patch, > HBASE-6721_12.patch, HBASE-6721_13.patch, HBASE-6721_14.patch, > HBASE-6721_15.patch, HBASE-6721_8.patch, HBASE-6721_9.patch, > HBASE-6721_9.patch, HBASE-6721_94.patch, HBASE-6721_94.patch, > HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch, > HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch, > HBASE-6721_94_7.patch, HBASE-6721_98_1.patch, HBASE-6721_98_2.patch, > HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk.patch, > HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch, > HBASE-6721_trunk2.patch, balanceCluster Sequence Diagram.svg, > hbase-6721-v15-branch-1.1.patch, hbase-6721-v16.patch, hbase-6721-v17.patch, > hbase-6721-v18.patch, hbase-6721-v19.patch, hbase-6721-v20.patch, > immediateAssignments Sequence Diagram.svg, randomAssignment Sequence > Diagram.svg, retainAssignment Sequence Diagram.svg, roundRobinAssignment > Sequence Diagram.svg > > > In multi-tenant deployments of HBase, it is likely that a RegionServer will > be serving out regions from a number of different tables owned by various > client applications. Being able to group a subset of running RegionServers > and assign specific tables to it, provides a client application a level of > isolation and resource allocation. > The proposal essentially is to have an AssignmentManager which is aware of > RegionServer groups and assigns tables to region servers based on groupings. > Load balancing will occur on a per group basis as well. > This is essentially a simplification of the approach taken in HBASE-4120. See > attached document. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14984) Allow memcached block cache to set optimze to false
[ https://issues.apache.org/jira/browse/HBASE-14984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060491#comment-15060491 ] Elliott Clark commented on HBASE-14984: --- Yeah that's been happening lately. Thanks > Allow memcached block cache to set optimze to false > --- > > Key: HBASE-14984 > URL: https://issues.apache.org/jira/browse/HBASE-14984 > Project: HBase > Issue Type: Improvement > Components: BlockCache >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14984.patch > > > In order to keep latency consistent it might not be good to allow the spy > memcached client to optimize. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14978) Don't allow Multi to retain too many blocks
[ https://issues.apache.org/jira/browse/HBASE-14978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060527#comment-15060527 ] Elliott Clark commented on HBASE-14978: --- Any thoughts on the last patch? It tries it's best to keep track of off heap blocks. The javadoc warning seems to be coming from somewhere else. > Don't allow Multi to retain too many blocks > --- > > Key: HBASE-14978 > URL: https://issues.apache.org/jira/browse/HBASE-14978 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Critical > Attachments: HBASE-14978-v1.patch, HBASE-14978-v2.patch, > HBASE-14978-v3.patch, HBASE-14978.patch > > > Scans and Multi's have limits on the total size of cells that can be > returned. However if those requests are not all pointing at the same blocks > then the KeyValues can keep alive a lot more data than their size. > Take the following example: > A multi with a list of 1 gets to a fat row. Each column being returned in > in a different block. Each column is small 32 bytes or so. > So the total cell size will be 32 * 1 = ~320kb. However if each block is > 128k then total retained heap size will be almost 2gigs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14863) Add missing test/resources/log4j files in hbase modules
[ https://issues.apache.org/jira/browse/HBASE-14863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14863: Resolution: Fixed Status: Resolved (was: Patch Available) addendum pushed. thanks for the quick turn around! > Add missing test/resources/log4j files in hbase modules > --- > > Key: HBASE-14863 > URL: https://issues.apache.org/jira/browse/HBASE-14863 > Project: HBase > Issue Type: Sub-task > Components: test >Affects Versions: 2.0.0, 1.2.0, 1.1.2 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Trivial > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17 > > Attachments: HBASE-14863-0.98-addendum.1.patch, HBASE-14863-v0.patch > > > HBASE-14848 will investigate what is changed and why now we require to have > the log4j file in every submodule. > for now with this jira we are going to commit the log4j files in the missing > hbase modules -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14730) region server needs to log warnings when there are attributes configured for cells with hfile v2
[ https://issues.apache.org/jira/browse/HBASE-14730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060653#comment-15060653 ] huaxiang sun commented on HBASE-14730: -- [~anoop.hbase] I checked the master branch, the v2 writer is gone, HFileWriterImpl is v3. I tried to search the boolean you mentioned and could not find it. The only thing I found is that in OffheapKeyValue, there is a boolean hasTag, which makes getTagsLength() faster when there is no tag. Thanks. > region server needs to log warnings when there are attributes configured for > cells with hfile v2 > > > Key: HBASE-14730 > URL: https://issues.apache.org/jira/browse/HBASE-14730 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 1.0.0, 1.1.0, 1.2.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > Attachments: HBASE-14730-v001-1.2.0.patch, HBASE-14730-v001.patch > > > User can configure cell attributes with hfile.format.version 2. When cells > are flushed from memStore to Hfile, attributes are not saved. Warnings need > to be logged. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14983) Create metrics for per block type hit/miss ratios
[ https://issues.apache.org/jira/browse/HBASE-14983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-14983: -- Attachment: HBASE-14983-v2.patch Rebased. > Create metrics for per block type hit/miss ratios > - > > Key: HBASE-14983 > URL: https://issues.apache.org/jira/browse/HBASE-14983 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-14983-v1.patch, HBASE-14983-v2.patch, > HBASE-14983.patch, Screen Shot 2015-12-15 at 3.33.09 PM.png > > > Missing a root index block is worse than missing a data block. We should know > the difference -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14978) Don't allow Multi to retain too many blocks
[ https://issues.apache.org/jira/browse/HBASE-14978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060609#comment-15060609 ] Ted Yu commented on HBASE-14978: >From https://builds.apache.org/job/PreCommit-HBASE-Build/16862/console : Fetching https://builds.apache.org/job/PreCommit-HBASE-Build/16862/console Printing hanging tests Hanging test : org.apache.hadoop.hbase.master.TestSplitLogManager Printing Failing tests > Don't allow Multi to retain too many blocks > --- > > Key: HBASE-14978 > URL: https://issues.apache.org/jira/browse/HBASE-14978 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Critical > Attachments: HBASE-14978-v1.patch, HBASE-14978-v2.patch, > HBASE-14978-v3.patch, HBASE-14978.patch > > > Scans and Multi's have limits on the total size of cells that can be > returned. However if those requests are not all pointing at the same blocks > then the KeyValues can keep alive a lot more data than their size. > Take the following example: > A multi with a list of 1 gets to a fat row. Each column being returned in > in a different block. Each column is small 32 bytes or so. > So the total cell size will be 32 * 1 = ~320kb. However if each block is > 128k then total retained heap size will be almost 2gigs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14863) Add missing test/resources/log4j files in hbase modules
[ https://issues.apache.org/jira/browse/HBASE-14863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060646#comment-15060646 ] Matteo Bertozzi commented on HBASE-14863: - +1 on the addendum > Add missing test/resources/log4j files in hbase modules > --- > > Key: HBASE-14863 > URL: https://issues.apache.org/jira/browse/HBASE-14863 > Project: HBase > Issue Type: Sub-task > Components: test >Affects Versions: 2.0.0, 1.2.0, 1.1.2 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Trivial > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17 > > Attachments: HBASE-14863-0.98-addendum.1.patch, HBASE-14863-v0.patch > > > HBASE-14848 will investigate what is changed and why now we require to have > the log4j file in every submodule. > for now with this jira we are going to commit the log4j files in the missing > hbase modules -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14977) ChoreService.shutdown may result in ConcurrentModificationException
[ https://issues.apache.org/jira/browse/HBASE-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060739#comment-15060739 ] Enis Soztutar commented on HBASE-14977: --- Good to know. They are not client-public (@InterfaceAudience.Public) anyway. > ChoreService.shutdown may result in ConcurrentModificationException > --- > > Key: HBASE-14977 > URL: https://issues.apache.org/jira/browse/HBASE-14977 > Project: HBase > Issue Type: Bug >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4 > > Attachments: HBASE-14977-add.patch, HBASE-14977-v1.patch > > > As seen in this test: > https://builds.apache.org/job/HBase-1.3/jdk=latest1.8,label=Hadoop/425/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy-output.txt > We need to make shutdown method synchronized to avoid this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14822) Renewing leases of scanners doesn't work
[
https://issues.apache.org/jira/browse/HBASE-14822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060747#comment-15060747
]
Hadoop QA commented on HBASE-14822:
---
{color:red}-1 overall{color}.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16881//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16881//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16881//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16881//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16881//console
This message is automatically generated.
> Renewing leases of scanners doesn't work
>
>
> Key: HBASE-14822
> URL: https://issues.apache.org/jira/browse/HBASE-14822
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.14
>Reporter: Samarth Jain
>Assignee: Lars Hofhansl
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17, 1.0.4
>
> Attachments: 14822-0.98-v2.txt, 14822-0.98-v3.txt, 14822-0.98.txt,
> 14822-v3-0.98.txt, 14822-v4-0.98.txt, 14822-v4.txt, 14822-v5-0.98.txt,
> 14822-v5.txt, 14822.txt
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Reopened] (HBASE-14863) Add missing test/resources/log4j files in hbase modules
[ https://issues.apache.org/jira/browse/HBASE-14863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey reopened HBASE-14863: - In 0.98 this added files into the hbase-spark and hbase-procedure modules that don't exist in that branch. > Add missing test/resources/log4j files in hbase modules > --- > > Key: HBASE-14863 > URL: https://issues.apache.org/jira/browse/HBASE-14863 > Project: HBase > Issue Type: Sub-task > Components: test >Affects Versions: 2.0.0, 1.2.0, 1.1.2 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Trivial > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17 > > Attachments: HBASE-14863-v0.patch > > > HBASE-14848 will investigate what is changed and why now we require to have > the log4j file in every submodule. > for now with this jira we are going to commit the log4j files in the missing > hbase modules -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14863) Add missing test/resources/log4j files in hbase modules
[ https://issues.apache.org/jira/browse/HBASE-14863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14863: Attachment: HBASE-14863-0.98-addendum.1.patch \- addendum for 0.98 - removes files in modules that don't exist. > Add missing test/resources/log4j files in hbase modules > --- > > Key: HBASE-14863 > URL: https://issues.apache.org/jira/browse/HBASE-14863 > Project: HBase > Issue Type: Sub-task > Components: test >Affects Versions: 2.0.0, 1.2.0, 1.1.2 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Trivial > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17 > > Attachments: HBASE-14863-0.98-addendum.1.patch, HBASE-14863-v0.patch > > > HBASE-14848 will investigate what is changed and why now we require to have > the log4j file in every submodule. > for now with this jira we are going to commit the log4j files in the missing > hbase modules -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14994) Clean up some broken links and references to old APIs
[ https://issues.apache.org/jira/browse/HBASE-14994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-14994: Attachment: HBASE-14994.patch Ready for review. > Clean up some broken links and references to old APIs > - > > Key: HBASE-14994 > URL: https://issues.apache.org/jira/browse/HBASE-14994 > Project: HBase > Issue Type: Task > Components: documentation >Affects Versions: 2.0.0 >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-14994.patch > > > Clean up broken links and references to old APIs found in > https://builds.apache.org/job/HBase%20Website%20Link%20Ckecker/14/artifact/link_report/errorX.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14977) ChoreService.shutdown may result in ConcurrentModificationException
[ https://issues.apache.org/jira/browse/HBASE-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060716#comment-15060716 ] Vladimir Rodionov commented on HBASE-14977: --- [~mbertozzi] they are public. > ChoreService.shutdown may result in ConcurrentModificationException > --- > > Key: HBASE-14977 > URL: https://issues.apache.org/jira/browse/HBASE-14977 > Project: HBase > Issue Type: Bug >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4 > > Attachments: HBASE-14977-add.patch, HBASE-14977-v1.patch > > > As seen in this test: > https://builds.apache.org/job/HBase-1.3/jdk=latest1.8,label=Hadoop/425/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy-output.txt > We need to make shutdown method synchronized to avoid this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14863) Add missing test/resources/log4j files in hbase modules
[ https://issues.apache.org/jira/browse/HBASE-14863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14863: Status: Patch Available (was: Reopened) > Add missing test/resources/log4j files in hbase modules > --- > > Key: HBASE-14863 > URL: https://issues.apache.org/jira/browse/HBASE-14863 > Project: HBase > Issue Type: Sub-task > Components: test >Affects Versions: 1.1.2, 2.0.0, 1.2.0 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Trivial > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17 > > Attachments: HBASE-14863-0.98-addendum.1.patch, HBASE-14863-v0.patch > > > HBASE-14848 will investigate what is changed and why now we require to have > the log4j file in every submodule. > for now with this jira we are going to commit the log4j files in the missing > hbase modules -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14994) Clean up some broken links and references to old APIs
[ https://issues.apache.org/jira/browse/HBASE-14994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-14994: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.0.0 Status: Resolved (was: Patch Available) Pushed to master. > Clean up some broken links and references to old APIs > - > > Key: HBASE-14994 > URL: https://issues.apache.org/jira/browse/HBASE-14994 > Project: HBase > Issue Type: Task > Components: documentation >Affects Versions: 2.0.0 >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Fix For: 2.0.0 > > Attachments: HBASE-14994.patch > > > Clean up broken links and references to old APIs found in > https://builds.apache.org/job/HBase%20Website%20Link%20Ckecker/14/artifact/link_report/errorX.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14994) Clean up some broken links and references to old APIs
Misty Stanley-Jones created HBASE-14994: --- Summary: Clean up some broken links and references to old APIs Key: HBASE-14994 URL: https://issues.apache.org/jira/browse/HBASE-14994 Project: HBase Issue Type: Task Components: documentation Affects Versions: 2.0.0 Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones Clean up broken links and references to old APIs found in https://builds.apache.org/job/HBase%20Website%20Link%20Ckecker/14/artifact/link_report/errorX.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-14988) java.lang.reflect.InvocationTargetException when doing batch with large number of increment
[
https://issues.apache.org/jira/browse/HBASE-14988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk resolved HBASE-14988.
--
Resolution: Invalid
Do other operations work with this onejar thing? I suspect it's muddled with
the classloaders and is causing trouble.
Regardless, this looks like it warrants a discussion on the mailing list until
we know what need fixed in HBase. Please start a thread over on
u...@hbase.apache.org. We can reopen this ticket once we've identified what
product fix (if any) is required.
> java.lang.reflect.InvocationTargetException when doing batch with large
> number of increment
> ---
>
> Key: HBASE-14988
> URL: https://issues.apache.org/jira/browse/HBASE-14988
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Keith Lui
>
> Tried to do a 10K increment with batch. When using
> public void batch(List actions, Object[] results)
> got
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at com.simontuffs.onejar.Boot.run(Boot.java:340)
> at com.simontuffs.onejar.Boot.main(Boot.java:166)
> Caused by: java.lang.AssertionError: results.length
> at
> org.apache.hadoop.hbase.client.AsyncProcess$AsyncRequestFutureImpl.(AsyncProcess.java:763)
> at
> org.apache.hadoop.hbase.client.AsyncProcess.createAsyncRequestFuture(AsyncProcess.java:1578)
> at
> org.apache.hadoop.hbase.client.AsyncProcess.submitAll(AsyncProcess.java:554)
> at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:1000)
> Interestingly no exception is thrown when using the deprecated method
> public Object[] batch(List actions)
> This is a sample code in Scala:
> val table = connection.getTable(TableName.valueOf("test_table"))
> val increments = for (i <- 0 until 1) yield {
> val increment = new Increment(Random.nextDouble().toString.getBytes)
> increment.addColumn(family, qualifier, Random.nextLong())
> increment
> }
> table.batch(increments, Array.empty[Object])
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14977) ChoreService.shutdown may result in ConcurrentModificationException
[ https://issues.apache.org/jira/browse/HBASE-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060704#comment-15060704 ] Enis Soztutar commented on HBASE-14977: --- bq. That is because of a deadlock. ScheduledChore is heavily synchronized, even getName() is synchronized and toString(). I attached small addendum which fixes this issue. Yeah, I was afraid of a possible deadlock in introducing an extra condition. But it seemed (from my brief look at the code) that we do not hold any locks in the thread calling shutdown(). ScheduledChore.getName() is synchronized? That does not make sense. Is the deadlock between ScheduledChore.cancel() calling choreService.cancel() while holding chore lock, and ChoreService.shutdown() calling toString() while holding lock for the service. I think a better addendum would be to declared name, period, etc final, and un-synchronize the getName() and getPeriod() methods. > ChoreService.shutdown may result in ConcurrentModificationException > --- > > Key: HBASE-14977 > URL: https://issues.apache.org/jira/browse/HBASE-14977 > Project: HBase > Issue Type: Bug >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4 > > Attachments: HBASE-14977-add.patch, HBASE-14977-v1.patch > > > As seen in this test: > https://builds.apache.org/job/HBase-1.3/jdk=latest1.8,label=Hadoop/425/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy-output.txt > We need to make shutdown method synchronized to avoid this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14984) Allow memcached block cache to set optimze to false
[ https://issues.apache.org/jira/browse/HBASE-14984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060706#comment-15060706 ] Elliott Clark commented on HBASE-14984: --- Yep, let me get that. > Allow memcached block cache to set optimze to false > --- > > Key: HBASE-14984 > URL: https://issues.apache.org/jira/browse/HBASE-14984 > Project: HBase > Issue Type: Improvement > Components: BlockCache >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14984.patch > > > In order to keep latency consistent it might not be good to allow the spy > memcached client to optimize. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14952) hbase-assembly source artifact has some incorrect modules
[
https://issues.apache.org/jira/browse/HBASE-14952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060763#comment-15060763
]
Hadoop QA commented on HBASE-14952:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12778041/HBASE-14952.2.patch
against master branch at commit ca7a82fb5704af27459557168afc96e1193ac7cb.
ATTACHMENT ID: 12778041
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16880//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16880//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16880//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16880//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16880//console
This message is automatically generated.
> hbase-assembly source artifact has some incorrect modules
> -
>
> Key: HBASE-14952
> URL: https://issues.apache.org/jira/browse/HBASE-14952
> Project: HBase
> Issue Type: Bug
> Components: build, dependencies
>Affects Versions: 2.0.0, 1.2.0
>Reporter: Esteban Gutierrez
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3
>
> Attachments: HBASE-14952-branch-1-addendum.1.patch,
> HBASE-14952.1.patch, HBASE-14952.2.patch
>
>
> After generating a tarball we noticed:
> * that hbase-external-blockcache was missing.
> * that hbase-spark is missing
> * that there are duplicate hbase-shaded-{client,server} modules
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14952) hbase-assembly has hbase-external-blockcache missing
[
https://issues.apache.org/jira/browse/HBASE-14952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-14952:
Attachment: HBASE-14952.2.patch
-02
- add hbase-spark
- remove duplicate inclusion of hbase-shaded-{client,server}. They get
included properly due to hbase-shaded and adding them individual causes
additional modules at the top level
> hbase-assembly has hbase-external-blockcache missing
>
>
> Key: HBASE-14952
> URL: https://issues.apache.org/jira/browse/HBASE-14952
> Project: HBase
> Issue Type: Bug
> Components: build, dependencies
>Affects Versions: 2.0.0, 1.2.0
>Reporter: Esteban Gutierrez
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-14952.1.patch, HBASE-14952.2.patch
>
>
> After generating a tarball we noticed that hbase-external-blockcache was
> missing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14984) Allow memcached block cache to set optimze to false
[ https://issues.apache.org/jira/browse/HBASE-14984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-14984: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) > Allow memcached block cache to set optimze to false > --- > > Key: HBASE-14984 > URL: https://issues.apache.org/jira/browse/HBASE-14984 > Project: HBase > Issue Type: Improvement > Components: BlockCache >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14984.patch > > > In order to keep latency consistent it might not be good to allow the spy > memcached client to optimize. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14460) [Perf Regression] Merge of MVCC and SequenceId (HBASE-HBASE-8763) slowed Increments, CheckAndPuts, batch operations
[ https://issues.apache.org/jira/browse/HBASE-14460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060538#comment-15060538 ] Elliott Clark commented on HBASE-14460: --- I think we should just make it so that the read for increment/ check and put doesn't have to wait for the mvcc number at all. That should be easily do-able by moving the row locks around a little. That won't affect normal put performance since they are read locks only. That way once the increment has the write lock all outstanding edits are visible. This will keep us from having to lug around extra state per row, which would be a perf loss for all other writes. > [Perf Regression] Merge of MVCC and SequenceId (HBASE-HBASE-8763) slowed > Increments, CheckAndPuts, batch operations > --- > > Key: HBASE-14460 > URL: https://issues.apache.org/jira/browse/HBASE-14460 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: stack >Assignee: stack >Priority: Critical > Attachments: 0.94.test.patch, 0.98.test.patch, > 1.0.80.flamegraph-7932.svg, 14460.txt, 98.80.flamegraph-11428.svg, > HBASE-14460-discussion.patch, client.test.patch, > flamegraph-13120.svg.master.singlecell.svg, flamegraph-26636.094.100.svg, > flamegraph-28066.098.singlecell.svg, flamegraph-28767.098.100.svg, > flamegraph-31647.master.100.svg, flamegraph-9466.094.singlecell.svg, > m.test.patch, region_lock.png, testincrement.094.patch, > testincrement.098.patch, testincrement.master.patch > > > As reported by ้ดๆจไฟ่ฃ up on the mailing list -- see "Performance degradation > between CDH5.3.1(HBase0.98.6) and CDH5.4.5(HBase1.0.0)" -- our unification of > sequenceid and MVCC slows Increments (and other ops) as the mvcc needs to > 'catch up' to our current point before we can read the last Increment value > that we need to update. > We can say that our Increment is just done wrong, we should just be writing > Increments and summing on read, but checkAndPut as well as batching > operations have the same issue. Fix. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (HBASE-14977) ChoreService.shutdown may result in ConcurrentModificationException
[ https://issues.apache.org/jira/browse/HBASE-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov reopened HBASE-14977: --- Small addendum to fix failing TestChoreService test > ChoreService.shutdown may result in ConcurrentModificationException > --- > > Key: HBASE-14977 > URL: https://issues.apache.org/jira/browse/HBASE-14977 > Project: HBase > Issue Type: Bug >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4 > > Attachments: HBASE-14977-add.patch, HBASE-14977-v1.patch > > > As seen in this test: > https://builds.apache.org/job/HBase-1.3/jdk=latest1.8,label=Hadoop/425/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy-output.txt > We need to make shutdown method synchronized to avoid this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14977) ChoreService.shutdown may result in ConcurrentModificationException
[ https://issues.apache.org/jira/browse/HBASE-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-14977: -- Attachment: HBASE-14977-add.patch > ChoreService.shutdown may result in ConcurrentModificationException > --- > > Key: HBASE-14977 > URL: https://issues.apache.org/jira/browse/HBASE-14977 > Project: HBase > Issue Type: Bug >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4 > > Attachments: HBASE-14977-add.patch, HBASE-14977-v1.patch > > > As seen in this test: > https://builds.apache.org/job/HBase-1.3/jdk=latest1.8,label=Hadoop/425/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy-output.txt > We need to make shutdown method synchronized to avoid this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14838) Clarify that SimpleRegionNormalizer does not merge empty (<1MB) regions
[ https://issues.apache.org/jira/browse/HBASE-14838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060575#comment-15060575 ] Hudson commented on HBASE-14838: SUCCESS: Integrated in HBase-1.3-IT #377 (See [https://builds.apache.org/job/HBase-1.3-IT/377/]) HBASE-14838 Add clarification docs to SimpleRegionNormalizer. (busbey: rev 00805b399f0b953efe510ff3ef8584c39171019b) * hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java > Clarify that SimpleRegionNormalizer does not merge empty (<1MB) regions > --- > > Key: HBASE-14838 > URL: https://issues.apache.org/jira/browse/HBASE-14838 > Project: HBase > Issue Type: Bug > Components: documentation, regionserver >Affects Versions: 1.2.0 >Reporter: Romil Choksi >Assignee: Josh Elser >Priority: Trivial > Fix For: 2.0.0, 1.2.0 > > Attachments: HBASE-14838.patch > > > SImpleRegionNormalizer does not merge empty region of a table > Steps to repro: > - Create an empty table with few, say 5-6 regions without any data in any of > them > - Verify hbase:meta table to verify the regions for the table or check > HMaster UI > - Enable normalizer switch and normalization for this table > - Run normalizer, by 'normalize' command from hbase shell > - Verify the regions for table by scanning hbase:meta table or checking > HMaster web UI > The empty regions are not merged on running the region normalizer. This seems > to be an edge case with completely empty regions since the Normalizer checks > for: smallestRegion (in this case 0 size) + smallestNeighborOfSmallestRegion > (in this case 0 size) > avg region size (in this case 0 size) > thanks to [~elserj] for verifying this from the source code side -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14983) Create metrics for per block type hit/miss ratios
[ https://issues.apache.org/jira/browse/HBASE-14983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-14983: -- Status: Patch Available (was: Open) > Create metrics for per block type hit/miss ratios > - > > Key: HBASE-14983 > URL: https://issues.apache.org/jira/browse/HBASE-14983 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-14983-v1.patch, HBASE-14983.patch, Screen Shot > 2015-12-15 at 3.33.09 PM.png > > > Missing a root index block is worse than missing a data block. We should know > the difference -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13158) When client supports CellBlock, return the result Cells as controller payload for get(Get) API also
[
https://issues.apache.org/jira/browse/HBASE-13158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060571#comment-15060571
]
Hadoop QA commented on HBASE-13158:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12778015/HBASE-13158_V7.patch
against master branch at commit ca7a82fb5704af27459557168afc96e1193ac7cb.
ATTACHMENT ID: 12778015
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ "\030\001 \002(\t\022\013\n\003url\030\002
\002(\t\022\020\n\010revision\030\003 \002(\t\022\014\n" +
+ new java.lang.String[] { "Version", "Url", "Revision", "User",
"Date", "SrcChecksum", "VersionMajor", "VersionMinor", });
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16878//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16878//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16878//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16878//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16878//console
This message is automatically generated.
> When client supports CellBlock, return the result Cells as controller payload
> for get(Get) API also
> ---
>
> Key: HBASE-13158
> URL: https://issues.apache.org/jira/browse/HBASE-13158
> Project: HBase
> Issue Type: Improvement
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
> Fix For: 2.0.0, 1.3.0
>
> Attachments: 13158v4.suggestion.txt, HBASE-13158.patch,
> HBASE-13158_V2.patch, HBASE-13158_V3.patch, HBASE-13158_V4.patch,
> HBASE-13158_V6.patch, HBASE-13158_V6.patch, HBASE-13158_V6.patch,
> HBASE-13158_V6.patch, HBASE-13158_V6.patch, HBASE-13158_V7.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14952) hbase-assembly has hbase-external-blockcache missing
[
https://issues.apache.org/jira/browse/HBASE-14952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060589#comment-15060589
]
Hadoop QA commented on HBASE-14952:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12778070/HBASE-14952-branch-1-addendum.1.patch
against branch-1 branch at commit 7099ab4c56518024a26f5c499a9a1f2ea6bacd6e.
ATTACHMENT ID: 12778070
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16883//console
This message is automatically generated.
> hbase-assembly has hbase-external-blockcache missing
>
>
> Key: HBASE-14952
> URL: https://issues.apache.org/jira/browse/HBASE-14952
> Project: HBase
> Issue Type: Bug
> Components: build, dependencies
>Affects Versions: 2.0.0, 1.2.0
>Reporter: Esteban Gutierrez
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-14952-branch-1-addendum.1.patch,
> HBASE-14952.1.patch, HBASE-14952.2.patch
>
>
> After generating a tarball we noticed that hbase-external-blockcache was
> missing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14849) Add option to set block cache to false on SparkSQL executions
[ https://issues.apache.org/jira/browse/HBASE-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060290#comment-15060290 ] stack commented on HBASE-14849: --- [~zhanzhang] If you click on the javadoc warnings link above and dig down passing out the warnings about our use of unsafe features, you'll find this: 1 warning [WARNING] Javadoc Warnings [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java:430: warning - @param argument "hfilesDir" is not a parameter name. [INFO] It may not be because of you but a warning left over by anothers' commit. > Add option to set block cache to false on SparkSQL executions > - > > Key: HBASE-14849 > URL: https://issues.apache.org/jira/browse/HBASE-14849 > Project: HBase > Issue Type: New Feature > Components: spark >Reporter: Ted Malaska >Assignee: Zhan Zhang > Attachments: HBASE-14849.patch > > > I was working at a client with a ported down version of the Spark module for > HBase and realized we didn't add an option to turn of block cache for the > scans. > At the client I just disabled all caching with Spark SQL, this is an easy but > very impactful fix. > The fix for this patch will make this configurable -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14952) hbase-assembly has hbase-external-blockcache missing
[ https://issues.apache.org/jira/browse/HBASE-14952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14952: Status: In Progress (was: Patch Available) also missing hbase-spark. updating. > hbase-assembly has hbase-external-blockcache missing > > > Key: HBASE-14952 > URL: https://issues.apache.org/jira/browse/HBASE-14952 > Project: HBase > Issue Type: Bug > Components: build, dependencies >Affects Versions: 2.0.0, 1.2.0 >Reporter: Esteban Gutierrez >Assignee: Sean Busbey >Priority: Blocker > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14952.1.patch > > > After generating a tarball we noticed that hbase-external-blockcache was > missing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14943) Procedure-V2: Deadlock in WALProcedureStore
[ https://issues.apache.org/jira/browse/HBASE-14943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060352#comment-15060352 ] Stephen Yuan Jiang commented on HBASE-14943: Correction: commit in HBASE-14947 which has a completed fix (the patch in this JIRA, HBASE-14943 is not committed) > Procedure-V2: Deadlock in WALProcedureStore > --- > > Key: HBASE-14943 > URL: https://issues.apache.org/jira/browse/HBASE-14943 > Project: HBase > Issue Type: Bug > Components: proc-v2 >Affects Versions: 2.0.0, 1.2.0, 1.1.3 >Reporter: Ashu Pachauri >Assignee: Matteo Bertozzi >Priority: Blocker > Fix For: 2.0.0, 1.2.0, 1.1.3 > > Attachments: HBASE-14943-v0.patch, deadlock-jstack.out > > > Found deadlock in WALProcedureStore between the WALProcedureSyncThread and > WALProcedureStore#update/delete/insert API which is used by the > ProcedureExecutor. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14952) hbase-assembly has hbase-external-blockcache missing
[ https://issues.apache.org/jira/browse/HBASE-14952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060384#comment-15060384 ] stack commented on HBASE-14952: --- +1 > hbase-assembly has hbase-external-blockcache missing > > > Key: HBASE-14952 > URL: https://issues.apache.org/jira/browse/HBASE-14952 > Project: HBase > Issue Type: Bug > Components: build, dependencies >Affects Versions: 2.0.0, 1.2.0 >Reporter: Esteban Gutierrez >Assignee: Sean Busbey >Priority: Blocker > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14952.1.patch, HBASE-14952.2.patch > > > After generating a tarball we noticed that hbase-external-blockcache was > missing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14952) hbase-assembly has hbase-external-blockcache missing
[
https://issues.apache.org/jira/browse/HBASE-14952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060408#comment-15060408
]
Hadoop QA commented on HBASE-14952:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12778007/HBASE-14952.1.patch
against master branch at commit 60d33ce34191533bb858852584bd9bddfeb16a23.
ATTACHMENT ID: 12778007
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16875//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16875//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16875//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16875//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16875//console
This message is automatically generated.
> hbase-assembly has hbase-external-blockcache missing
>
>
> Key: HBASE-14952
> URL: https://issues.apache.org/jira/browse/HBASE-14952
> Project: HBase
> Issue Type: Bug
> Components: build, dependencies
>Affects Versions: 2.0.0, 1.2.0
>Reporter: Esteban Gutierrez
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-14952.1.patch, HBASE-14952.2.patch
>
>
> After generating a tarball we noticed that hbase-external-blockcache was
> missing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14977) ChoreService.shutdown may result in ConcurrentModificationException
[ https://issues.apache.org/jira/browse/HBASE-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060509#comment-15060509 ] Vladimir Rodionov commented on HBASE-14977: --- Looking into it. > ChoreService.shutdown may result in ConcurrentModificationException > --- > > Key: HBASE-14977 > URL: https://issues.apache.org/jira/browse/HBASE-14977 > Project: HBase > Issue Type: Bug >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4 > > Attachments: HBASE-14977-v1.patch > > > As seen in this test: > https://builds.apache.org/job/HBase-1.3/jdk=latest1.8,label=Hadoop/425/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy-output.txt > We need to make shutdown method synchronized to avoid this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14977) ChoreService.shutdown may result in ConcurrentModificationException
[ https://issues.apache.org/jira/browse/HBASE-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060554#comment-15060554 ] Vladimir Rodionov commented on HBASE-14977: --- That is because of a deadlock. ScheduledChore is heavily synchronized, even getName() is synchronized and toString(). I attached small addendum which fixes this issue. > ChoreService.shutdown may result in ConcurrentModificationException > --- > > Key: HBASE-14977 > URL: https://issues.apache.org/jira/browse/HBASE-14977 > Project: HBase > Issue Type: Bug >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4 > > Attachments: HBASE-14977-add.patch, HBASE-14977-v1.patch > > > As seen in this test: > https://builds.apache.org/job/HBase-1.3/jdk=latest1.8,label=Hadoop/425/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy-output.txt > We need to make shutdown method synchronized to avoid this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14807) TestWALLockup is flakey
[
https://issues.apache.org/jira/browse/HBASE-14807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-14807:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Will open new one if this shows up again.
> TestWALLockup is flakey
> ---
>
> Key: HBASE-14807
> URL: https://issues.apache.org/jira/browse/HBASE-14807
> Project: HBase
> Issue Type: Bug
> Components: flakey, test
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 14807.patch, 14807.second.attempt.txt,
> 14807.second.attempt.txt
>
>
> Fails frequently.
> Looks like this:
> {code}
> 2015-11-12 10:38:51,812 DEBUG [Time-limited test] regionserver.HRegion(3882):
> Found 0 recovered edits file(s) under
> /home/jenkins/jenkins-slave/workspace/HBase-1.2/jdk/latest1.7/label/Hadoop/hbase-server/target/test-data/8b8f8f12-1819-47e3-b1f1-8ffa789438ad/data/default/testLockupWhenSyncInMiddleOfZigZagSetup/c8694b53368f3301a8d370089120388d
> 2015-11-12 10:38:51,821 DEBUG [Time-limited test]
> regionserver.FlushLargeStoresPolicy(56):
> hbase.hregion.percolumnfamilyflush.size.lower.bound is not specified, use
> global config(16777216) instead
> 2015-11-12 10:38:51,880 DEBUG [Time-limited test] wal.WALSplitter(729): Wrote
> region
> seqId=/home/jenkins/jenkins-slave/workspace/HBase-1.2/jdk/latest1.7/label/Hadoop/hbase-server/target/test-data/8b8f8f12-1819-47e3-b1f1-8ffa789438ad/data/default/testLockupWhenSyncInMiddleOfZigZagSetup/c8694b53368f3301a8d370089120388d/recovered.edits/2.seqid
> to file, newSeqId=2, maxSeqId=0
> 2015-11-12 10:38:51,881 INFO [Time-limited test] regionserver.HRegion(868):
> Onlined c8694b53368f3301a8d370089120388d; next sequenceid=2
> 2015-11-12 10:38:51,994 ERROR [sync.1] wal.FSHLog$SyncRunner(1226): Error
> syncing, request close of WAL
> java.io.IOException: FAKE! Failed to replace a bad datanode...SYNC
> at
> org.apache.hadoop.hbase.regionserver.TestWALLockup$1DodgyFSLog$1.sync(TestWALLockup.java:162)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$SyncRunner.run(FSHLog.java:1222)
> at java.lang.Thread.run(Thread.java:745)
> 2015-11-12 10:38:51,997 DEBUG [Thread-4] regionserver.LogRoller(139): WAL
> roll requested
> 2015-11-12 10:38:52,019 DEBUG [flusher]
> regionserver.FlushLargeStoresPolicy(100): Since none of the CFs were above
> the size, flushing all.
> 2015-11-12 10:38:52,192 INFO [Thread-4]
> regionserver.TestWALLockup$1DodgyFSLog(129): LATCHED
> java.lang.InterruptedException: sleep interrupted
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.util.Threads.sleep(Threads.java:146)
> at
> org.apache.hadoop.hbase.regionserver.TestWALLockup.testLockupWhenSyncInMiddleOfZigZagSetup(TestWALLockup.java:245)
> 2015-11-12 10:39:18,609 INFO [main] regionserver.TestWALLockup(91): Cleaning
> test directory:
> /home/jenkins/jenkins-slave/workspace/HBase-1.2/jdk/latest1.7/label/Hadoop/hbase-server/target/test-data/8b8f8f12-1819-47e3-b1f1-8ffa789438ad
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> ... then times out after being locked up for 30 seconds. Writes 50+MB of
> logs while spinning.
> Reported as this:
> {code}
> ---
> Test set: org.apache.hadoop.hbase.regionserver.TestWALLockup
> ---
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 198.23 sec
> <<< FAILURE! - in org.apache.hadoop.hbase.regionserver.TestWALLockup
> testLockupWhenSyncInMiddleOfZigZagSetup(org.apache.hadoop.hbase.regionserver.TestWALLockup)
> Time elapsed: 0.049 sec <<< ERROR!
> org.junit.runners.model.TestTimedOutException: test timed
[jira] [Commented] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060429#comment-15060429 ] Francis Liu commented on HBASE-6721: I upload v20 on RB. Will address the checkstyle. > RegionServer Group based Assignment > --- > > Key: HBASE-6721 > URL: https://issues.apache.org/jira/browse/HBASE-6721 > Project: HBase > Issue Type: New Feature >Reporter: Francis Liu >Assignee: Francis Liu > Labels: hbase-6721 > Attachments: 6721-master-webUI.patch, HBASE-6721 > GroupBasedLoadBalancer Sequence Diagram.xml, HBASE-6721-DesigDoc.pdf, > HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, > HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, HBASE-6721_11.patch, > HBASE-6721_12.patch, HBASE-6721_13.patch, HBASE-6721_14.patch, > HBASE-6721_15.patch, HBASE-6721_8.patch, HBASE-6721_9.patch, > HBASE-6721_9.patch, HBASE-6721_94.patch, HBASE-6721_94.patch, > HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch, > HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch, > HBASE-6721_94_7.patch, HBASE-6721_98_1.patch, HBASE-6721_98_2.patch, > HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk.patch, > HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch, > HBASE-6721_trunk2.patch, balanceCluster Sequence Diagram.svg, > hbase-6721-v15-branch-1.1.patch, hbase-6721-v16.patch, hbase-6721-v17.patch, > hbase-6721-v18.patch, hbase-6721-v19.patch, hbase-6721-v20.patch, > immediateAssignments Sequence Diagram.svg, randomAssignment Sequence > Diagram.svg, retainAssignment Sequence Diagram.svg, roundRobinAssignment > Sequence Diagram.svg > > > In multi-tenant deployments of HBase, it is likely that a RegionServer will > be serving out regions from a number of different tables owned by various > client applications. Being able to group a subset of running RegionServers > and assign specific tables to it, provides a client application a level of > isolation and resource allocation. > The proposal essentially is to have an AssignmentManager which is aware of > RegionServer groups and assigns tables to region servers based on groupings. > Load balancing will occur on a per group basis as well. > This is essentially a simplification of the approach taken in HBASE-4120. See > attached document. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14977) ChoreService.shutdown may result in ConcurrentModificationException
[ https://issues.apache.org/jira/browse/HBASE-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060433#comment-15060433 ] stack commented on HBASE-14977: --- Master build just failed chore-related test. Is it because of this patch? https://builds.apache.org/job/HBase-Trunk_matrix/jdk=latest1.8,label=Hadoop/lastCompletedBuild/testReport/ Thanks. > ChoreService.shutdown may result in ConcurrentModificationException > --- > > Key: HBASE-14977 > URL: https://issues.apache.org/jira/browse/HBASE-14977 > Project: HBase > Issue Type: Bug >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4 > > Attachments: HBASE-14977-v1.patch > > > As seen in this test: > https://builds.apache.org/job/HBase-1.3/jdk=latest1.8,label=Hadoop/425/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy-output.txt > We need to make shutdown method synchronized to avoid this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14919) Infrastructure refactoring
[
https://issues.apache.org/jira/browse/HBASE-14919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060501#comment-15060501
]
Hadoop QA commented on HBASE-14919:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12777998/HBASE-14919-V03.patch
against master branch at commit 60d33ce34191533bb858852584bd9bddfeb16a23.
ATTACHMENT ID: 12777998
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 29 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
new checkstyle errors. Check build console for list of new errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16874//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16874//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16874//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16874//console
This message is automatically generated.
> Infrastructure refactoring
> --
>
> Key: HBASE-14919
> URL: https://issues.apache.org/jira/browse/HBASE-14919
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.0.0
>Reporter: Eshcar Hillel
>Assignee: Eshcar Hillel
> Attachments: HBASE-14919-V01.patch, HBASE-14919-V01.patch,
> HBASE-14919-V02.patch, HBASE-14919-V03.patch
>
>
> Refactoring the MemStore hierarchy, introducing segment (StoreSegment) as
> first-class citizen and decoupling memstore scanner from the memstore
> implementation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14952) hbase-assembly has hbase-external-blockcache missing
[ https://issues.apache.org/jira/browse/HBASE-14952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14952: Attachment: HBASE-14952-branch-1-addendum.1.patch - addendum I pushed for branch-1 and earlier. missed appending my commit to branch-1 before pushing. :/ > hbase-assembly has hbase-external-blockcache missing > > > Key: HBASE-14952 > URL: https://issues.apache.org/jira/browse/HBASE-14952 > Project: HBase > Issue Type: Bug > Components: build, dependencies >Affects Versions: 2.0.0, 1.2.0 >Reporter: Esteban Gutierrez >Assignee: Sean Busbey >Priority: Blocker > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14952-branch-1-addendum.1.patch, > HBASE-14952.1.patch, HBASE-14952.2.patch > > > After generating a tarball we noticed that hbase-external-blockcache was > missing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14952) hbase-assembly has hbase-external-blockcache missing
[
https://issues.apache.org/jira/browse/HBASE-14952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-14952:
Status: Patch Available (was: In Progress)
this version makes a source assembly in line with our prior releases.
{code}
hbase busbey$ tar -C /tmp/hbase-master-source/ -xzf
hbase-assembly/target/hbase-2.0.0-SNAPSHOT-src.tar.gz
hbase busbey$ git clean -xdf
Removing hbase-assembly/target/
hbase busbey$ diff -r . /tmp/hbase-master-source/hbase-2.0.0-SNAPSHOT/
Only in .: .arcconfig
Only in .: .git
Only in .: .gitattributes
Only in .: .gitignore
Only in .: hbase-native-client
{code}
> hbase-assembly has hbase-external-blockcache missing
>
>
> Key: HBASE-14952
> URL: https://issues.apache.org/jira/browse/HBASE-14952
> Project: HBase
> Issue Type: Bug
> Components: build, dependencies
>Affects Versions: 2.0.0, 1.2.0
>Reporter: Esteban Gutierrez
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-14952.1.patch, HBASE-14952.2.patch
>
>
> After generating a tarball we noticed that hbase-external-blockcache was
> missing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14968) ConcurrentModificationException in region close resulting in the region staying in closing state
[
https://issues.apache.org/jira/browse/HBASE-14968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060316#comment-15060316
]
Hudson commented on HBASE-14968:
SUCCESS: Integrated in HBase-1.0 #1123 (See
[https://builds.apache.org/job/HBase-1.0/1123/])
HBASE-14968 ConcurrentModificationException in region close resulting in (enis:
rev 6b8662c9c268dc17eb988e5b70d819d9b8fbb105)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/executor/TestExecutorService.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
> ConcurrentModificationException in region close resulting in the region
> staying in closing state
>
>
> Key: HBASE-14968
> URL: https://issues.apache.org/jira/browse/HBASE-14968
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4, 1.0.4
>
> Attachments: hbase-14968_v1.patch
>
>
> We have seen this in our tests. The region gets closed, but the region close
> handler gets an unexpected exception causing the region to stay in closing
> state until RS is restarted.
> The Phoenix and security coprocessors were loaded in that region. Here is the
> stack trace:
> {code}
> java.util.ConcurrentModificationException
> at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
> at java.util.ArrayList$Itr.next(ArrayList.java:851)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment.shutdown(CoprocessorHost.java:442)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionEnvironment.shutdown(RegionCoprocessorHost.java:155)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.shutdown(CoprocessorHost.java:272)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$5.postEnvCall(RegionCoprocessorHost.java:496)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1761)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postClose(RegionCoprocessorHost.java:489)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1502)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1349)
> at
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:138)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14984) Allow memcached block cache to set optimze to false
[ https://issues.apache.org/jira/browse/HBASE-14984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060334#comment-15060334 ] Sean Busbey commented on HBASE-14984: - +1. I can't figure out what the javadoc warning is supposed to be based on looking at the precommit artifacts and the patch. > Allow memcached block cache to set optimze to false > --- > > Key: HBASE-14984 > URL: https://issues.apache.org/jira/browse/HBASE-14984 > Project: HBase > Issue Type: Improvement > Components: BlockCache >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14984.patch > > > In order to keep latency consistent it might not be good to allow the spy > memcached client to optimize. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-6721) RegionServer Group based Assignment
[ https://issues.apache.org/jira/browse/HBASE-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060399#comment-15060399 ] Ted Yu commented on HBASE-6721: --- I see. TestShellGroups passes locally. Mind addressing checkstyle warnings ? Patch on review board is v18. Please upload v20 (or v21) there. Thanks > RegionServer Group based Assignment > --- > > Key: HBASE-6721 > URL: https://issues.apache.org/jira/browse/HBASE-6721 > Project: HBase > Issue Type: New Feature >Reporter: Francis Liu >Assignee: Francis Liu > Labels: hbase-6721 > Attachments: 6721-master-webUI.patch, HBASE-6721 > GroupBasedLoadBalancer Sequence Diagram.xml, HBASE-6721-DesigDoc.pdf, > HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, HBASE-6721-DesigDoc.pdf, > HBASE-6721_0.98_2.patch, HBASE-6721_10.patch, HBASE-6721_11.patch, > HBASE-6721_12.patch, HBASE-6721_13.patch, HBASE-6721_14.patch, > HBASE-6721_15.patch, HBASE-6721_8.patch, HBASE-6721_9.patch, > HBASE-6721_9.patch, HBASE-6721_94.patch, HBASE-6721_94.patch, > HBASE-6721_94_2.patch, HBASE-6721_94_3.patch, HBASE-6721_94_3.patch, > HBASE-6721_94_4.patch, HBASE-6721_94_5.patch, HBASE-6721_94_6.patch, > HBASE-6721_94_7.patch, HBASE-6721_98_1.patch, HBASE-6721_98_2.patch, > HBASE-6721_hbase-6721_addendum.patch, HBASE-6721_trunk.patch, > HBASE-6721_trunk.patch, HBASE-6721_trunk.patch, HBASE-6721_trunk1.patch, > HBASE-6721_trunk2.patch, balanceCluster Sequence Diagram.svg, > hbase-6721-v15-branch-1.1.patch, hbase-6721-v16.patch, hbase-6721-v17.patch, > hbase-6721-v18.patch, hbase-6721-v19.patch, hbase-6721-v20.patch, > immediateAssignments Sequence Diagram.svg, randomAssignment Sequence > Diagram.svg, retainAssignment Sequence Diagram.svg, roundRobinAssignment > Sequence Diagram.svg > > > In multi-tenant deployments of HBase, it is likely that a RegionServer will > be serving out regions from a number of different tables owned by various > client applications. Being able to group a subset of running RegionServers > and assign specific tables to it, provides a client application a level of > isolation and resource allocation. > The proposal essentially is to have an AssignmentManager which is aware of > RegionServer groups and assigns tables to region servers based on groupings. > Load balancing will occur on a per group basis as well. > This is essentially a simplification of the approach taken in HBASE-4120. See > attached document. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14919) Infrastructure refactoring
[ https://issues.apache.org/jira/browse/HBASE-14919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060512#comment-15060512 ] Ted Yu commented on HBASE-14919: For https://builds.apache.org/job/PreCommit-HBASE-Build/16874/consoleFull : Printing hanging tests Hanging test : org.apache.hadoop.hbase.snapshot.TestFlushSnapshotFromClient Hanging test : org.apache.hadoop.hbase.snapshot.TestMobExportSnapshot Hanging test : org.apache.hadoop.hbase.snapshot.TestSecureExportSnapshot Hanging test : org.apache.hadoop.hbase.snapshot.TestMobSecureExportSnapshot Hanging test : org.apache.hadoop.hbase.master.TestSplitLogManager Hanging test : org.apache.hadoop.hbase.snapshot.TestRestoreFlushSnapshotFromClient Printing Failing tests Failing test : org.apache.hadoop.hbase.regionserver.TestHRegionServerBulkLoad > Infrastructure refactoring > -- > > Key: HBASE-14919 > URL: https://issues.apache.org/jira/browse/HBASE-14919 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Attachments: HBASE-14919-V01.patch, HBASE-14919-V01.patch, > HBASE-14919-V02.patch, HBASE-14919-V03.patch > > > Refactoring the MemStore hierarchy, introducing segment (StoreSegment) as > first-class citizen and decoupling memstore scanner from the memstore > implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14838) Clarify that SimpleRegionNormalizer does not merge empty (<1MB) regions
[ https://issues.apache.org/jira/browse/HBASE-14838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060541#comment-15060541 ] Hudson commented on HBASE-14838: FAILURE: Integrated in HBase-1.3 #440 (See [https://builds.apache.org/job/HBase-1.3/440/]) HBASE-14838 Add clarification docs to SimpleRegionNormalizer. (busbey: rev 00805b399f0b953efe510ff3ef8584c39171019b) * hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java > Clarify that SimpleRegionNormalizer does not merge empty (<1MB) regions > --- > > Key: HBASE-14838 > URL: https://issues.apache.org/jira/browse/HBASE-14838 > Project: HBase > Issue Type: Bug > Components: documentation, regionserver >Affects Versions: 1.2.0 >Reporter: Romil Choksi >Assignee: Josh Elser >Priority: Trivial > Fix For: 2.0.0, 1.2.0 > > Attachments: HBASE-14838.patch > > > SImpleRegionNormalizer does not merge empty region of a table > Steps to repro: > - Create an empty table with few, say 5-6 regions without any data in any of > them > - Verify hbase:meta table to verify the regions for the table or check > HMaster UI > - Enable normalizer switch and normalization for this table > - Run normalizer, by 'normalize' command from hbase shell > - Verify the regions for table by scanning hbase:meta table or checking > HMaster web UI > The empty regions are not merged on running the region normalizer. This seems > to be an edge case with completely empty regions since the Normalizer checks > for: smallestRegion (in this case 0 size) + smallestNeighborOfSmallestRegion > (in this case 0 size) > avg region size (in this case 0 size) > thanks to [~elserj] for verifying this from the source code side -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14807) TestWALLockup is flakey
[
https://issues.apache.org/jira/browse/HBASE-14807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060347#comment-15060347
]
Sean Busbey commented on HBASE-14807:
-
this ready to close out now?
> TestWALLockup is flakey
> ---
>
> Key: HBASE-14807
> URL: https://issues.apache.org/jira/browse/HBASE-14807
> Project: HBase
> Issue Type: Bug
> Components: flakey, test
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: 14807.patch, 14807.second.attempt.txt,
> 14807.second.attempt.txt
>
>
> Fails frequently.
> Looks like this:
> {code}
> 2015-11-12 10:38:51,812 DEBUG [Time-limited test] regionserver.HRegion(3882):
> Found 0 recovered edits file(s) under
> /home/jenkins/jenkins-slave/workspace/HBase-1.2/jdk/latest1.7/label/Hadoop/hbase-server/target/test-data/8b8f8f12-1819-47e3-b1f1-8ffa789438ad/data/default/testLockupWhenSyncInMiddleOfZigZagSetup/c8694b53368f3301a8d370089120388d
> 2015-11-12 10:38:51,821 DEBUG [Time-limited test]
> regionserver.FlushLargeStoresPolicy(56):
> hbase.hregion.percolumnfamilyflush.size.lower.bound is not specified, use
> global config(16777216) instead
> 2015-11-12 10:38:51,880 DEBUG [Time-limited test] wal.WALSplitter(729): Wrote
> region
> seqId=/home/jenkins/jenkins-slave/workspace/HBase-1.2/jdk/latest1.7/label/Hadoop/hbase-server/target/test-data/8b8f8f12-1819-47e3-b1f1-8ffa789438ad/data/default/testLockupWhenSyncInMiddleOfZigZagSetup/c8694b53368f3301a8d370089120388d/recovered.edits/2.seqid
> to file, newSeqId=2, maxSeqId=0
> 2015-11-12 10:38:51,881 INFO [Time-limited test] regionserver.HRegion(868):
> Onlined c8694b53368f3301a8d370089120388d; next sequenceid=2
> 2015-11-12 10:38:51,994 ERROR [sync.1] wal.FSHLog$SyncRunner(1226): Error
> syncing, request close of WAL
> java.io.IOException: FAKE! Failed to replace a bad datanode...SYNC
> at
> org.apache.hadoop.hbase.regionserver.TestWALLockup$1DodgyFSLog$1.sync(TestWALLockup.java:162)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$SyncRunner.run(FSHLog.java:1222)
> at java.lang.Thread.run(Thread.java:745)
> 2015-11-12 10:38:51,997 DEBUG [Thread-4] regionserver.LogRoller(139): WAL
> roll requested
> 2015-11-12 10:38:52,019 DEBUG [flusher]
> regionserver.FlushLargeStoresPolicy(100): Since none of the CFs were above
> the size, flushing all.
> 2015-11-12 10:38:52,192 INFO [Thread-4]
> regionserver.TestWALLockup$1DodgyFSLog(129): LATCHED
> java.lang.InterruptedException: sleep interrupted
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.util.Threads.sleep(Threads.java:146)
> at
> org.apache.hadoop.hbase.regionserver.TestWALLockup.testLockupWhenSyncInMiddleOfZigZagSetup(TestWALLockup.java:245)
> 2015-11-12 10:39:18,609 INFO [main] regionserver.TestWALLockup(91): Cleaning
> test directory:
> /home/jenkins/jenkins-slave/workspace/HBase-1.2/jdk/latest1.7/label/Hadoop/hbase-server/target/test-data/8b8f8f12-1819-47e3-b1f1-8ffa789438ad
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> ... then times out after being locked up for 30 seconds. Writes 50+MB of
> logs while spinning.
> Reported as this:
> {code}
> ---
> Test set: org.apache.hadoop.hbase.regionserver.TestWALLockup
> ---
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 198.23 sec
> <<< FAILURE! - in org.apache.hadoop.hbase.regionserver.TestWALLockup
> testLockupWhenSyncInMiddleOfZigZagSetup(org.apache.hadoop.hbase.regionserver.TestWALLockup)
> Time elapsed: 0.049 sec <<< ERROR!
> org.junit.runners.model.TestTimedOutException: test timed out after 3
> milliseconds
> at
[jira] [Commented] (HBASE-14964) Backport HBASE-14901 to brach-1 - There is duplicated code to create/manage encryption keys
[
https://issues.apache.org/jira/browse/HBASE-14964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060452#comment-15060452
]
Hadoop QA commented on HBASE-14964:
---
{color:green}+1 overall{color}.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16877//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16877//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16877//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16877//console
This message is automatically generated.
> Backport HBASE-14901 to brach-1 - There is duplicated code to create/manage
> encryption keys
> ---
>
> Key: HBASE-14964
> URL: https://issues.apache.org/jira/browse/HBASE-14964
> Project: HBase
> Issue Type: Improvement
> Components: encryption
>Reporter: Nate Edel
>Assignee: Nate Edel
>Priority: Minor
> Fix For: 1.2.0, 1.3.0
>
> Attachments: HBASE-14964-branch-1.1.patch, HBASE-14964.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> There is duplicated code from MobUtils.createEncryptionContext in HStore, and
> there is a subset of that code in HFileReaderImpl.
> Refactored key selection
> Moved both to EncryptionUtil.java
> Can't figure out how to write a unit test for this, but there's no new code
> just refactoring.
> A lot of the Mob stuff hasn't been backported, so this is a very small patch.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14822) Renewing leases of scanners doesn't work
[ https://issues.apache.org/jira/browse/HBASE-14822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060344#comment-15060344 ] Sean Busbey commented on HBASE-14822: - bump? is this ready to go in? I restarted a jenkins build. > Renewing leases of scanners doesn't work > > > Key: HBASE-14822 > URL: https://issues.apache.org/jira/browse/HBASE-14822 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.14 >Reporter: Samarth Jain >Assignee: Lars Hofhansl > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17, 1.0.4 > > Attachments: 14822-0.98-v2.txt, 14822-0.98-v3.txt, 14822-0.98.txt, > 14822-v3-0.98.txt, 14822-v4-0.98.txt, 14822-v4.txt, 14822-v5-0.98.txt, > 14822-v5.txt, 14822.txt > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14838) Clarify that SimpleRegionNormalizer does not merge empty (<1MB) regions
[ https://issues.apache.org/jira/browse/HBASE-14838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060387#comment-15060387 ] Hudson commented on HBASE-14838: FAILURE: Integrated in HBase-Trunk_matrix #557 (See [https://builds.apache.org/job/HBase-Trunk_matrix/557/]) HBASE-14838 Add clarification docs to SimpleRegionNormalizer. (busbey: rev ca7a82fb5704af27459557168afc96e1193ac7cb) * hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java > Clarify that SimpleRegionNormalizer does not merge empty (<1MB) regions > --- > > Key: HBASE-14838 > URL: https://issues.apache.org/jira/browse/HBASE-14838 > Project: HBase > Issue Type: Bug > Components: documentation, regionserver >Affects Versions: 1.2.0 >Reporter: Romil Choksi >Assignee: Josh Elser >Priority: Trivial > Fix For: 2.0.0, 1.2.0 > > Attachments: HBASE-14838.patch > > > SImpleRegionNormalizer does not merge empty region of a table > Steps to repro: > - Create an empty table with few, say 5-6 regions without any data in any of > them > - Verify hbase:meta table to verify the regions for the table or check > HMaster UI > - Enable normalizer switch and normalization for this table > - Run normalizer, by 'normalize' command from hbase shell > - Verify the regions for table by scanning hbase:meta table or checking > HMaster web UI > The empty regions are not merged on running the region normalizer. This seems > to be an edge case with completely empty regions since the Normalizer checks > for: smallestRegion (in this case 0 size) + smallestNeighborOfSmallestRegion > (in this case 0 size) > avg region size (in this case 0 size) > thanks to [~elserj] for verifying this from the source code side -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14838) Clarify that SimpleRegionNormalizer does not merge empty (<1MB) regions
[ https://issues.apache.org/jira/browse/HBASE-14838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060403#comment-15060403 ] Hudson commented on HBASE-14838: SUCCESS: Integrated in HBase-1.2 #446 (See [https://builds.apache.org/job/HBase-1.2/446/]) HBASE-14838 Add clarification docs to SimpleRegionNormalizer. (busbey: rev e2d5689e3b0b435aeefb0dafde7f50309e1008fb) * hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java > Clarify that SimpleRegionNormalizer does not merge empty (<1MB) regions > --- > > Key: HBASE-14838 > URL: https://issues.apache.org/jira/browse/HBASE-14838 > Project: HBase > Issue Type: Bug > Components: documentation, regionserver >Affects Versions: 1.2.0 >Reporter: Romil Choksi >Assignee: Josh Elser >Priority: Trivial > Fix For: 2.0.0, 1.2.0 > > Attachments: HBASE-14838.patch > > > SImpleRegionNormalizer does not merge empty region of a table > Steps to repro: > - Create an empty table with few, say 5-6 regions without any data in any of > them > - Verify hbase:meta table to verify the regions for the table or check > HMaster UI > - Enable normalizer switch and normalization for this table > - Run normalizer, by 'normalize' command from hbase shell > - Verify the regions for table by scanning hbase:meta table or checking > HMaster web UI > The empty regions are not merged on running the region normalizer. This seems > to be an edge case with completely empty regions since the Normalizer checks > for: smallestRegion (in this case 0 size) + smallestNeighborOfSmallestRegion > (in this case 0 size) > avg region size (in this case 0 size) > thanks to [~elserj] for verifying this from the source code side -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14977) ChoreService.shutdown may result in ConcurrentModificationException
[
https://issues.apache.org/jira/browse/HBASE-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060404#comment-15060404
]
Matteo Bertozzi commented on HBASE-14977:
-
on a local run I get:
{noformat}
org.junit.runners.model.TestTimedOutException: test timed out after 2
milliseconds
at
org.apache.hadoop.hbase.ScheduledChore.getName(ScheduledChore.java:272)
at
org.apache.hadoop.hbase.ScheduledChore.toString(ScheduledChore.java:344)
at java.lang.String.valueOf(String.java:2847)
at java.lang.StringBuilder.append(StringBuilder.java:128)
at java.util.AbstractCollection.toString(AbstractCollection.java:458)
at java.lang.String.valueOf(String.java:2847)
at java.lang.StringBuilder.append(StringBuilder.java:128)
at org.apache.hadoop.hbase.ChoreService.shutdown(ChoreService.java:323)
at
org.apache.hadoop.hbase.TestChoreService.shutdownService(TestChoreService.java:363)
at
org.apache.hadoop.hbase.TestChoreService.testShutdownRejectsNewSchedules(TestChoreService.java:844)
{noformat}
{noformat}
Potentially hanging thread: testShutdownRejectsNewSchedules_ChoreService_1
org.apache.hadoop.hbase.ChoreService.isChoreScheduled(ChoreService.java:205)
org.apache.hadoop.hbase.ScheduledChore.isScheduled(ScheduledChore.java:314)
org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:176)
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
java.lang.Thread.run(Thread.java:745)
Potentially hanging thread: Time-limited test
org.apache.hadoop.hbase.ScheduledChore.getName(ScheduledChore.java:272)
org.apache.hadoop.hbase.ScheduledChore.toString(ScheduledChore.java:344)
java.lang.String.valueOf(String.java:2847)
java.lang.StringBuilder.append(StringBuilder.java:128)
java.util.AbstractCollection.toString(AbstractCollection.java:458)
java.lang.String.valueOf(String.java:2847)
java.lang.StringBuilder.append(StringBuilder.java:128)
org.apache.hadoop.hbase.ChoreService.shutdown(ChoreService.java:323)
org.apache.hadoop.hbase.TestChoreService.shutdownService(TestChoreService.java:363)
org.apache.hadoop.hbase.TestChoreService.testShutdownRejectsNewSchedules(TestChoreService.java:844)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:606)
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
java.util.concurrent.FutureTask.run(FutureTask.java:262)
java.lang.Thread.run(Thread.java:745)
{noformat}
> ChoreService.shutdown may result in ConcurrentModificationException
> ---
>
> Key: HBASE-14977
> URL: https://issues.apache.org/jira/browse/HBASE-14977
> Project: HBase
> Issue Type: Bug
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
>Priority: Minor
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.4
>
> Attachments: HBASE-14977-v1.patch
>
>
> As seen in this test:
> https://builds.apache.org/job/HBase-1.3/jdk=latest1.8,label=Hadoop/425/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.regionserver.compactions.TestFIFOCompactionPolicy-output.txt
> We need to make shutdown method synchronized to avoid this issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HBASE-14977) ChoreService.shutdown may result in ConcurrentModificationException
[
https://issues.apache.org/jira/browse/HBASE-14977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060404#comment-15060404
]
Matteo Bertozzi edited comment on HBASE-14977 at 12/16/15 5:55 PM:
---
on a local run branch-1.2 I got:
{noformat}
org.junit.runners.model.TestTimedOutException: test timed out after 2
milliseconds
at
org.apache.hadoop.hbase.ScheduledChore.getName(ScheduledChore.java:272)
at
org.apache.hadoop.hbase.ScheduledChore.toString(ScheduledChore.java:344)
at java.lang.String.valueOf(String.java:2847)
at java.lang.StringBuilder.append(StringBuilder.java:128)
at java.util.AbstractCollection.toString(AbstractCollection.java:458)
at java.lang.String.valueOf(String.java:2847)
at java.lang.StringBuilder.append(StringBuilder.java:128)
at org.apache.hadoop.hbase.ChoreService.shutdown(ChoreService.java:323)
at
org.apache.hadoop.hbase.TestChoreService.shutdownService(TestChoreService.java:363)
at
org.apache.hadoop.hbase.TestChoreService.testShutdownRejectsNewSchedules(TestChoreService.java:844)
{noformat}
{noformat}
Potentially hanging thread: testShutdownRejectsNewSchedules_ChoreService_1
org.apache.hadoop.hbase.ChoreService.isChoreScheduled(ChoreService.java:205)
org.apache.hadoop.hbase.ScheduledChore.isScheduled(ScheduledChore.java:314)
org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:176)
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
java.lang.Thread.run(Thread.java:745)
Potentially hanging thread: Time-limited test
org.apache.hadoop.hbase.ScheduledChore.getName(ScheduledChore.java:272)
org.apache.hadoop.hbase.ScheduledChore.toString(ScheduledChore.java:344)
java.lang.String.valueOf(String.java:2847)
java.lang.StringBuilder.append(StringBuilder.java:128)
java.util.AbstractCollection.toString(AbstractCollection.java:458)
java.lang.String.valueOf(String.java:2847)
java.lang.StringBuilder.append(StringBuilder.java:128)
org.apache.hadoop.hbase.ChoreService.shutdown(ChoreService.java:323)
org.apache.hadoop.hbase.TestChoreService.shutdownService(TestChoreService.java:363)
org.apache.hadoop.hbase.TestChoreService.testShutdownRejectsNewSchedules(TestChoreService.java:844)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:606)
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
java.util.concurrent.FutureTask.run(FutureTask.java:262)
java.lang.Thread.run(Thread.java:745)
{noformat}
but does not happen every time
was (Author: mbertozzi):
on a local run I get:
{noformat}
org.junit.runners.model.TestTimedOutException: test timed out after 2
milliseconds
at
org.apache.hadoop.hbase.ScheduledChore.getName(ScheduledChore.java:272)
at
org.apache.hadoop.hbase.ScheduledChore.toString(ScheduledChore.java:344)
at java.lang.String.valueOf(String.java:2847)
at java.lang.StringBuilder.append(StringBuilder.java:128)
at java.util.AbstractCollection.toString(AbstractCollection.java:458)
at java.lang.String.valueOf(String.java:2847)
at java.lang.StringBuilder.append(StringBuilder.java:128)
at org.apache.hadoop.hbase.ChoreService.shutdown(ChoreService.java:323)
at
org.apache.hadoop.hbase.TestChoreService.shutdownService(TestChoreService.java:363)
at
org.apache.hadoop.hbase.TestChoreService.testShutdownRejectsNewSchedules(TestChoreService.java:844)
{noformat}
{noformat}
Potentially hanging thread:
[jira] [Commented] (HBASE-14983) Create metrics for per block type hit/miss ratios
[
https://issues.apache.org/jira/browse/HBASE-14983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060593#comment-15060593
]
Hadoop QA commented on HBASE-14983:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12777867/HBASE-14983-v1.patch
against master branch at commit 7099ab4c56518024a26f5c499a9a1f2ea6bacd6e.
ATTACHMENT ID: 12777867
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 11 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16882//console
This message is automatically generated.
> Create metrics for per block type hit/miss ratios
> -
>
> Key: HBASE-14983
> URL: https://issues.apache.org/jira/browse/HBASE-14983
> Project: HBase
> Issue Type: Improvement
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Attachments: HBASE-14983-v1.patch, HBASE-14983.patch, Screen Shot
> 2015-12-15 at 3.33.09 PM.png
>
>
> Missing a root index block is worse than missing a data block. We should know
> the difference
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14635) Reenable TestSnapshotCloneIndependence.testOnlineSnapshotDeleteIndependent
[ https://issues.apache.org/jira/browse/HBASE-14635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14635: Fix Version/s: (was: 1.2.0) > Reenable TestSnapshotCloneIndependence.testOnlineSnapshotDeleteIndependent > -- > > Key: HBASE-14635 > URL: https://issues.apache.org/jira/browse/HBASE-14635 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: stack > Fix For: 2.0.0, 1.3.0 > > > Was disabled in the parent issue because flakey. This issue is about > reenabling it after figuring why its flakey. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14498) Master stuck in infinite loop when all Zookeeper servers are unreachable
[
https://issues.apache.org/jira/browse/HBASE-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-14498:
Fix Version/s: (was: 1.2.0)
1.2.1
> Master stuck in infinite loop when all Zookeeper servers are unreachable
>
>
> Key: HBASE-14498
> URL: https://issues.apache.org/jira/browse/HBASE-14498
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Y. SREENIVASULU REDDY
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 2.0.0, 1.3.0, 1.2.1, 1.1.4
>
> Attachments: HBASE-14498-V2.patch, HBASE-14498-V3.patch,
> HBASE-14498-V4.patch, HBASE-14498.patch
>
>
> We met a weird scenario in our production environment.
> In a HA cluster,
> > Active Master (HM1) is not able to connect to any Zookeeper server (due to
> > N/w breakdown on master machine network with Zookeeper servers).
> {code}
> 2015-09-26 15:24:47,508 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 33463ms for sessionid 0x104576b8dda0002, closing socket connection and
> attempting reconnect
> 2015-09-26 15:24:47,877 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:48,236 INFO [main-SendThread(ZK-Host1:2181)]
> client.FourLetterWordMain: connecting to ZK-Host1 2181
> 2015-09-26 15:24:49,879 WARN
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:49,879 INFO
> [HM1-Host:16000.activeMasterManager-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-IP1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:24:50,238 WARN [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host1
> 2015-09-26 15:24:50,238 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server
> ZK-Host1/ZK-Host1:2181. Will not attempt to authenticate using SASL (unknown
> error)
> 2015-09-26 15:25:17,470 INFO [main-SendThread(ZK-Host1:2181)]
> zookeeper.ClientCnxn: Client session timed out, have not heard from server in
> 30023ms for sessionid 0x2045762cc710006, closing socket connection and
> attempting reconnect
> 2015-09-26 15:25:17,571 WARN [master/HM1-Host/HM1-IP:16000]
> zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper,
> quorum=ZK-Host:2181,ZK-Host1:2181,ZK-Host2:2181,
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
> 2015-09-26 15:25:17,872 INFO [main-SendThread(ZK-Host:2181)]
> client.FourLetterWordMain: connecting to ZK-Host 2181
> 2015-09-26 15:25:19,874 WARN [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Can not get the principle name from server ZK-Host
> 2015-09-26 15:25:19,874 INFO [main-SendThread(ZK-Host:2181)]
> zookeeper.ClientCnxn: Opening socket connection to server ZK-Host/ZK-IP:2181.
> Will not attempt to authenticate using SASL (unknown error)
> {code}
> > Since HM1 was not able to connect to any ZK, so session timeout didnt
> > happen at Zookeeper server side and HM1 didnt abort.
> > On Zookeeper session timeout standby master (HM2) registered himself as an
> > active master.
> > HM2 is keep on waiting for region server to report him as part of active
> > master intialization.
> {noformat}
> 2015-09-26 15:24:44,928 | INFO | HM2-Host:21300.activeMasterManager | Waiting
> for region servers count to settle; currently checked in 0, slept for 0 ms,
> expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, interval
> of 1500 ms. |
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> ---
> ---
> 2015-09-26 15:32:50,841 | INFO | HM2-Host:21300.activeMasterManager | Waiting
> for region servers count to settle; currently checked in 0, slept for 483913
> ms, expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms,
> interval of 1500 ms. |
> org.apache.hadoop.hbase.master.ServerManager.waitForRegionServers(ServerManager.java:1011)
> {noformat}
> > At other end, region servers are reporting to HM1 on 3 sec interval. Here
> > region server retrieve master location from zookeeper only when they
> > couldn't connect to Master (ServiceException).
> Region Server will not report HM2 as per current design until unless HM1
> abort,so HM2 will exit(InitializationMonitor) and again wait for region
> servers in loop.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14989) Implementation of Mutation.getWriteToWAL() is backwards
[
https://issues.apache.org/jira/browse/HBASE-14989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-14989:
Fix Version/s: (was: 1.2.0)
1.2.1
> Implementation of Mutation.getWriteToWAL() is backwards
> ---
>
> Key: HBASE-14989
> URL: https://issues.apache.org/jira/browse/HBASE-14989
> Project: HBase
> Issue Type: Bug
>Reporter: James Taylor
> Fix For: 1.3.0, 1.2.1, 1.1.4, 0.98.17, 1.0.4
>
>
> The implementation of the deprecated getWriteToWAL is backwards. It should
> return true if this.durability == Durability.SYNC_WAL:
> {code}
> /**
>* @deprecated Use {@link #getDurability()} instead.
>* @return true if edits should be applied to WAL, false if not
>*/
> @Deprecated
> public boolean getWriteToWAL() {
> return this.durability == Durability.SKIP_WAL;
> }
> {code}
> For example, if mutation.durability is Durability.SYNC_WAL and the following
> code is called {{clonedMutation.setWriteToWAL(mutation.getWriteToWAL())}}, it
> will disable writing to the WAL for clonedMutation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14470) Reduce memory pressure generated by client
[ https://issues.apache.org/jira/browse/HBASE-14470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14470: Fix Version/s: (was: 1.2.0) > Reduce memory pressure generated by client > -- > > Key: HBASE-14470 > URL: https://issues.apache.org/jira/browse/HBASE-14470 > Project: HBase > Issue Type: Task > Components: Client, Performance >Affects Versions: 1.3.0 >Reporter: Nick Dimiduk >Assignee: stack > Fix For: 2.0.0, 1.3.0 > > Attachments: allocation-by-class.jpg, allocation-by-thread.jpg, > c+ma.jfc, object-stats.jpg > > > I think there's room for improvement in our client's memory profile. I ran > ltt with jfr running, attaching some snaps of what my client sees. Looks like > some kind of object pool or block encoding for result objects will give us a > lot of bang for the buck re: allocations and GC pressure. We probably also > want to look for an alternative way to represent result objects, something > besides the java Map interface with it's Entry bloat. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14978) Don't allow Multi to retain too many blocks
[ https://issues.apache.org/jira/browse/HBASE-14978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-14978: -- Fix Version/s: 1.3.0 3.0.0 1.2.0 > Don't allow Multi to retain too many blocks > --- > > Key: HBASE-14978 > URL: https://issues.apache.org/jira/browse/HBASE-14978 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Elliott Clark >Assignee: Elliott Clark >Priority: Critical > Fix For: 1.2.0, 3.0.0, 1.3.0 > > Attachments: HBASE-14978-v1.patch, HBASE-14978-v2.patch, > HBASE-14978-v3.patch, HBASE-14978-v4.patch, HBASE-14978.patch > > > Scans and Multi's have limits on the total size of cells that can be > returned. However if those requests are not all pointing at the same blocks > then the KeyValues can keep alive a lot more data than their size. > Take the following example: > A multi with a list of 1 gets to a fat row. Each column being returned in > in a different block. Each column is small 32 bytes or so. > So the total cell size will be 32 * 1 = ~320kb. However if each block is > 128k then total retained heap size will be almost 2gigs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14730) region server needs to log warnings when there are attributes configured for cells with hfile v2
[ https://issues.apache.org/jira/browse/HBASE-14730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huaxiang sun updated HBASE-14730: - Attachment: HBASE-14730-v002_branch-1.2.patch > region server needs to log warnings when there are attributes configured for > cells with hfile v2 > > > Key: HBASE-14730 > URL: https://issues.apache.org/jira/browse/HBASE-14730 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 1.0.0, 1.1.0, 1.2.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > Attachments: HBASE-14730-v001-1.2.0.patch, HBASE-14730-v001.patch, > HBASE-14730-v002-1.2.0.patch, HBASE-14730-v002_branch-1.2.patch > > > User can configure cell attributes with hfile.format.version 2. When cells > are flushed from memStore to Hfile, attributes are not saved. Warnings need > to be logged. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14730) region server needs to log warnings when there are attributes configured for cells with hfile v2
[ https://issues.apache.org/jira/browse/HBASE-14730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061029#comment-15061029 ] huaxiang sun commented on HBASE-14730: -- Rename the patch so it can be picked up for branch-1.2. > region server needs to log warnings when there are attributes configured for > cells with hfile v2 > > > Key: HBASE-14730 > URL: https://issues.apache.org/jira/browse/HBASE-14730 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 1.0.0, 1.1.0, 1.2.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > Attachments: HBASE-14730-v001-1.2.0.patch, HBASE-14730-v001.patch, > HBASE-14730-v002-1.2.0.patch, HBASE-14730-v002_branch-1.2.patch > > > User can configure cell attributes with hfile.format.version 2. When cells > are flushed from memStore to Hfile, attributes are not saved. Warnings need > to be logged. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14730) region server needs to log warnings when there are attributes configured for cells with hfile v2
[ https://issues.apache.org/jira/browse/HBASE-14730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061030#comment-15061030 ] huaxiang sun commented on HBASE-14730: -- Created HBASE-14995 > region server needs to log warnings when there are attributes configured for > cells with hfile v2 > > > Key: HBASE-14730 > URL: https://issues.apache.org/jira/browse/HBASE-14730 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 1.0.0, 1.1.0, 1.2.0 >Reporter: huaxiang sun >Assignee: huaxiang sun > Attachments: HBASE-14730-v001-1.2.0.patch, HBASE-14730-v001.patch, > HBASE-14730-v002-1.2.0.patch, HBASE-14730-v002_branch-1.2.patch > > > User can configure cell attributes with hfile.format.version 2. When cells > are flushed from memStore to Hfile, attributes are not saved. Warnings need > to be logged. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14995) Optimize setting tagsPresent in DefaultMemStore.java
[
https://issues.apache.org/jira/browse/HBASE-14995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061062#comment-15061062
]
Ted Yu commented on HBASE-14995:
lgtm
> Optimize setting tagsPresent in DefaultMemStore.java
>
>
> Key: HBASE-14995
> URL: https://issues.apache.org/jira/browse/HBASE-14995
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Affects Versions: 2.0.0, 1.2.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Attachments: HBASE-14995-v001.patch
>
>
> The current implementation calls e.getTagsLength() for each cell. Once
> tagsPresent is set, e.getTagsLength() can be avoided.
> {code}
> private boolean addToCellSet(Cell e) {
> boolean b = this.cellSet.add(e);
> // In no tags case this NoTagsKeyValue.getTagsLength() is a cheap call.
> // When we use ACL CP or Visibility CP which deals with Tags during
> // mutation, the TagRewriteCell.getTagsLength() is a cheaper call. We do
> not
> // parse the byte[] to identify the tags length.
> if(e.getTagsLength() > 0) {
> tagsPresent = true;
> }
> setOldestEditTimeToNow();
> return b;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14951) Make hbase.regionserver.maxlogs obsolete
[ https://issues.apache.org/jira/browse/HBASE-14951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-14951: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 1.3.0 1.2.0 Status: Resolved (was: Patch Available) I've pushed to 1.2+. Thanks Vladimir. Sorry I forgot to add the author at the commit message. > Make hbase.regionserver.maxlogs obsolete > > > Key: HBASE-14951 > URL: https://issues.apache.org/jira/browse/HBASE-14951 > Project: HBase > Issue Type: Improvement >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14951-v1.patch, HBASE-14951-v2.patch > > > There was a discussion in HBASE-14388 related to maximum number of log files. > It was an agreement that we should calculate this number in a code but still > need to honor user's setting. > Maximum number of log files now is calculated as following: > maxLogs = HEAP_SIZE * memstoreRatio * 2/ LogRollSize -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14730) region server needs to log warnings when there are attributes configured for cells with hfile v2
[
https://issues.apache.org/jira/browse/HBASE-14730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060956#comment-15060956
]
huaxiang sun commented on HBASE-14730:
--
[~anoop.hbase] With help from [~mbertozzi], we found the flag you mentioned. I
uploaded the latest patch. Also found that in DefaultMemStore.java, it can be
optimized for tagsPresent, will create another jira for the optimization.
Thanks.
private boolean addToCellSet(Cell e) {
boolean b = this.cellSet.add(e);
// In no tags case this NoTagsKeyValue.getTagsLength() is a cheap call.
// When we use ACL CP or Visibility CP which deals with Tags during
// mutation, the TagRewriteCell.getTagsLength() is a cheaper call. We do not
// parse the byte[] to identify the tags length.
-if(e.getTagsLength() > 0) {
+if(!tagsPresent && (e.getTagsLength() > 0)) {
tagsPresent = true;
}
setOldestEditTimeToNow();
return b;
}
> region server needs to log warnings when there are attributes configured for
> cells with hfile v2
>
>
> Key: HBASE-14730
> URL: https://issues.apache.org/jira/browse/HBASE-14730
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Affects Versions: 1.0.0, 1.1.0, 1.2.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
> Attachments: HBASE-14730-v001-1.2.0.patch, HBASE-14730-v001.patch,
> HBASE-14730-v002-1.2.0.patch
>
>
> User can configure cell attributes with hfile.format.version 2. When cells
> are flushed from memStore to Hfile, attributes are not saved. Warnings need
> to be logged.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14610) IntegrationTestRpcClient from HBASE-14535 is failing with Async RPC client
[ https://issues.apache.org/jira/browse/HBASE-14610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14610: Issue Type: Bug (was: Sub-task) Parent: (was: HBASE-14474) > IntegrationTestRpcClient from HBASE-14535 is failing with Async RPC client > -- > > Key: HBASE-14610 > URL: https://issues.apache.org/jira/browse/HBASE-14610 > Project: HBase > Issue Type: Bug > Components: rpc >Reporter: Enis Soztutar > Fix For: 2.0.0, 1.3.0, 1.2.1, 1.1.4, 1.0.4 > > Attachments: output > > > HBASE-14535 introduces an IT to simulate a running cluster with RPC servers > and RPC clients doing requests against the servers. > It passes with the sync client, but fails with async client. Probably we need > to take a look. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14610) IntegrationTestRpcClient from HBASE-14535 is failing with Async RPC client
[ https://issues.apache.org/jira/browse/HBASE-14610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060957#comment-15060957 ] Sean Busbey commented on HBASE-14610: - updated to top-level issue from subtask of HBASE-14474 pending release. > IntegrationTestRpcClient from HBASE-14535 is failing with Async RPC client > -- > > Key: HBASE-14610 > URL: https://issues.apache.org/jira/browse/HBASE-14610 > Project: HBase > Issue Type: Bug > Components: rpc >Reporter: Enis Soztutar > Fix For: 2.0.0, 1.3.0, 1.2.1, 1.1.4, 1.0.4 > > Attachments: output > > > HBASE-14535 introduces an IT to simulate a running cluster with RPC servers > and RPC clients doing requests against the servers. > It passes with the sync client, but fails with async client. Probably we need > to take a look. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14610) IntegrationTestRpcClient from HBASE-14535 is failing with Async RPC client
[ https://issues.apache.org/jira/browse/HBASE-14610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-14610: Fix Version/s: (was: 1.2.0) 1.2.1 > IntegrationTestRpcClient from HBASE-14535 is failing with Async RPC client > -- > > Key: HBASE-14610 > URL: https://issues.apache.org/jira/browse/HBASE-14610 > Project: HBase > Issue Type: Bug > Components: rpc >Reporter: Enis Soztutar > Fix For: 2.0.0, 1.3.0, 1.2.1, 1.1.4, 1.0.4 > > Attachments: output > > > HBASE-14535 introduces an IT to simulate a running cluster with RPC servers > and RPC clients doing requests against the servers. > It passes with the sync client, but fails with async client. Probably we need > to take a look. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14994) Clean up some broken links and references to old APIs
[
https://issues.apache.org/jira/browse/HBASE-14994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060977#comment-15060977
]
Hadoop QA commented on HBASE-14994:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12778083/HBASE-14994.patch
against master branch at commit 7099ab4c56518024a26f5c499a9a1f2ea6bacd6e.
ATTACHMENT ID: 12778083
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not generate new
checkstyle errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+The file info block is a serialized map from byte arrays to byte arrays,
with the following keys, among others.
+link:http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/constraint/Constraint.html[Constraint]
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16885//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16885//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16885//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16885//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16885//console
This message is automatically generated.
> Clean up some broken links and references to old APIs
> -
>
> Key: HBASE-14994
> URL: https://issues.apache.org/jira/browse/HBASE-14994
> Project: HBase
> Issue Type: Task
> Components: documentation
>Affects Versions: 2.0.0
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Fix For: 2.0.0
>
> Attachments: HBASE-14994.patch
>
>
> Clean up broken links and references to old APIs found in
> https://builds.apache.org/job/HBase%20Website%20Link%20Ckecker/14/artifact/link_report/errorX.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13865) Increase the default value for hbase.hregion.memstore.block.multipler from 2 to 4 (part 2)
[ https://issues.apache.org/jira/browse/HBASE-13865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060991#comment-15060991 ] Sean Busbey commented on HBASE-13865: - (yay! admins can edit fix versions on closed issues. nevermind. :) ) > Increase the default value for hbase.hregion.memstore.block.multipler from 2 > to 4 (part 2) > -- > > Key: HBASE-13865 > URL: https://issues.apache.org/jira/browse/HBASE-13865 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Gabor Liptak >Priority: Trivial > Fix For: 2.0.0, 0.98.14, 1.0.2, 1.2.0, 1.1.2, 1.3.0 > > Attachments: HBASE-13865.1.patch, HBASE-13865.2.patch, > HBASE-13865.2.patch > > > Its 4 in the book and 2 in a current master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14995) Optimize setting tagsPresent in DefaultMemStore.java
huaxiang sun created HBASE-14995:
Summary: Optimize setting tagsPresent in DefaultMemStore.java
Key: HBASE-14995
URL: https://issues.apache.org/jira/browse/HBASE-14995
Project: HBase
Issue Type: Improvement
Components: regionserver
Affects Versions: 2.0.0, 1.2.0
Reporter: huaxiang sun
Priority: Minor
The current implementation calls e.getTagsLength() for each cell. Once
tagsPresent is set, e.getTagsLength() can be avoided.
private boolean addToCellSet(Cell e) {
boolean b = this.cellSet.add(e);
// In no tags case this NoTagsKeyValue.getTagsLength() is a cheap call.
// When we use ACL CP or Visibility CP which deals with Tags during
// mutation, the TagRewriteCell.getTagsLength() is a cheaper call. We do not
// parse the byte[] to identify the tags length.
if(e.getTagsLength() > 0) {
tagsPresent = true;
}
setOldestEditTimeToNow();
return b;
}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13865) Increase the default value for hbase.hregion.memstore.block.multipler from 2 to 4 (part 2)
[ https://issues.apache.org/jira/browse/HBASE-13865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-13865: Fix Version/s: (was: 1.2.1) 1.2.0 > Increase the default value for hbase.hregion.memstore.block.multipler from 2 > to 4 (part 2) > -- > > Key: HBASE-13865 > URL: https://issues.apache.org/jira/browse/HBASE-13865 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Gabor Liptak >Priority: Trivial > Fix For: 2.0.0, 0.98.14, 1.0.2, 1.2.0, 1.1.2, 1.3.0 > > Attachments: HBASE-13865.1.patch, HBASE-13865.2.patch, > HBASE-13865.2.patch > > > Its 4 in the book and 2 in a current master. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14996) Some more API cleanup for 2.0
Enis Soztutar created HBASE-14996: - Summary: Some more API cleanup for 2.0 Key: HBASE-14996 URL: https://issues.apache.org/jira/browse/HBASE-14996 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Fix For: 2.0.0 Parent jira to keep track of some more API cleanup that did not happen in 1.x timeframe. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14995) Optimize setting tagsPresent in DefaultMemStore.java
[
https://issues.apache.org/jira/browse/HBASE-14995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matteo Bertozzi updated HBASE-14995:
Description:
The current implementation calls e.getTagsLength() for each cell. Once
tagsPresent is set, e.getTagsLength() can be avoided.
{code}
private boolean addToCellSet(Cell e) {
boolean b = this.cellSet.add(e);
// In no tags case this NoTagsKeyValue.getTagsLength() is a cheap call.
// When we use ACL CP or Visibility CP which deals with Tags during
// mutation, the TagRewriteCell.getTagsLength() is a cheaper call. We do not
// parse the byte[] to identify the tags length.
if(e.getTagsLength() > 0) {
tagsPresent = true;
}
setOldestEditTimeToNow();
return b;
}
{code}
was:
The current implementation calls e.getTagsLength() for each cell. Once
tagsPresent is set, e.getTagsLength() can be avoided.
private boolean addToCellSet(Cell e) {
boolean b = this.cellSet.add(e);
// In no tags case this NoTagsKeyValue.getTagsLength() is a cheap call.
// When we use ACL CP or Visibility CP which deals with Tags during
// mutation, the TagRewriteCell.getTagsLength() is a cheaper call. We do not
// parse the byte[] to identify the tags length.
if(e.getTagsLength() > 0) {
tagsPresent = true;
}
setOldestEditTimeToNow();
return b;
}
> Optimize setting tagsPresent in DefaultMemStore.java
>
>
> Key: HBASE-14995
> URL: https://issues.apache.org/jira/browse/HBASE-14995
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
>Affects Versions: 2.0.0, 1.2.0
>Reporter: huaxiang sun
>Assignee: huaxiang sun
>Priority: Minor
> Attachments: HBASE-14995-v001.patch
>
>
> The current implementation calls e.getTagsLength() for each cell. Once
> tagsPresent is set, e.getTagsLength() can be avoided.
> {code}
> private boolean addToCellSet(Cell e) {
> boolean b = this.cellSet.add(e);
> // In no tags case this NoTagsKeyValue.getTagsLength() is a cheap call.
> // When we use ACL CP or Visibility CP which deals with Tags during
> // mutation, the TagRewriteCell.getTagsLength() is a cheaper call. We do
> not
> // parse the byte[] to identify the tags length.
> if(e.getTagsLength() > 0) {
> tagsPresent = true;
> }
> setOldestEditTimeToNow();
> return b;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14983) Create metrics for per block type hit/miss ratios
[
https://issues.apache.org/jira/browse/HBASE-14983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15061153#comment-15061153
]
Hadoop QA commented on HBASE-14983:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12778082/HBASE-14983-v2.patch
against master branch at commit 7099ab4c56518024a26f5c499a9a1f2ea6bacd6e.
ATTACHMENT ID: 12778082
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 11 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
new checkstyle errors. Check build console for list of new errors.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ public BlockCacheKey(String hfileName, long offset, boolean
isPrimaryReplica, BlockType blockType) {
+ return lruCacheStats.getIntermediateIndexMissCount() +
bucketCacheStats.getIntermediateIndexMissCount();
+ return lruCacheStats.getGeneralBloomMetaMissCount() +
bucketCacheStats.getGeneralBloomMetaMissCount();
+ return lruCacheStats.getDeleteFamilyBloomMissCount() +
bucketCacheStats.getDeleteFamilyBloomMissCount();
+ return lruCacheStats.getIntermediateIndexHitCount() +
bucketCacheStats.getIntermediateIndexHitCount();
+ return lruCacheStats.getGeneralBloomMetaHitCount() +
bucketCacheStats.getGeneralBloomMetaHitCount();
+ return lruCacheStats.getDeleteFamilyBloomHitCount() +
bucketCacheStats.getDeleteFamilyBloomHitCount();
+ if (!repeat && updateCacheMetrics) stats.miss(caching,
cacheKey.isPrimary(), cacheKey.getBlockType());
{color:green}+1 site{color}. The mvn post-site goal succeeds with this
patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 zombies{color}. No zombie tests found running at the end of
the build.
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16884//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16884//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16884//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16884//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16884//console
This message is automatically generated.
> Create metrics for per block type hit/miss ratios
> -
>
> Key: HBASE-14983
> URL: https://issues.apache.org/jira/browse/HBASE-14983
> Project: HBase
> Issue Type: Improvement
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Attachments: HBASE-14983-v1.patch, HBASE-14983-v2.patch,
> HBASE-14983.patch, Screen Shot 2015-12-15 at 3.33.09 PM.png
>
>
> Missing a root index block is worse than missing a data block. We should know
> the difference
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14964) Backport HBASE-14901 to brach-1 - There is duplicated code to create/manage encryption keys
[
https://issues.apache.org/jira/browse/HBASE-14964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15060920#comment-15060920
]
Sean Busbey commented on HBASE-14964:
-
running through things locally claims new checkstyle errors:
{code}
| Vote | Subsystem | Runtime | Comment
| +1 | hbaseanti | 0m 0s | Patch does not have any anti-patterns.
| +1 |@author | 0m 0s | The patch does not contain any @author
| | || tags.
| +1 | test4tests | 0m 0s | The patch appears to include 1 new or
| | || modified test files.
| +1 | mvninstall | 4m 28s| branch-1 passed
| +1 | checkstyle | 0m 36s| branch-1 passed
| +1 | mvneclipse | 0m 39s| branch-1 passed
| 0 | findbugs | 1m 15s| findbugs executables are not available.
| +1 | mvninstall | 1m 13s| the patch passed
| -1 | checkstyle | 0m 13s| Patch generated 2 new checkstyle issues
| | || in hbase-client (total was 1, now 2).
| -1 | checkstyle | 0m 20s| Patch generated 5 new checkstyle issues
| | || in hbase-server (total was 11, now 16).
| +1 | mvneclipse | 0m 30s| the patch passed
| +1 | whitespace | 0m 0s | Patch has no whitespace issues.
| +1 |hadoopcheck | 5m 24s| Patch does not cause any errors with
| | || Hadoop 2.4.1 2.5.2 2.6.0.
| +1 | asflicense | 0m 19s| Patch does not generate ASF License
| | || warnings.
| | | 13m 44s |
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12778014/HBASE-14964-branch-1.1.patch
|
| JIRA Issue | HBASE-14964 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux busbey.vpc.cloudera.com 3.10.0-229.11.1.el7.x86_64 #1 SMP Wed
Jul 22 12:06:11 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/busbey/yetus/yetus-0.1.0/lib/precommit/personality/hbase.sh |
| git revision | branch-1 / d2fb7e6 |
| checkstyle | /tmp/yetus-21889.12586/diff-checkstyle-hbase-client.txt |
| checkstyle | /tmp/yetus-21889.12586/diff-checkstyle-hbase-server.txt |
| modules | C: hbase-client hbase-server U: . |
| Max memory used | 135MB |
| Powered by | Apache Yetus 0.1.0 http://yetus.apache.org |
{code}
contents of {{/tmp/yetus-21889.12586/diff-checkstyle-hbase-client.txt}}
{code}
./hbase-client/src/main/java/org/apache/hadoop/hbase/security/EncryptionUtil.java:37:1:
Duplicate import to line 32 - org.apache.hadoop.conf.Configuration.
./hbase-client/src/main/java/org/apache/hadoop/hbase/security/EncryptionUtil.java:38:1:
Duplicate import to line 34 - org.apache.hadoop.hbase.HConstants.
{code}
contents of {{/tmp/yetus-21889.12586/diff-checkstyle-hbase-server.txt}}
{code}
./hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV3.java:22:8:
Unused import - java.security.KeyException.
./hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV3.java:40:8:
Unused import - org.apache.hadoop.hbase.security.User.
./hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java:24:8:
Unused import - java.security.Key.
./hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java:25:8:
Unused import - java.security.KeyException.
./hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java:67:8:
Unused import - org.apache.hadoop.hbase.io.crypto.Cipher.
{code}
it looks like the unit tests didn't fire, so I'm going to see if I can figure
out why before moving this out of patch available.
> Backport HBASE-14901 to brach-1 - There is duplicated code to create/manage
> encryption keys
> ---
>
> Key: HBASE-14964
> URL: https://issues.apache.org/jira/browse/HBASE-14964
> Project: HBase
> Issue Type: Improvement
> Components: encryption
>Reporter: Nate Edel
>Assignee: Nate Edel
>Priority: Minor
> Fix For: 1.2.0, 1.3.0
>
> Attachments: HBASE-14964-branch-1.1.patch, HBASE-14964.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> There is duplicated code from MobUtils.createEncryptionContext in HStore, and
> there is a subset of that code in HFileReaderImpl.
> Refactored key selection
> Moved both to EncryptionUtil.java
> Can't figure out how to write a unit test for this, but there's no
