[jira] [Commented] (HBASE-16143) Change MemstoreScanner constructor to accept List
[ https://issues.apache.org/jira/browse/HBASE-16143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354689#comment-15354689 ] Anoop Sam John commented on HBASE-16143: So instead of SegmentScanner, a KVScanner is being used. SegmentScanner IS A KVScanner? Change looks good Nits: Not from ur patch but I feel we can clean up bq.getListOfScanners Call it getScanners() No need to say data structure name in method name bq.LinkedList list = new LinkedList(); Make the type as List and new ArrayList? Dont think we need a LinkedList anyway. > Change MemstoreScanner constructor to accept List > -- > > Key: HBASE-16143 > URL: https://issues.apache.org/jira/browse/HBASE-16143 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16143.patch > > > A minor change that helps in creating a memstore that avoids the compaction > process and just allows to creates a pipeline of segments and on flush > directly reads the segments in the pipeline and flushes it out after creating > a snapshot of the pipeline. Based on test results and updated patch on > HBASE-14921 (to be provided) will see how much flattening helps us. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16144) Replication queue's lock will live forever if RS acquiring the lock has dead
Phil Yang created HBASE-16144: - Summary: Replication queue's lock will live forever if RS acquiring the lock has dead Key: HBASE-16144 URL: https://issues.apache.org/jira/browse/HBASE-16144 Project: HBase Issue Type: Bug Affects Versions: 0.98.20, 1.1.5, 1.2.1 Reporter: Phil Yang Assignee: Phil Yang In default, we will use multi operation when we claimQueues from ZK. But if we set hbase.zookeeper.useMulti=false, we will add a lock first, then copy nodes, finally clean old queue and the lock. However, if the RS acquiring the lock crash before claimQueues done, the lock will always be there and other RS can never claim the queue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16143) Change MemstoreScanner constructor to accept List
[ https://issues.apache.org/jira/browse/HBASE-16143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-16143: --- Status: Patch Available (was: Open) > Change MemstoreScanner constructor to accept List > -- > > Key: HBASE-16143 > URL: https://issues.apache.org/jira/browse/HBASE-16143 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16143.patch > > > A minor change that helps in creating a memstore that avoids the compaction > process and just allows to creates a pipeline of segments and on flush > directly reads the segments in the pipeline and flushes it out after creating > a snapshot of the pipeline. Based on test results and updated patch on > HBASE-14921 (to be provided) will see how much flattening helps us. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16143) Change MemstoreScanner constructor to accept List
[ https://issues.apache.org/jira/browse/HBASE-16143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-16143: --- Attachment: HBASE-16143.patch Patch changing the constructor of MemstoreScanner and removing SegmentScanner#shouldSeek(). We could call the KeyValueScanner#shouldUseScanner() only instead. > Change MemstoreScanner constructor to accept List > -- > > Key: HBASE-16143 > URL: https://issues.apache.org/jira/browse/HBASE-16143 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-16143.patch > > > A minor change that helps in creating a memstore that avoids the compaction > process and just allows to creates a pipeline of segments and on flush > directly reads the segments in the pipeline and flushes it out after creating > a snapshot of the pipeline. Based on test results and updated patch on > HBASE-14921 (to be provided) will see how much flattening helps us. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15800) check_compatibility.sh broken as of upstream b8f125f
[
https://issues.apache.org/jira/browse/HBASE-15800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354653#comment-15354653
]
Dima Spivak commented on HBASE-15800:
-
If you have a few minutes, would you mind trying out v3 of the patch I posted
over in HBASE-16129, [~ndimiduk]? Had a chance to dig into this and I think the
problem was how we specified annotations, which is fixed in that JIRA.
> check_compatibility.sh broken as of upstream b8f125f
>
>
> Key: HBASE-15800
> URL: https://issues.apache.org/jira/browse/HBASE-15800
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 2.0.0
>Reporter: Nick Dimiduk
>Priority: Minor
>
> Previously (recently as 2 weeks ago) it was possible to use the
> {{dev-support/check_compatibility.sh}} script against the tags under {{rel}}.
> Now ({{5e0}}) this is no longer the case. On a mac,
> {noformat}
> $ echo $JAVA_HOME
> /Library/Java/JavaVirtualMachines/jdk1.7.0_80.jdk/Contents/Home
> $ GETOPT=/usr/local/Cellar/gnu-getopt/1.1.6/bin/getopt
> ./dev-support/check_compatibility.sh -r
> https://git-wip-us.apache.org/repos/asf/hbase.git rel/1.1.4 branch-1.1
> ...
> Running the Java API Compliance Checker...
> ERROR: can't access
> './dev-support/target/compatibility/1/hbase-annotations/target/hbase-annotations-1.1.4.jar,./dev-support/target/compatibility/1/hbase-checkstyle/target/hbase-checkstyle-1.1.
> 4.jar,./dev-support/target/compatibility/1/hbase-client/target/hbase-client-1.1.4.jar,./dev-support/target/compatibility/1/hbase-common/target/hbase-common-1.1.4.jar,./dev-support/target/compat
> ibility/1/hbase-examples/target/hbase-examples-1.1.4.jar,./dev-support/target/compatibility/1/hbase-hadoop-compat/target/hbase-hadoop-compat-1.1.4.jar,./dev-support/target/compatibility/1/hbase
> -hadoop2-compat/target/hbase-hadoop2-compat-1.1.4.jar,./dev-support/target/compatibility/1/hbase-it/target/hbase-it-1.1.4.jar,./dev-support/target/compatibility/1/hbase-prefix-tree/target/hbase
> -prefix-tree-1.1.4.jar,./dev-support/target/compatibility/1/hbase-procedure/target/hbase-procedure-1.1.4.jar,./dev-support/target/compatibility/1/hbase-protocol/target/hbase-protocol-1.1.4.jar,
> ./dev-support/target/compatibility/1/hbase-resource-bundle/target/hbase-resource-bundle-1.1.4.jar,./dev-support/target/compatibility/1/hbase-rest/target/hbase-rest-1.1.4.jar,./dev-support/targe
> t/compatibility/1/hbase-server/target/hbase-server-1.1.4.jar,./dev-support/target/compatibility/1/hbase-shell/target/hbase-shell-1.1.4.jar,./dev-support/target/compatibility/1/hbase-testing-uti
> l/target/hbase-testing-util-1.1.4.jar,./dev-support/target/compatibility/1/hbase-thrift/target/hbase-thrift-1.1.4.jar'
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16129) check_compatibility.sh is broken when using Java API Compliance Checker v1.7
[ https://issues.apache.org/jira/browse/HBASE-16129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak updated HBASE-16129: Attachment: HBASE-16129_v3.patch Attaching a patch that I think this fixes things once and for all. The problem we were having had to do with our specification of annotations; whereas in the past we could just provide a simple name, later versions of Java ACC require specification of the canonical name. With that in place, things seem to behave again. > check_compatibility.sh is broken when using Java API Compliance Checker v1.7 > > > Key: HBASE-16129 > URL: https://issues.apache.org/jira/browse/HBASE-16129 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Dima Spivak >Assignee: Dima Spivak > Attachments: HBASE-16129_v1.patch, HBASE-16129_v2.patch, > HBASE-16129_v3.patch > > > As part of HBASE-16073, we hardcoded check_compatiblity.sh to check out the > v1.7 tag of Java ACC. Unfortunately, just running it between two branches > that I know have incompatibilities, I get 0 incompatibilities (and 0 classes > read). Looks like this version doesn't properly traverse through JARs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16143) Change MemstoreScanner constructor to accept List
ramkrishna.s.vasudevan created HBASE-16143: -- Summary: Change MemstoreScanner constructor to accept List Key: HBASE-16143 URL: https://issues.apache.org/jira/browse/HBASE-16143 Project: HBase Issue Type: Sub-task Affects Versions: 2.0.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Priority: Minor Fix For: 2.0.0 A minor change that helps in creating a memstore that avoids the compaction process and just allows to creates a pipeline of segments and on flush directly reads the segments in the pipeline and flushes it out after creating a snapshot of the pipeline. Based on test results and updated patch on HBASE-14921 (to be provided) will see how much flattening helps us. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16106) HBase Rest API: unexpected behavior of get with timestamp
[ https://issues.apache.org/jira/browse/HBASE-16106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354645#comment-15354645 ] Blaye Nicolas commented on HBASE-16106: --- Thank you for your precise answer. About the rest api behavior, do you consider any change? > HBase Rest API: unexpected behavior of get with timestamp > - > > Key: HBASE-16106 > URL: https://issues.apache.org/jira/browse/HBASE-16106 > Project: HBase > Issue Type: Bug > Components: API, REST >Affects Versions: 1.2.1 >Reporter: Blaye Nicolas >Priority: Minor > Labels: easyfix, newbie > > Issue seen there: > http://stackoverflow.com/questions/37985426/hbase-get-request-for-row-data-with-timestamp?noredirect=1#comment63464266_37985426 > > The *adress:port/table/row/column/timestamp* returns the first value > *strictly inferior* to the timestamp provided. > This behavior is not the one seen in bash and java as explained in the > question. > I haven't found the bug here but I may be wrong, and it may be fixed already. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16134) Introduce InternalCell
[
https://issues.apache.org/jira/browse/HBASE-16134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354639#comment-15354639
]
Hadoop QA commented on HBASE-16134:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
5s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 22s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 17s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
9s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
45s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 41s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
57s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 17s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 45s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 37m 45s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12814579/HBASE-16134_V2.patch |
| JIRA Issue | HBASE-16134 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 394e722 |
| Default Java | 1.7.0_80 |
| Multi-JDK versions | /home/jenkins/tools/java/jdk1.8.0:1.8.0
/home/jenkins/jenkins-slave/tools/hudson.model.
[jira] [Commented] (HBASE-15976) RegionServerMetricsWrapperRunnable will be failure when disable blockcache.
[
https://issues.apache.org/jira/browse/HBASE-15976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354636#comment-15354636
]
Jingcheng Du commented on HBASE-15976:
--
Thanks for pointing it out [~ndimiduk]. Most of the code check the null before
using. But still one method misses that {{public long
getBlockCacheFailedInsertions()}}.
This is my mistake, I will add an addendum for master and branch-1, and provide
new patches for other branches. Thanks.
> RegionServerMetricsWrapperRunnable will be failure when disable blockcache.
>
>
> Key: HBASE-15976
> URL: https://issues.apache.org/jira/browse/HBASE-15976
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.15, 1.2.1, 1.0.3, 1.1.5
>Reporter: Liu Junhong
>Assignee: Jingcheng Du
> Attachments: HBASE-15976-0.98.patch,
> HBASE-15976-branch-1&branch-1.3.patch, HBASE-15976-branch-1.0.patch,
> HBASE-15976-branch-1.1.patch, HBASE-15976-branch-1.2.patch,
> HBASE-15976-master.patch
>
>
> When i disable blockcache, the code "cacheStats = blockCache.getStats();"
> will occur NPE in
> org.apache.hadoop.hbase.regionserver.MetricsRegionServerWrapperImpl.RegionServerMetricsWrapperRunnable.
> It lead to many regionserver's metrics' value always equal 0.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16122) PerformanceEvaluation should provide user friendly hint when client threads argument is missing
[
https://issues.apache.org/jira/browse/HBASE-16122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354633#comment-15354633
]
Hudson commented on HBASE-16122:
SUCCESS: Integrated in HBase-1.4 #257 (See
[https://builds.apache.org/job/HBase-1.4/257/])
HBASE-16122 PerformanceEvaluation should provide user friendly hint when
(tedyu: rev 96a24aede7760974df7ea9e762e1d63d658abd55)
* hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/TestPerformanceEvaluation.java
> PerformanceEvaluation should provide user friendly hint when client threads
> argument is missing
> ---
>
> Key: HBASE-16122
> URL: https://issues.apache.org/jira/browse/HBASE-16122
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Konstantin Ryakhovskiy
>Priority: Minor
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-16122-v2.master.patch
>
>
> I tried to run this command:
> hbase pe --rows=1 --nomapred --valueSize=200 sequentialWrite
> {code}
> java.util.NoSuchElementException
> at java.util.LinkedList.removeFirst(LinkedList.java:270)
> at java.util.LinkedList.remove(LinkedList.java:685)
> at
> org.apache.hadoop.hbase.PerformanceEvaluation.parseOpts(PerformanceEvaluation.java:2077)
> at
> org.apache.hadoop.hbase.PerformanceEvaluation.run(PerformanceEvaluation.java:2122)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at
> org.apache.hadoop.hbase.PerformanceEvaluation.main(PerformanceEvaluation.java:2159)
> {code}
> Number of client threads argument was missing.
> PerformanceEvaluation should print user friendly message informing user of
> the missing argument.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15965) Shell test changes. Use @shell.command instead directly calling functions in admin.rb and other libraries.
[ https://issues.apache.org/jira/browse/HBASE-15965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354631#comment-15354631 ] Hudson commented on HBASE-15965: SUCCESS: Integrated in HBase-1.4 #257 (See [https://builds.apache.org/job/HBase-1.4/257/]) Revert HBASE-15965 and HBASE-15849. While it's fine to introduce these (appy: rev 48492ec7fd72a89ac67b2ef834ccfa8021fbadd5) * hbase-shell/src/main/ruby/shell/commands/delete_all_snapshot.rb * hbase-shell/src/main/ruby/shell/commands.rb * hbase-shell/src/main/ruby/shell/commands/get_peer_config.rb * hbase-shell/src/main/ruby/shell/commands/list.rb * hbase-shell/src/main/ruby/shell/commands/list_replicated_tables.rb * hbase-shell/src/test/ruby/test_helper.rb * hbase-shell/src/test/ruby/hbase/replication_admin_test.rb * hbase-shell/src/main/ruby/shell/commands/drop_namespace.rb * hbase-shell/src/main/ruby/shell/commands/update_config.rb * hbase-shell/src/test/ruby/hbase/admin_test.rb * hbase-shell/src/main/ruby/shell/commands/create_namespace.rb * hbase-shell/src/main/ruby/shell/commands/remove_peer_tableCFs.rb * hbase-shell/src/test/ruby/shell/shell_test.rb * hbase-shell/src/main/ruby/shell/commands/add_labels.rb * hbase-shell/src/main/ruby/shell/commands/list_procedures.rb * hbase-shell/src/main/ruby/shell/commands/locate_region.rb * hbase-shell/src/main/ruby/shell/commands/drop.rb * hbase-shell/src/main/ruby/shell/commands/normalize.rb * hbase-shell/src/main/ruby/shell/commands/scan.rb * hbase-shell/src/main/ruby/shell/commands/describe_namespace.rb * hbase-shell/src/main/ruby/shell/commands/close_region.rb * hbase-shell/src/main/ruby/shell/commands/truncate_preserve.rb * hbase-shell/src/main/ruby/shell/commands/wal_roll.rb * hbase-shell/src/main/ruby/shell/commands/splitormerge_enabled.rb * hbase-shell/src/main/ruby/shell/commands/catalogjanitor_enabled.rb * hbase-shell/src/main/ruby/shell/commands/get_table.rb * hbase-shell/src/main/ruby/hbase/table.rb * hbase-shell/src/main/ruby/shell/commands/incr.rb * hbase-shell/src/main/ruby/shell/commands/create.rb * hbase-shell/src/main/ruby/shell/commands/delete_snapshot.rb * hbase-shell/src/main/ruby/shell/commands/alter_namespace.rb * hbase-shell/src/main/ruby/shell/commands/disable.rb * hbase-shell/src/main/ruby/shell/commands/is_enabled.rb * hbase-shell/src/main/ruby/shell/commands/list_table_snapshots.rb * hbase-shell/src/main/ruby/shell/commands/get.rb * hbase-shell/src/main/ruby/shell/commands/assign.rb * hbase-shell/src/main/ruby/shell/commands/snapshot.rb * hbase-shell/src/main/ruby/shell/commands/flush.rb * hbase-shell/src/main/ruby/shell/commands/add_peer.rb * hbase-shell/src/main/ruby/shell/commands/list_peer_configs.rb * hbase-shell/src/main/ruby/shell/commands/enable.rb * hbase-shell/src/main/ruby/shell/commands/alter_async.rb * hbase-shell/src/main/ruby/shell/commands/revoke.rb * hbase-shell/src/main/ruby/shell/commands/append_peer_tableCFs.rb * hbase-shell/src/main/ruby/shell/commands/show_peer_tableCFs.rb * hbase-shell/src/main/ruby/shell/commands/alter.rb * hbase-shell/src/main/ruby/shell/commands/truncate.rb * hbase-shell/src/main/ruby/shell/commands/list_labels.rb * hbase-shell/src/main/ruby/shell/commands/normalizer_switch.rb * hbase-shell/src/main/ruby/shell/commands/count.rb * hbase-shell/src/main/ruby/shell/commands/abort_procedure.rb * hbase-shell/src/main/ruby/shell/commands/balancer.rb * hbase-shell/src/main/ruby/shell/commands/user_permission.rb * hbase-shell/src/main/ruby/shell/commands/move.rb * hbase-shell/src/main/ruby/shell/commands/deleteall.rb * hbase-shell/src/main/ruby/shell/commands/compact.rb * hbase-shell/src/main/ruby/shell/commands/compact_rs.rb * hbase-shell/src/main/ruby/shell/commands/list_quotas.rb * hbase-shell/src/main/ruby/shell/commands/update_peer_config.rb * hbase-shell/src/main/ruby/shell/commands/enable_peer.rb * hbase-shell/src/test/java/org/apache/hadoop/hbase/client/TestReplicationShell.java * hbase-shell/src/main/ruby/shell/commands/delete_table_snapshots.rb * hbase-shell/src/main/ruby/shell/commands/balance_switch.rb * hbase-shell/src/main/ruby/shell/commands/list_namespace_tables.rb * hbase-shell/src/main/ruby/shell/commands/split.rb * hbase-shell/src/main/ruby/shell/commands/put.rb * hbase-shell/src/main/ruby/shell.rb * hbase-shell/src/main/ruby/shell/commands/merge_region.rb * hbase-shell/src/main/ruby/shell/commands/set_peer_tableCFs.rb * hbase-shell/src/main/ruby/shell/commands/splitormerge_switch.rb * hbase-shell/src/main/ruby/shell/commands/clone_snapshot.rb * hbase-shell/src/main/ruby/shell/commands/major_compact.rb * hbase-shell/src/main/ruby/shell/commands/normalizer_enabled.rb * hbase-shell/src/main/ruby/shell/commands/describe.rb * hbase-shell/src/main/ruby/shell/commands/balancer_enabled.rb * hbase-shell/src/main/ruby/shell/commands/list_peers.rb * hbase-shell/src/main/ruby/shell/commands/grant.rb * hbase-shell/src/main/ruby/shell/commands/catalogjanitor_switch.
[jira] [Commented] (HBASE-15849) Shell Cleanup: Simplify handling of commands' runtime
[ https://issues.apache.org/jira/browse/HBASE-15849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354632#comment-15354632 ] Hudson commented on HBASE-15849: SUCCESS: Integrated in HBase-1.4 #257 (See [https://builds.apache.org/job/HBase-1.4/257/]) Revert HBASE-15965 and HBASE-15849. While it's fine to introduce these (appy: rev 48492ec7fd72a89ac67b2ef834ccfa8021fbadd5) * hbase-shell/src/main/ruby/shell/commands/delete_all_snapshot.rb * hbase-shell/src/main/ruby/shell/commands.rb * hbase-shell/src/main/ruby/shell/commands/get_peer_config.rb * hbase-shell/src/main/ruby/shell/commands/list.rb * hbase-shell/src/main/ruby/shell/commands/list_replicated_tables.rb * hbase-shell/src/test/ruby/test_helper.rb * hbase-shell/src/test/ruby/hbase/replication_admin_test.rb * hbase-shell/src/main/ruby/shell/commands/drop_namespace.rb * hbase-shell/src/main/ruby/shell/commands/update_config.rb * hbase-shell/src/test/ruby/hbase/admin_test.rb * hbase-shell/src/main/ruby/shell/commands/create_namespace.rb * hbase-shell/src/main/ruby/shell/commands/remove_peer_tableCFs.rb * hbase-shell/src/test/ruby/shell/shell_test.rb * hbase-shell/src/main/ruby/shell/commands/add_labels.rb * hbase-shell/src/main/ruby/shell/commands/list_procedures.rb * hbase-shell/src/main/ruby/shell/commands/locate_region.rb * hbase-shell/src/main/ruby/shell/commands/drop.rb * hbase-shell/src/main/ruby/shell/commands/normalize.rb * hbase-shell/src/main/ruby/shell/commands/scan.rb * hbase-shell/src/main/ruby/shell/commands/describe_namespace.rb * hbase-shell/src/main/ruby/shell/commands/close_region.rb * hbase-shell/src/main/ruby/shell/commands/truncate_preserve.rb * hbase-shell/src/main/ruby/shell/commands/wal_roll.rb * hbase-shell/src/main/ruby/shell/commands/splitormerge_enabled.rb * hbase-shell/src/main/ruby/shell/commands/catalogjanitor_enabled.rb * hbase-shell/src/main/ruby/shell/commands/get_table.rb * hbase-shell/src/main/ruby/hbase/table.rb * hbase-shell/src/main/ruby/shell/commands/incr.rb * hbase-shell/src/main/ruby/shell/commands/create.rb * hbase-shell/src/main/ruby/shell/commands/delete_snapshot.rb * hbase-shell/src/main/ruby/shell/commands/alter_namespace.rb * hbase-shell/src/main/ruby/shell/commands/disable.rb * hbase-shell/src/main/ruby/shell/commands/is_enabled.rb * hbase-shell/src/main/ruby/shell/commands/list_table_snapshots.rb * hbase-shell/src/main/ruby/shell/commands/get.rb * hbase-shell/src/main/ruby/shell/commands/assign.rb * hbase-shell/src/main/ruby/shell/commands/snapshot.rb * hbase-shell/src/main/ruby/shell/commands/flush.rb * hbase-shell/src/main/ruby/shell/commands/add_peer.rb * hbase-shell/src/main/ruby/shell/commands/list_peer_configs.rb * hbase-shell/src/main/ruby/shell/commands/enable.rb * hbase-shell/src/main/ruby/shell/commands/alter_async.rb * hbase-shell/src/main/ruby/shell/commands/revoke.rb * hbase-shell/src/main/ruby/shell/commands/append_peer_tableCFs.rb * hbase-shell/src/main/ruby/shell/commands/show_peer_tableCFs.rb * hbase-shell/src/main/ruby/shell/commands/alter.rb * hbase-shell/src/main/ruby/shell/commands/truncate.rb * hbase-shell/src/main/ruby/shell/commands/list_labels.rb * hbase-shell/src/main/ruby/shell/commands/normalizer_switch.rb * hbase-shell/src/main/ruby/shell/commands/count.rb * hbase-shell/src/main/ruby/shell/commands/abort_procedure.rb * hbase-shell/src/main/ruby/shell/commands/balancer.rb * hbase-shell/src/main/ruby/shell/commands/user_permission.rb * hbase-shell/src/main/ruby/shell/commands/move.rb * hbase-shell/src/main/ruby/shell/commands/deleteall.rb * hbase-shell/src/main/ruby/shell/commands/compact.rb * hbase-shell/src/main/ruby/shell/commands/compact_rs.rb * hbase-shell/src/main/ruby/shell/commands/list_quotas.rb * hbase-shell/src/main/ruby/shell/commands/update_peer_config.rb * hbase-shell/src/main/ruby/shell/commands/enable_peer.rb * hbase-shell/src/test/java/org/apache/hadoop/hbase/client/TestReplicationShell.java * hbase-shell/src/main/ruby/shell/commands/delete_table_snapshots.rb * hbase-shell/src/main/ruby/shell/commands/balance_switch.rb * hbase-shell/src/main/ruby/shell/commands/list_namespace_tables.rb * hbase-shell/src/main/ruby/shell/commands/split.rb * hbase-shell/src/main/ruby/shell/commands/put.rb * hbase-shell/src/main/ruby/shell.rb * hbase-shell/src/main/ruby/shell/commands/merge_region.rb * hbase-shell/src/main/ruby/shell/commands/set_peer_tableCFs.rb * hbase-shell/src/main/ruby/shell/commands/splitormerge_switch.rb * hbase-shell/src/main/ruby/shell/commands/clone_snapshot.rb * hbase-shell/src/main/ruby/shell/commands/major_compact.rb * hbase-shell/src/main/ruby/shell/commands/normalizer_enabled.rb * hbase-shell/src/main/ruby/shell/commands/describe.rb * hbase-shell/src/main/ruby/shell/commands/balancer_enabled.rb * hbase-shell/src/main/ruby/shell/commands/list_peers.rb * hbase-shell/src/main/ruby/shell/commands/grant.rb * hbase-shell/src/main/ruby/shell/commands/catalogjanitor_switch.
[jira] [Commented] (HBASE-14921) Memory optimizations
[ https://issues.apache.org/jira/browse/HBASE-14921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354612#comment-15354612 ] ramkrishna.s.vasudevan commented on HBASE-14921: I still doubt the sizing part of the memstore. Trying to find the root cause. Will get back here. BTW any updates [~anastas]? > Memory optimizations > > > Key: HBASE-14921 > URL: https://issues.apache.org/jira/browse/HBASE-14921 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Eshcar Hillel >Assignee: Anastasia Braginsky > Attachments: CellBlocksSegmentInMemStore.pdf, > CellBlocksSegmentinthecontextofMemStore(1).pdf, HBASE-14921-V01.patch, > HBASE-14921-V02.patch, HBASE-14921-V03.patch, HBASE-14921-V04-CA.patch, > InitialCellArrayMapEvaluation.pdf, IntroductiontoNewFlatandCompactMemStore.pdf > > > Memory optimizations including compressed format representation and offheap > allocations -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16130) Add comments to ProcedureStoreTracker
[ https://issues.apache.org/jira/browse/HBASE-16130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354610#comment-15354610 ] Matteo Bertozzi commented on HBASE-16130: - +1 > Add comments to ProcedureStoreTracker > - > > Key: HBASE-16130 > URL: https://issues.apache.org/jira/browse/HBASE-16130 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy >Priority: Minor > Attachments: HBASE-16130.master.001.patch, > HBASE-16130.master.002.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16134) Introduce InternalCell
[ https://issues.apache.org/jira/browse/HBASE-16134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354608#comment-15354608 ] Anoop Sam John commented on HBASE-16134: We do have 2 Codecs , with and without Tags [~saint@gmail.com]. The issue is different. The Streamable interface (and now the method in new interface) helps us to write whole of the Cell (Key, Value , Tags) in one shot to the OS. Or else the Codec impl has to parse all the lengths and write each item one after the other. When the Cell is having the write(OS) implemented, the Codec just calls that. The Cell impl will do these writes. Now how we can instruct the Cell object to write tags or not with out passing any param? So the simple way of passing boolean was adopted. Or else we will need pass some CodecContext or so. Or else the Context to be avail in a ThreadLocal or so. But that is perf overhead. Another way would be to make the contract of this write(OS) to write only Key and Value but not tags. That can be done by the Codec itself. Thoughts? > Introduce InternalCell > -- > > Key: HBASE-16134 > URL: https://issues.apache.org/jira/browse/HBASE-16134 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0 > > Attachments: HBASE-16134.patch, HBASE-16134.patch, > HBASE-16134_V2.patch > > > Came after the discussion under HBASE-15721 and HBASE-15879. > InternalCell is a Cell extension. We do have Cell extensions across different > interfaces now. > SettableSeqId > SettableTimestamp > Streamable. > And demand for this keep growing. > So let us include everything into one Interface. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15716) HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random read
[ https://issues.apache.org/jira/browse/HBASE-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354603#comment-15354603 ] Hiroshi Ikeda commented on HBASE-15716: --- I find I'm an administrator by the wheel icon, but still don't find the menu item. > HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random > read > -- > > Key: HBASE-15716 > URL: https://issues.apache.org/jira/browse/HBASE-15716 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: stack >Assignee: stack > Attachments: 15716.prune.synchronizations.patch, > 15716.prune.synchronizations.v3.patch, 15716.prune.synchronizations.v4.patch, > 15716.prune.synchronizations.v4.patch, 15716.wip.more_to_be_done.patch, > HBASE-15716.branch-1.001.patch, HBASE-15716.branch-1.002.patch, > HBASE-15716.branch-1.003.patch, HBASE-15716.branch-1.004.patch, Screen Shot > 2016-04-26 at 2.05.45 PM.png, Screen Shot 2016-04-26 at 2.06.14 PM.png, > Screen Shot 2016-04-26 at 2.07.06 PM.png, Screen Shot 2016-04-26 at 2.25.26 > PM.png, Screen Shot 2016-04-26 at 6.02.29 PM.png, Screen Shot 2016-04-27 at > 9.49.35 AM.png, before_after.png, > current-branch-1.vs.NoSynchronization.vs.Patch.png, hits.png, > remove.locks.patch, remove_cslm.patch > > > Here is a [~lhofhansl] special. > When we construct the region scanner, we get our read point and then store it > with the scanner instance in a Region scoped CSLM. This is done under a > synchronize on the CSLM. > This synchronize on a region-scoped Map creating region scanners is the > outstanding point of lock contention according to flight recorder (My work > load is workload c, random reads). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15716) HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random read
[ https://issues.apache.org/jira/browse/HBASE-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354601#comment-15354601 ] Phil Yang commented on HBASE-15716: --- Try "assign to me" first, seems only assignee can attach files > HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random > read > -- > > Key: HBASE-15716 > URL: https://issues.apache.org/jira/browse/HBASE-15716 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: stack >Assignee: stack > Attachments: 15716.prune.synchronizations.patch, > 15716.prune.synchronizations.v3.patch, 15716.prune.synchronizations.v4.patch, > 15716.prune.synchronizations.v4.patch, 15716.wip.more_to_be_done.patch, > HBASE-15716.branch-1.001.patch, HBASE-15716.branch-1.002.patch, > HBASE-15716.branch-1.003.patch, HBASE-15716.branch-1.004.patch, Screen Shot > 2016-04-26 at 2.05.45 PM.png, Screen Shot 2016-04-26 at 2.06.14 PM.png, > Screen Shot 2016-04-26 at 2.07.06 PM.png, Screen Shot 2016-04-26 at 2.25.26 > PM.png, Screen Shot 2016-04-26 at 6.02.29 PM.png, Screen Shot 2016-04-27 at > 9.49.35 AM.png, before_after.png, > current-branch-1.vs.NoSynchronization.vs.Patch.png, hits.png, > remove.locks.patch, remove_cslm.patch > > > Here is a [~lhofhansl] special. > When we construct the region scanner, we get our read point and then store it > with the scanner instance in a Region scoped CSLM. This is done under a > synchronize on the CSLM. > This synchronize on a region-scoped Map creating region scanners is the > outstanding point of lock contention according to flight recorder (My work > load is workload c, random reads). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16134) Introduce InternalCell
[ https://issues.apache.org/jira/browse/HBASE-16134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354598#comment-15354598 ] stack commented on HBASE-16134: --- Can we not have a WithTagsCodec and a WithoutTagsCodec and then dependent on context do one or the other? (I've asked this before and the answer is not neat and tidy if I remember correctly). Tags needs to be integral, not an appendage requires that we override a bunch of our API just to add the withTags... Can do in another issue. This one is doing enough already. > Introduce InternalCell > -- > > Key: HBASE-16134 > URL: https://issues.apache.org/jira/browse/HBASE-16134 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0 > > Attachments: HBASE-16134.patch, HBASE-16134.patch, > HBASE-16134_V2.patch > > > Came after the discussion under HBASE-15721 and HBASE-15879. > InternalCell is a Cell extension. We do have Cell extensions across different > interfaces now. > SettableSeqId > SettableTimestamp > Streamable. > And demand for this keep growing. > So let us include everything into one Interface. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16134) Introduce InternalCell
[ https://issues.apache.org/jira/browse/HBASE-16134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-16134: --- Attachment: HBASE-16134_V2.patch > Introduce InternalCell > -- > > Key: HBASE-16134 > URL: https://issues.apache.org/jira/browse/HBASE-16134 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0 > > Attachments: HBASE-16134.patch, HBASE-16134.patch, > HBASE-16134_V2.patch > > > Came after the discussion under HBASE-15721 and HBASE-15879. > InternalCell is a Cell extension. We do have Cell extensions across different > interfaces now. > SettableSeqId > SettableTimestamp > Streamable. > And demand for this keep growing. > So let us include everything into one Interface. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15716) HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random read
[ https://issues.apache.org/jira/browse/HBASE-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354592#comment-15354592 ] stack commented on HBASE-15716: --- [~ikeda] I made you an administrator. Try now. > HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random > read > -- > > Key: HBASE-15716 > URL: https://issues.apache.org/jira/browse/HBASE-15716 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: stack >Assignee: stack > Attachments: 15716.prune.synchronizations.patch, > 15716.prune.synchronizations.v3.patch, 15716.prune.synchronizations.v4.patch, > 15716.prune.synchronizations.v4.patch, 15716.wip.more_to_be_done.patch, > HBASE-15716.branch-1.001.patch, HBASE-15716.branch-1.002.patch, > HBASE-15716.branch-1.003.patch, HBASE-15716.branch-1.004.patch, Screen Shot > 2016-04-26 at 2.05.45 PM.png, Screen Shot 2016-04-26 at 2.06.14 PM.png, > Screen Shot 2016-04-26 at 2.07.06 PM.png, Screen Shot 2016-04-26 at 2.25.26 > PM.png, Screen Shot 2016-04-26 at 6.02.29 PM.png, Screen Shot 2016-04-27 at > 9.49.35 AM.png, before_after.png, > current-branch-1.vs.NoSynchronization.vs.Patch.png, hits.png, > remove.locks.patch, remove_cslm.patch > > > Here is a [~lhofhansl] special. > When we construct the region scanner, we get our read point and then store it > with the scanner instance in a Region scoped CSLM. This is done under a > synchronize on the CSLM. > This synchronize on a region-scoped Map creating region scanners is the > outstanding point of lock contention according to flight recorder (My work > load is workload c, random reads). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16142) Trigger JFR session when under duress -- e.g. backed-up request queue count -- and dump the recording to log dir
[ https://issues.apache.org/jira/browse/HBASE-16142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354591#comment-15354591 ] Andrew Purtell commented on HBASE-16142: Cool idea, but it should be configurable and default off. JFR is not licensed by Oracle for free use in production. > Trigger JFR session when under duress -- e.g. backed-up request queue count > -- and dump the recording to log dir > > > Key: HBASE-16142 > URL: https://issues.apache.org/jira/browse/HBASE-16142 > Project: HBase > Issue Type: Task > Components: Operability >Reporter: stack >Priority: Minor > Labels: beginner > > Chatting today w/ a mighty hbase operator on how to figure what is happening > during transitory latency spike or any other transitory 'weirdness' in a > server, the idea came up that a java flight recording during a spike would > include a pretty good picture of what is going on during the time of duress > (more ideal would be a trace of the explicit slow queries showing call stack > with timings dumped to a sink for later review; i.e. trigger an htrace when a > query is slow...). > Taking a look, programmatically triggering a JFR recording seems doable, if > awkward (MBean invocations). There is even a means of specifying 'triggers' > based off any published mbean emission -- e.g. a query queue count threshold > -- which looks nice. See > https://community.oracle.com/thread/3676275?start=0&tstart=0 and > https://docs.oracle.com/javacomponents/jmc-5-4/jfr-runtime-guide/run.htm#JFRUH184 > This feature could start out as a blog post describing how to do it for one > server. A plugin on Canary that looks at mbean values and if over a > configured threshold, triggers a recording remotely could be next. Finally > could integrate a couple of triggers that fire when issue via the trigger > mechanism. > Marking as beginner feature. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15716) HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random read
[ https://issues.apache.org/jira/browse/HBASE-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354589#comment-15354589 ] stack commented on HBASE-15716: --- You are logged in? > HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random > read > -- > > Key: HBASE-15716 > URL: https://issues.apache.org/jira/browse/HBASE-15716 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: stack >Assignee: stack > Attachments: 15716.prune.synchronizations.patch, > 15716.prune.synchronizations.v3.patch, 15716.prune.synchronizations.v4.patch, > 15716.prune.synchronizations.v4.patch, 15716.wip.more_to_be_done.patch, > HBASE-15716.branch-1.001.patch, HBASE-15716.branch-1.002.patch, > HBASE-15716.branch-1.003.patch, HBASE-15716.branch-1.004.patch, Screen Shot > 2016-04-26 at 2.05.45 PM.png, Screen Shot 2016-04-26 at 2.06.14 PM.png, > Screen Shot 2016-04-26 at 2.07.06 PM.png, Screen Shot 2016-04-26 at 2.25.26 > PM.png, Screen Shot 2016-04-26 at 6.02.29 PM.png, Screen Shot 2016-04-27 at > 9.49.35 AM.png, before_after.png, > current-branch-1.vs.NoSynchronization.vs.Patch.png, hits.png, > remove.locks.patch, remove_cslm.patch > > > Here is a [~lhofhansl] special. > When we construct the region scanner, we get our read point and then store it > with the scanner instance in a Region scoped CSLM. This is done under a > synchronize on the CSLM. > This synchronize on a region-scoped Map creating region scanners is the > outstanding point of lock contention according to flight recorder (My work > load is workload c, random reads). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16134) Introduce InternalCell
[ https://issues.apache.org/jira/browse/HBASE-16134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354587#comment-15354587 ] Anoop Sam John commented on HBASE-16134: We need the withTags boolean param as diff Codecs pass true/false for this. The entire Cell bytes is written to OS by the Cell implementation itself. So it has to know whether tags bytes to be written or not. > Introduce InternalCell > -- > > Key: HBASE-16134 > URL: https://issues.apache.org/jira/browse/HBASE-16134 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0 > > Attachments: HBASE-16134.patch, HBASE-16134.patch > > > Came after the discussion under HBASE-15721 and HBASE-15879. > InternalCell is a Cell extension. We do have Cell extensions across different > interfaces now. > SettableSeqId > SettableTimestamp > Streamable. > And demand for this keep growing. > So let us include everything into one Interface. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15716) HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random read
[ https://issues.apache.org/jira/browse/HBASE-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354577#comment-15354577 ] Hiroshi Ikeda commented on HBASE-15716: --- I still don't find a menu item or something to attach files :( > HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random > read > -- > > Key: HBASE-15716 > URL: https://issues.apache.org/jira/browse/HBASE-15716 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: stack >Assignee: stack > Attachments: 15716.prune.synchronizations.patch, > 15716.prune.synchronizations.v3.patch, 15716.prune.synchronizations.v4.patch, > 15716.prune.synchronizations.v4.patch, 15716.wip.more_to_be_done.patch, > HBASE-15716.branch-1.001.patch, HBASE-15716.branch-1.002.patch, > HBASE-15716.branch-1.003.patch, HBASE-15716.branch-1.004.patch, Screen Shot > 2016-04-26 at 2.05.45 PM.png, Screen Shot 2016-04-26 at 2.06.14 PM.png, > Screen Shot 2016-04-26 at 2.07.06 PM.png, Screen Shot 2016-04-26 at 2.25.26 > PM.png, Screen Shot 2016-04-26 at 6.02.29 PM.png, Screen Shot 2016-04-27 at > 9.49.35 AM.png, before_after.png, > current-branch-1.vs.NoSynchronization.vs.Patch.png, hits.png, > remove.locks.patch, remove_cslm.patch > > > Here is a [~lhofhansl] special. > When we construct the region scanner, we get our read point and then store it > with the scanner instance in a Region scoped CSLM. This is done under a > synchronize on the CSLM. > This synchronize on a region-scoped Map creating region scanners is the > outstanding point of lock contention according to flight recorder (My work > load is workload c, random reads). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16142) Trigger JFR session when under duress -- e.g. backed-up request queue count -- and dump the recording to log dir
stack created HBASE-16142: - Summary: Trigger JFR session when under duress -- e.g. backed-up request queue count -- and dump the recording to log dir Key: HBASE-16142 URL: https://issues.apache.org/jira/browse/HBASE-16142 Project: HBase Issue Type: Task Components: Operability Reporter: stack Priority: Minor Chatting today w/ a mighty hbase operator on how to figure what is happening during transitory latency spike or any other transitory 'weirdness' in a server, the idea came up that a java flight recording during a spike would include a pretty good picture of what is going on during the time of duress (more ideal would be a trace of the explicit slow queries showing call stack with timings dumped to a sink for later review; i.e. trigger an htrace when a query is slow...). Taking a look, programmatically triggering a JFR recording seems doable, if awkward (MBean invocations). There is even a means of specifying 'triggers' based off any published mbean emission -- e.g. a query queue count threshold -- which looks nice. See https://community.oracle.com/thread/3676275?start=0&tstart=0 and https://docs.oracle.com/javacomponents/jmc-5-4/jfr-runtime-guide/run.htm#JFRUH184 This feature could start out as a blog post describing how to do it for one server. A plugin on Canary that looks at mbean values and if over a configured threshold, triggers a recording remotely could be next. Finally could integrate a couple of triggers that fire when issue via the trigger mechanism. Marking as beginner feature. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16141) Unwind use of UserGroupInformation.doAs() to convey requester identity in coprocessor upcalls
Gary Helmling created HBASE-16141: - Summary: Unwind use of UserGroupInformation.doAs() to convey requester identity in coprocessor upcalls Key: HBASE-16141 URL: https://issues.apache.org/jira/browse/HBASE-16141 Project: HBase Issue Type: Improvement Components: Coprocessors, security Reporter: Gary Helmling Assignee: Gary Helmling Fix For: 2.0.0, 1.4.0 In discussion on HBASE-16115, there is some discussion of whether UserGroupInformation.doAs() is the right mechanism for propagating the original requester's identify in certain system contexts (splits, compactions, some procedure calls). It has the unfortunately of overriding the current user, which makes for very confusing semantics for coprocessor implementors. We should instead find an alternate mechanism for conveying the caller identity, which does not override the current user context. I think we should instead look at passing this through as part of the ObserverContext passed to every coprocessor hook. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16115) Missing security context in RegionObserver coprocessor when a compaction/split is triggered manually
[ https://issues.apache.org/jira/browse/HBASE-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354556#comment-15354556 ] Gary Helmling commented on HBASE-16115: --- Alright, I'll open up a separate JIRA for that effort. Though triggered here, it seems to be a separate issue. If the consensus is to resolve this by pulling that in to 0.98, then we can always mark this as a dupe. > Missing security context in RegionObserver coprocessor when a > compaction/split is triggered manually > > > Key: HBASE-16115 > URL: https://issues.apache.org/jira/browse/HBASE-16115 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.20 >Reporter: Lars Hofhansl > > We ran into an interesting phenomenon which can easily render a cluster > unusable. > We loaded some tests data into a test table and forced a manual compaction > through the UI. We have some compaction hooks implemented in a region > observer, which writes back to another HBase table when the compaction > finishes. We noticed that this coprocessor is not setup correctly, it seems > the security context is missing. > The interesting part is that this _only_ happens when the compaction is > triggere through the UI. Automatic compactions (major or minor) or when > triggered via the HBase shell (folling a kinit) work fine. Only the > UI-triggered compactions cause this issues and lead to essentially > neverending compactions, immovable regions, etc. > Not sure what exactly the issue is, but I wanted to make sure I capture this. > [~apurtell], [~ghelmling], FYI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16115) Missing security context in RegionObserver coprocessor when a compaction/split is triggered manually
[ https://issues.apache.org/jira/browse/HBASE-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354549#comment-15354549 ] Gary Helmling commented on HBASE-16115: --- Yes, I was looking at HBASE-14475 as well while tracing back the origin of this. The context of that seems specific to security auditing of the compaction request, which was a valid issue, but I don't think implies any expectations of the executing user context for other coprocessors. I agree with the sentiment of it and HBASE-14686, I'm just not sure in hindsight that we reached the best implementation. I think that using an alternate mechanism for conveying that caller context would avoid conflicting with the current user and possibly be more consistent with the RpcCallContext. Maybe it would be better to pass through the requester as part of the ObserverContext. That is present for each coprocessor hook and already prepared for each upcall. If anything, this seems like it belongs there. We could even shim in a call to RpcCallContext where appropriate so this would consistently provide the requester identity for all upcalls. bq. My concern here is we don't ask Phoenix or other implementations to change back and forth for this twice in 0.98. I generally agree with this as well, but it's your call. > Missing security context in RegionObserver coprocessor when a > compaction/split is triggered manually > > > Key: HBASE-16115 > URL: https://issues.apache.org/jira/browse/HBASE-16115 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.20 >Reporter: Lars Hofhansl > > We ran into an interesting phenomenon which can easily render a cluster > unusable. > We loaded some tests data into a test table and forced a manual compaction > through the UI. We have some compaction hooks implemented in a region > observer, which writes back to another HBase table when the compaction > finishes. We noticed that this coprocessor is not setup correctly, it seems > the security context is missing. > The interesting part is that this _only_ happens when the compaction is > triggere through the UI. Automatic compactions (major or minor) or when > triggered via the HBase shell (folling a kinit) work fine. Only the > UI-triggered compactions cause this issues and lead to essentially > neverending compactions, immovable regions, etc. > Not sure what exactly the issue is, but I wanted to make sure I capture this. > [~apurtell], [~ghelmling], FYI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16115) Missing security context in RegionObserver coprocessor when a compaction/split is triggered manually
[ https://issues.apache.org/jira/browse/HBASE-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354548#comment-15354548 ] Andrew Purtell commented on HBASE-16115: I think we should take up [~ghelmling] 's suggestion above for 1.4 and 2.0. I'm not sure about 0.98. If the consensus is to patch there too I won't veto of course. > Missing security context in RegionObserver coprocessor when a > compaction/split is triggered manually > > > Key: HBASE-16115 > URL: https://issues.apache.org/jira/browse/HBASE-16115 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.20 >Reporter: Lars Hofhansl > > We ran into an interesting phenomenon which can easily render a cluster > unusable. > We loaded some tests data into a test table and forced a manual compaction > through the UI. We have some compaction hooks implemented in a region > observer, which writes back to another HBase table when the compaction > finishes. We noticed that this coprocessor is not setup correctly, it seems > the security context is missing. > The interesting part is that this _only_ happens when the compaction is > triggere through the UI. Automatic compactions (major or minor) or when > triggered via the HBase shell (folling a kinit) work fine. Only the > UI-triggered compactions cause this issues and lead to essentially > neverending compactions, immovable regions, etc. > Not sure what exactly the issue is, but I wanted to make sure I capture this. > [~apurtell], [~ghelmling], FYI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16139) Use CellUtil instead of KeyValueUtil in Import.java
[
https://issues.apache.org/jira/browse/HBASE-16139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354544#comment-15354544
]
Anoop Sam John commented on HBASE-16139:
{code}
+ import org.apache.hadoop.hbase.HConstants;
+ import org.apache.hadoop.hbase.KeyValue;
{code}
Seem unused imports. Pls remove.
Otherwise +1
> Use CellUtil instead of KeyValueUtil in Import.java
> ---
>
> Key: HBASE-16139
> URL: https://issues.apache.org/jira/browse/HBASE-16139
> Project: HBase
> Issue Type: Improvement
>Reporter: NIDHI GAMBHIR
>Assignee: NIDHI GAMBHIR
>Priority: Minor
> Attachments: HBASE-16139.patch
>
>
> Currently in Import.java, map function uses KeyValueUtil.createFirstOnRow.
> This method calls createByteArray which causes a lot of copying. Instead use
> CellUtil's createFirstOnRow method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16115) Missing security context in RegionObserver coprocessor when a compaction/split is triggered manually
[ https://issues.apache.org/jira/browse/HBASE-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354543#comment-15354543 ] Andrew Purtell commented on HBASE-16115: What we are discussing as a problem on this JIRA was a bug fix refining an earlier behavioral change to match legacy expectations on another bug report. Calling a change here a fix isn't the whole story. From another point of view it would be a revert of a fix for a regression. > Missing security context in RegionObserver coprocessor when a > compaction/split is triggered manually > > > Key: HBASE-16115 > URL: https://issues.apache.org/jira/browse/HBASE-16115 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.20 >Reporter: Lars Hofhansl > > We ran into an interesting phenomenon which can easily render a cluster > unusable. > We loaded some tests data into a test table and forced a manual compaction > through the UI. We have some compaction hooks implemented in a region > observer, which writes back to another HBase table when the compaction > finishes. We noticed that this coprocessor is not setup correctly, it seems > the security context is missing. > The interesting part is that this _only_ happens when the compaction is > triggere through the UI. Automatic compactions (major or minor) or when > triggered via the HBase shell (folling a kinit) work fine. Only the > UI-triggered compactions cause this issues and lead to essentially > neverending compactions, immovable regions, etc. > Not sure what exactly the issue is, but I wanted to make sure I capture this. > [~apurtell], [~ghelmling], FYI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16134) Introduce InternalCell
[ https://issues.apache.org/jira/browse/HBASE-16134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354541#comment-15354541 ] stack commented on HBASE-16134: --- ExtendedCell works for me. Thanks for explanations. > Introduce InternalCell > -- > > Key: HBASE-16134 > URL: https://issues.apache.org/jira/browse/HBASE-16134 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0 > > Attachments: HBASE-16134.patch, HBASE-16134.patch > > > Came after the discussion under HBASE-15721 and HBASE-15879. > InternalCell is a Cell extension. We do have Cell extensions across different > interfaces now. > SettableSeqId > SettableTimestamp > Streamable. > And demand for this keep growing. > So let us include everything into one Interface. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14422) Fix TestFastFailWithoutTestUtil
[ https://issues.apache.org/jira/browse/HBASE-14422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Ryakhovskiy updated HBASE-14422: --- Status: Patch Available (was: Open) > Fix TestFastFailWithoutTestUtil > --- > > Key: HBASE-14422 > URL: https://issues.apache.org/jira/browse/HBASE-14422 > Project: HBase > Issue Type: Task > Components: test >Reporter: stack >Assignee: Konstantin Ryakhovskiy >Priority: Minor > Labels: beginner > Attachments: HBASE-14422.master.001.patch, > HBASE-14422.master.002.patch, log.txt, trace.log > > > TestFastFailWithoutTestUtil has a unit test that does > testInterceptorIntercept50Times Usually it passes but on occasion, the > latching between thread 1 and thread 2 goes awry and the test hangs and the > test hangs out. Depends on the hardware but it seems to happen about one in > four runs here on an internal rig. > HBASE-14421 changed the wait-on-latch to timeout and do a thread dump and > just let the test keep going. > This issue is about digging in on figuring why the hang up on latches and > then fixing it so the test doesn't have to have the latch timeout. Hopefully > the threaddump helps. > This one could be hard to fix since it not easy to reproduce. Marking it > beginner anyways. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16115) Missing security context in RegionObserver coprocessor when a compaction/split is triggered manually
[ https://issues.apache.org/jira/browse/HBASE-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354533#comment-15354533 ] Lars Hofhansl commented on HBASE-16115: --- Created PHOENIX-3037. Maybe we can suggest an implementation there. > Missing security context in RegionObserver coprocessor when a > compaction/split is triggered manually > > > Key: HBASE-16115 > URL: https://issues.apache.org/jira/browse/HBASE-16115 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.20 >Reporter: Lars Hofhansl > > We ran into an interesting phenomenon which can easily render a cluster > unusable. > We loaded some tests data into a test table and forced a manual compaction > through the UI. We have some compaction hooks implemented in a region > observer, which writes back to another HBase table when the compaction > finishes. We noticed that this coprocessor is not setup correctly, it seems > the security context is missing. > The interesting part is that this _only_ happens when the compaction is > triggere through the UI. Automatic compactions (major or minor) or when > triggered via the HBase shell (folling a kinit) work fine. Only the > UI-triggered compactions cause this issues and lead to essentially > neverending compactions, immovable regions, etc. > Not sure what exactly the issue is, but I wanted to make sure I capture this. > [~apurtell], [~ghelmling], FYI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16115) Missing security context in RegionObserver coprocessor when a compaction/split is triggered manually
[ https://issues.apache.org/jira/browse/HBASE-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354527#comment-15354527 ] Lars Hofhansl commented on HBASE-16115: --- Fair enough. Phoenix fixed one instance of this by taking the stats-update process asynchronous (so it's a thread spawned at startup, which is running as the system user). There's a remaining problem with splits and the local index updater, for that we should do the saving-the-current-user-and-do-doAs fix as you said. [~samarthjain], [~giacomotaylor]. On the other hand this is broken since 0.98.16 (Nov '15) and nobody noticed(!) Might be OK to just "fix" it now (if we agree that that is the fix). > Missing security context in RegionObserver coprocessor when a > compaction/split is triggered manually > > > Key: HBASE-16115 > URL: https://issues.apache.org/jira/browse/HBASE-16115 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.20 >Reporter: Lars Hofhansl > > We ran into an interesting phenomenon which can easily render a cluster > unusable. > We loaded some tests data into a test table and forced a manual compaction > through the UI. We have some compaction hooks implemented in a region > observer, which writes back to another HBase table when the compaction > finishes. We noticed that this coprocessor is not setup correctly, it seems > the security context is missing. > The interesting part is that this _only_ happens when the compaction is > triggere through the UI. Automatic compactions (major or minor) or when > triggered via the HBase shell (folling a kinit) work fine. Only the > UI-triggered compactions cause this issues and lead to essentially > neverending compactions, immovable regions, etc. > Not sure what exactly the issue is, but I wanted to make sure I capture this. > [~apurtell], [~ghelmling], FYI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16115) Missing security context in RegionObserver coprocessor when a compaction/split is triggered manually
[ https://issues.apache.org/jira/browse/HBASE-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354522#comment-15354522 ] Andrew Purtell commented on HBASE-16115: Also, the doAs, while certainly reasonable to discuss changing now, were added to solve a problem. It started with HBASE-14475. See also HBASE-14686. I don't really remember the context of these decisions now but believe it was done to preserve expectations about the request environment (effective user) after refactor of code since Coprocessors and security were introduced in 0.92. > Missing security context in RegionObserver coprocessor when a > compaction/split is triggered manually > > > Key: HBASE-16115 > URL: https://issues.apache.org/jira/browse/HBASE-16115 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.20 >Reporter: Lars Hofhansl > > We ran into an interesting phenomenon which can easily render a cluster > unusable. > We loaded some tests data into a test table and forced a manual compaction > through the UI. We have some compaction hooks implemented in a region > observer, which writes back to another HBase table when the compaction > finishes. We noticed that this coprocessor is not setup correctly, it seems > the security context is missing. > The interesting part is that this _only_ happens when the compaction is > triggere through the UI. Automatic compactions (major or minor) or when > triggered via the HBase shell (folling a kinit) work fine. Only the > UI-triggered compactions cause this issues and lead to essentially > neverending compactions, immovable regions, etc. > Not sure what exactly the issue is, but I wanted to make sure I capture this. > [~apurtell], [~ghelmling], FYI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16115) Missing security context in RegionObserver coprocessor when a compaction/split is triggered manually
[ https://issues.apache.org/jira/browse/HBASE-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354519#comment-15354519 ] Andrew Purtell commented on HBASE-16115: My concern here is we don't ask Phoenix or other implementations to change back and forth for this twice in 0.98. > Missing security context in RegionObserver coprocessor when a > compaction/split is triggered manually > > > Key: HBASE-16115 > URL: https://issues.apache.org/jira/browse/HBASE-16115 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.20 >Reporter: Lars Hofhansl > > We ran into an interesting phenomenon which can easily render a cluster > unusable. > We loaded some tests data into a test table and forced a manual compaction > through the UI. We have some compaction hooks implemented in a region > observer, which writes back to another HBase table when the compaction > finishes. We noticed that this coprocessor is not setup correctly, it seems > the security context is missing. > The interesting part is that this _only_ happens when the compaction is > triggere through the UI. Automatic compactions (major or minor) or when > triggered via the HBase shell (folling a kinit) work fine. Only the > UI-triggered compactions cause this issues and lead to essentially > neverending compactions, immovable regions, etc. > Not sure what exactly the issue is, but I wanted to make sure I capture this. > [~apurtell], [~ghelmling], FYI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16115) Missing security context in RegionObserver coprocessor when a compaction/split is triggered manually
[
https://issues.apache.org/jira/browse/HBASE-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354516#comment-15354516
]
Lars Hofhansl commented on HBASE-16115:
---
bq. Since coprocessors are a system extension mechanism, I don't see why they
should be executing as anything other than the system user.
That's my thinking too.
When a coprocess hook is executed as result of a compaction that is a system
extension. As such why would such a hook care whether the compaction was
triggered by the region server itself or via the master on behalf of a user?
Putting this on the coprocessor implementer seems onerous.
Now there are other hooks ({pre|post}{Get|Scan} etc) that are clearly on behalf
of a user... Or are they?
We introduced the extra doAs in patch releases (1.0.3, 1.1.3, and 0.98.16). 1.0
and 1.1 are or will be retired, right? So maybe we fix it only on 0.98, 1.4,
and 2.0.
> Missing security context in RegionObserver coprocessor when a
> compaction/split is triggered manually
>
>
> Key: HBASE-16115
> URL: https://issues.apache.org/jira/browse/HBASE-16115
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.20
>Reporter: Lars Hofhansl
>
> We ran into an interesting phenomenon which can easily render a cluster
> unusable.
> We loaded some tests data into a test table and forced a manual compaction
> through the UI. We have some compaction hooks implemented in a region
> observer, which writes back to another HBase table when the compaction
> finishes. We noticed that this coprocessor is not setup correctly, it seems
> the security context is missing.
> The interesting part is that this _only_ happens when the compaction is
> triggere through the UI. Automatic compactions (major or minor) or when
> triggered via the HBase shell (folling a kinit) work fine. Only the
> UI-triggered compactions cause this issues and lead to essentially
> neverending compactions, immovable regions, etc.
> Not sure what exactly the issue is, but I wanted to make sure I capture this.
> [~apurtell], [~ghelmling], FYI.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15800) check_compatibility.sh broken as of upstream b8f125f
[
https://issues.apache.org/jira/browse/HBASE-15800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354506#comment-15354506
]
Dima Spivak commented on HBASE-15800:
-
Are you confident that {{b8f125f}} is working? Ran a {{git bisect}} between
{{master}} and {{1.4.3}} (the tag we definitely successfully ran with in the
past) and at least on my machine, it was [this
commit|https://github.com/lvc/japi-compliance-checker/commit/d83910abb7f0aee8e6b588c5d63c87a223874f2c]
that made the difference between the tool being able to read methods (when
using annotations) and not. It's a handful of commits older than {{b8f125f}}
and seems to be changing things more central to how the tool functions than
{{6ccf0d7}}.
> check_compatibility.sh broken as of upstream b8f125f
>
>
> Key: HBASE-15800
> URL: https://issues.apache.org/jira/browse/HBASE-15800
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 2.0.0
>Reporter: Nick Dimiduk
>Priority: Minor
>
> Previously (recently as 2 weeks ago) it was possible to use the
> {{dev-support/check_compatibility.sh}} script against the tags under {{rel}}.
> Now ({{5e0}}) this is no longer the case. On a mac,
> {noformat}
> $ echo $JAVA_HOME
> /Library/Java/JavaVirtualMachines/jdk1.7.0_80.jdk/Contents/Home
> $ GETOPT=/usr/local/Cellar/gnu-getopt/1.1.6/bin/getopt
> ./dev-support/check_compatibility.sh -r
> https://git-wip-us.apache.org/repos/asf/hbase.git rel/1.1.4 branch-1.1
> ...
> Running the Java API Compliance Checker...
> ERROR: can't access
> './dev-support/target/compatibility/1/hbase-annotations/target/hbase-annotations-1.1.4.jar,./dev-support/target/compatibility/1/hbase-checkstyle/target/hbase-checkstyle-1.1.
> 4.jar,./dev-support/target/compatibility/1/hbase-client/target/hbase-client-1.1.4.jar,./dev-support/target/compatibility/1/hbase-common/target/hbase-common-1.1.4.jar,./dev-support/target/compat
> ibility/1/hbase-examples/target/hbase-examples-1.1.4.jar,./dev-support/target/compatibility/1/hbase-hadoop-compat/target/hbase-hadoop-compat-1.1.4.jar,./dev-support/target/compatibility/1/hbase
> -hadoop2-compat/target/hbase-hadoop2-compat-1.1.4.jar,./dev-support/target/compatibility/1/hbase-it/target/hbase-it-1.1.4.jar,./dev-support/target/compatibility/1/hbase-prefix-tree/target/hbase
> -prefix-tree-1.1.4.jar,./dev-support/target/compatibility/1/hbase-procedure/target/hbase-procedure-1.1.4.jar,./dev-support/target/compatibility/1/hbase-protocol/target/hbase-protocol-1.1.4.jar,
> ./dev-support/target/compatibility/1/hbase-resource-bundle/target/hbase-resource-bundle-1.1.4.jar,./dev-support/target/compatibility/1/hbase-rest/target/hbase-rest-1.1.4.jar,./dev-support/targe
> t/compatibility/1/hbase-server/target/hbase-server-1.1.4.jar,./dev-support/target/compatibility/1/hbase-shell/target/hbase-shell-1.1.4.jar,./dev-support/target/compatibility/1/hbase-testing-uti
> l/target/hbase-testing-util-1.1.4.jar,./dev-support/target/compatibility/1/hbase-thrift/target/hbase-thrift-1.1.4.jar'
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15976) RegionServerMetricsWrapperRunnable will be failure when disable blockcache.
[
https://issues.apache.org/jira/browse/HBASE-15976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354505#comment-15354505
]
Nick Dimiduk commented on HBASE-15976:
--
Yep, sounds good. You're sure this isn't going to leave a NPE dangling
someplace? I see a lot of unchecked references to {{cacheStats}}.
> RegionServerMetricsWrapperRunnable will be failure when disable blockcache.
>
>
> Key: HBASE-15976
> URL: https://issues.apache.org/jira/browse/HBASE-15976
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.15, 1.2.1, 1.0.3, 1.1.5
>Reporter: Liu Junhong
>Assignee: Jingcheng Du
> Attachments: HBASE-15976-0.98.patch,
> HBASE-15976-branch-1&branch-1.3.patch, HBASE-15976-branch-1.0.patch,
> HBASE-15976-branch-1.1.patch, HBASE-15976-branch-1.2.patch,
> HBASE-15976-master.patch
>
>
> When i disable blockcache, the code "cacheStats = blockCache.getStats();"
> will occur NPE in
> org.apache.hadoop.hbase.regionserver.MetricsRegionServerWrapperImpl.RegionServerMetricsWrapperRunnable.
> It lead to many regionserver's metrics' value always equal 0.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16134) Introduce InternalCell
[ https://issues.apache.org/jira/browse/HBASE-16134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354503#comment-15354503 ] Anoop Sam John commented on HBASE-16134: My bad... Missed InternalCell extend Cell.. Will do. ServersideCell - No Because it can be in hbase-common only !!! Name confuses.. ExtendedCell sounds fine to me. I can change if Stack also ok with it. bq.What about HeapSize, Cloneable? A SuperCell should do these too I'd say? Can put those also into new Interface. bq.Agreed that we should remove SettableSeqId and timestamp if we can. I am not removing them because it is there in 1.x versions. And it is not Private. I dont think any user creating their own Cell impls. (Phoenix I saw in the past .. Some IndexKV or some thing). So deprecate in 2.0. [~saint@gmail.com] SettableTimestamp came in along with KV -> Cell in read path. In write path we will have to reset the TS on cells (when Cell TS is maxTS and we reset it with RS time). So we want all the Cells flowing through write path to have setTimestamp() implemented. SettableSeqId is used in both R and W paths. > Introduce InternalCell > -- > > Key: HBASE-16134 > URL: https://issues.apache.org/jira/browse/HBASE-16134 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0 > > Attachments: HBASE-16134.patch, HBASE-16134.patch > > > Came after the discussion under HBASE-15721 and HBASE-15879. > InternalCell is a Cell extension. We do have Cell extensions across different > interfaces now. > SettableSeqId > SettableTimestamp > Streamable. > And demand for this keep growing. > So let us include everything into one Interface. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16044) Fix 'hbase shell' output parsing in graceful_stop.sh
[
https://issues.apache.org/jira/browse/HBASE-16044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354495#comment-15354495
]
Hadoop QA commented on HBASE-16044:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} rubocop {color} | {color:blue} 0m 0s
{color} | {color:blue} rubocop was not available. {color} |
| {color:blue}0{color} | {color:blue} ruby-lint {color} | {color:blue} 0m 0s
{color} | {color:blue} Ruby-lint was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
51s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
17s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 40s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 30s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 8s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
3s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
3s {color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
2s {color} | {color:green} There were no new shelldocs issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 40s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 1s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 45s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m 57s
{color} | {color:green} hbase-shell in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 95m 39s {color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
33s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 150m 55s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.regionserver.TestRegionMergeTransactionOnCluster |
| Timed out junit tests | org.apache.hadoop.hbase.TestAcidGuarantees |
| | org.apache.hadoop.hbase.io.hfile.TestCacheOnWrite |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12814361/HBASE-16044.master.001.patch
|
| JIRA Issue | HBASE-16044 |
| Optional Tests | asflicense shellcheck shelldocs javac javadoc unit
rubocop ruby_lint |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 394e722 |
| Default Java | 1.7.0_80 |
| Multi-JDK versions | /home/jenkins/tools/java/jdk1.8.0:1.8.0
/home/jenkins/jenkins-slave/tools/hudson.model.JDK/JDK_1.7_latest_:1.7.0_80 |
| shellcheck | v0.3.3 (This is an old version that has serious bugs. Consider
upgrading.) |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/2411/artifact/patchprocess/patch-unit-root.txt
|
| unit test logs |
https://builds.apache.org/job/PreC
[jira] [Commented] (HBASE-14933) check_compatibility.sh does not work with jdk8
[ https://issues.apache.org/jira/browse/HBASE-14933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354489#comment-15354489 ] Nick Dimiduk commented on HBASE-14933: -- IIRC, last time I ran it, I used JDK7 (branch-1.1 is still built with java7) and some commit before HEAD. See also HBASE-15800. > check_compatibility.sh does not work with jdk8 > -- > > Key: HBASE-14933 > URL: https://issues.apache.org/jira/browse/HBASE-14933 > Project: HBase > Issue Type: Task > Components: scripts >Reporter: Nick Dimiduk >Priority: Minor > Fix For: 2.0.0 > > > Specifically, Oracle jdk1.8.0_65 on OSX and OpenJDk 1.8.0_45-internal-b14 on > ubuntu. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16122) PerformanceEvaluation should provide user friendly hint when client threads argument is missing
[
https://issues.apache.org/jira/browse/HBASE-16122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-16122:
---
Resolution: Fixed
Fix Version/s: 1.4.0
Status: Resolved (was: Patch Available)
Thanks for the patch, Konstantin
> PerformanceEvaluation should provide user friendly hint when client threads
> argument is missing
> ---
>
> Key: HBASE-16122
> URL: https://issues.apache.org/jira/browse/HBASE-16122
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Konstantin Ryakhovskiy
>Priority: Minor
> Fix For: 2.0.0, 1.4.0
>
> Attachments: HBASE-16122-v2.master.patch
>
>
> I tried to run this command:
> hbase pe --rows=1 --nomapred --valueSize=200 sequentialWrite
> {code}
> java.util.NoSuchElementException
> at java.util.LinkedList.removeFirst(LinkedList.java:270)
> at java.util.LinkedList.remove(LinkedList.java:685)
> at
> org.apache.hadoop.hbase.PerformanceEvaluation.parseOpts(PerformanceEvaluation.java:2077)
> at
> org.apache.hadoop.hbase.PerformanceEvaluation.run(PerformanceEvaluation.java:2122)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at
> org.apache.hadoop.hbase.PerformanceEvaluation.main(PerformanceEvaluation.java:2159)
> {code}
> Number of client threads argument was missing.
> PerformanceEvaluation should print user friendly message informing user of
> the missing argument.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16122) PerformanceEvaluation should provide user friendly hint when client threads argument is missing
[
https://issues.apache.org/jira/browse/HBASE-16122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354442#comment-15354442
]
Hudson commented on HBASE-16122:
FAILURE: Integrated in HBase-Trunk_matrix #1133 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/1133/])
HBASE-16122 PerformanceEvaluation should provide user friendly hint when
(tedyu: rev 394e7224a9727538e1c08e5dece7e502cbc8397d)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/TestPerformanceEvaluation.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java
> PerformanceEvaluation should provide user friendly hint when client threads
> argument is missing
> ---
>
> Key: HBASE-16122
> URL: https://issues.apache.org/jira/browse/HBASE-16122
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Konstantin Ryakhovskiy
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16122-v2.master.patch
>
>
> I tried to run this command:
> hbase pe --rows=1 --nomapred --valueSize=200 sequentialWrite
> {code}
> java.util.NoSuchElementException
> at java.util.LinkedList.removeFirst(LinkedList.java:270)
> at java.util.LinkedList.remove(LinkedList.java:685)
> at
> org.apache.hadoop.hbase.PerformanceEvaluation.parseOpts(PerformanceEvaluation.java:2077)
> at
> org.apache.hadoop.hbase.PerformanceEvaluation.run(PerformanceEvaluation.java:2122)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at
> org.apache.hadoop.hbase.PerformanceEvaluation.main(PerformanceEvaluation.java:2159)
> {code}
> Number of client threads argument was missing.
> PerformanceEvaluation should print user friendly message informing user of
> the missing argument.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16132) Scan does not return all the result when regionserver is busy
[
https://issues.apache.org/jira/browse/HBASE-16132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354432#comment-15354432
]
Hadoop QA commented on HBASE-16132:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
0s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 12s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 50s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
55s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
3s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 25s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 58s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 96m 53s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 145m 56s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestHRegion |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12814275/HBASE-16132_v3.patch |
| JIRA Issue | HBASE-16132 |
| Optional Tests | asflicense javac javadoc unit find
[jira] [Commented] (HBASE-14933) check_compatibility.sh does not work with jdk8
[
https://issues.apache.org/jira/browse/HBASE-14933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354431#comment-15354431
]
Dima Spivak commented on HBASE-14933:
-
Hm, interesting. I'm seeing that everything works swimmingly unless I use the
{{\-\-annotations-list}} flag, then everything goes to hell. Filed [an
issue|https://github.com/lvc/japi-compliance-checker/issues/28], but wanna see
if you can repro on your machine? Trying on a few cloud instances now, too...
> check_compatibility.sh does not work with jdk8
> --
>
> Key: HBASE-14933
> URL: https://issues.apache.org/jira/browse/HBASE-14933
> Project: HBase
> Issue Type: Task
> Components: scripts
>Reporter: Nick Dimiduk
>Priority: Minor
> Fix For: 2.0.0
>
>
> Specifically, Oracle jdk1.8.0_65 on OSX and OpenJDk 1.8.0_45-internal-b14 on
> ubuntu.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14933) check_compatibility.sh does not work with jdk8
[ https://issues.apache.org/jira/browse/HBASE-14933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354426#comment-15354426 ] Andrew Purtell commented on HBASE-14933: Works for me with 0.98 and JDK 6 and 7. I used to get reports indicating incompatibilities. > check_compatibility.sh does not work with jdk8 > -- > > Key: HBASE-14933 > URL: https://issues.apache.org/jira/browse/HBASE-14933 > Project: HBase > Issue Type: Task > Components: scripts >Reporter: Nick Dimiduk >Priority: Minor > Fix For: 2.0.0 > > > Specifically, Oracle jdk1.8.0_65 on OSX and OpenJDk 1.8.0_45-internal-b14 on > ubuntu. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16115) Missing security context in RegionObserver coprocessor when a compaction/split is triggered manually
[ https://issues.apache.org/jira/browse/HBASE-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354423#comment-15354423 ] Andrew Purtell commented on HBASE-16115: The suggestion to undo use of doAs for upcalls and authorization checks in general sounds good for 2.0 [~ghelmling] , maybe 1.4. We don't have anyone actively working on integrating the AccessController with core. Would be great if someone steps up to do it. > Missing security context in RegionObserver coprocessor when a > compaction/split is triggered manually > > > Key: HBASE-16115 > URL: https://issues.apache.org/jira/browse/HBASE-16115 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.20 >Reporter: Lars Hofhansl > > We ran into an interesting phenomenon which can easily render a cluster > unusable. > We loaded some tests data into a test table and forced a manual compaction > through the UI. We have some compaction hooks implemented in a region > observer, which writes back to another HBase table when the compaction > finishes. We noticed that this coprocessor is not setup correctly, it seems > the security context is missing. > The interesting part is that this _only_ happens when the compaction is > triggere through the UI. Automatic compactions (major or minor) or when > triggered via the HBase shell (folling a kinit) work fine. Only the > UI-triggered compactions cause this issues and lead to essentially > neverending compactions, immovable regions, etc. > Not sure what exactly the issue is, but I wanted to make sure I capture this. > [~apurtell], [~ghelmling], FYI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16115) Missing security context in RegionObserver coprocessor when a compaction/split is triggered manually
[
https://issues.apache.org/jira/browse/HBASE-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354413#comment-15354413
]
Gary Helmling commented on HBASE-16115:
---
{quote}This obviously doesn't impact compactions normally. Core code is
executed in the context of the logged in user. We switch contexts only for the
upcalls. When scheduled compactions take place there is no request user, so the
logged in user context is used, so you won't see this unless and until requests
are made from another host.{quote}
Hmm, why do we wrap the compaction upcalls in a User.doAs() in the first place?
I understand that we want to authorize the compaction request as the user who
initiated it, but is User.doAs() the right mechanism? It seems like we want
something more akin to RpcCallContext to stash the authenticated user making
the request, instead of using User.doAs() just so that User.getCurrent() can be
hacked to return the requesting user's identity. This seems like a very
confusing semantic to handle for anyone writing a coprocessor. For contrast,
we don't use User.doAs() to authorize normal RPC requests for most of the other
pre/post upcalls. Since coprocessors are a system extension mechanism, I don't
see why they should be executing as anything other than the system user.
I would also suggest that the reliance on doAs() for the procedure code
authorization checks may not be what we want there either. Since carrying
through the requesting user is a requirement of the security code, it seems
like the security code should handle it, rather than muddying the behavior for
all coprocessors. Or maybe it is time to finally pull AccessController up out
of the coprocessor APIs and more tightly integrate it, so that we can avoid
this kind of conflict over requirements.
In any case, it seems like an alternate approach to passing the user
credentials would avoid the confusing semantics entirely, while perhaps
bleeding through the abstraction a bit for the security code.
> Missing security context in RegionObserver coprocessor when a
> compaction/split is triggered manually
>
>
> Key: HBASE-16115
> URL: https://issues.apache.org/jira/browse/HBASE-16115
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.20
>Reporter: Lars Hofhansl
>
> We ran into an interesting phenomenon which can easily render a cluster
> unusable.
> We loaded some tests data into a test table and forced a manual compaction
> through the UI. We have some compaction hooks implemented in a region
> observer, which writes back to another HBase table when the compaction
> finishes. We noticed that this coprocessor is not setup correctly, it seems
> the security context is missing.
> The interesting part is that this _only_ happens when the compaction is
> triggere through the UI. Automatic compactions (major or minor) or when
> triggered via the HBase shell (folling a kinit) work fine. Only the
> UI-triggered compactions cause this issues and lead to essentially
> neverending compactions, immovable regions, etc.
> Not sure what exactly the issue is, but I wanted to make sure I capture this.
> [~apurtell], [~ghelmling], FYI.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14933) check_compatibility.sh does not work with jdk8
[ https://issues.apache.org/jira/browse/HBASE-14933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354412#comment-15354412 ] Dima Spivak commented on HBASE-14933: - Hey [~ndimiduk], I'm getting empty reports regardless of my JDK; I think there's a problem upstream with the annotation support in the tool. Or did you find that it worked with JDK7 but failed with JDK8? > check_compatibility.sh does not work with jdk8 > -- > > Key: HBASE-14933 > URL: https://issues.apache.org/jira/browse/HBASE-14933 > Project: HBase > Issue Type: Task > Components: scripts >Reporter: Nick Dimiduk >Priority: Minor > Fix For: 2.0.0 > > > Specifically, Oracle jdk1.8.0_65 on OSX and OpenJDk 1.8.0_45-internal-b14 on > ubuntu. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16115) Missing security context in RegionObserver coprocessor when a compaction/split is triggered manually
[ https://issues.apache.org/jira/browse/HBASE-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354410#comment-15354410 ] Devaraj Das commented on HBASE-16115: - bq. I think the fix is for Phoenix to save off the result of User.getCurrent() at init time and do a doAs() with that UGI whenever attempting RPC. Either that, or do the Phoenix RPC within a User.runAsLoginUser() context... The latter should be simpler. > Missing security context in RegionObserver coprocessor when a > compaction/split is triggered manually > > > Key: HBASE-16115 > URL: https://issues.apache.org/jira/browse/HBASE-16115 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.20 >Reporter: Lars Hofhansl > > We ran into an interesting phenomenon which can easily render a cluster > unusable. > We loaded some tests data into a test table and forced a manual compaction > through the UI. We have some compaction hooks implemented in a region > observer, which writes back to another HBase table when the compaction > finishes. We noticed that this coprocessor is not setup correctly, it seems > the security context is missing. > The interesting part is that this _only_ happens when the compaction is > triggere through the UI. Automatic compactions (major or minor) or when > triggered via the HBase shell (folling a kinit) work fine. Only the > UI-triggered compactions cause this issues and lead to essentially > neverending compactions, immovable regions, etc. > Not sure what exactly the issue is, but I wanted to make sure I capture this. > [~apurtell], [~ghelmling], FYI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16106) HBase Rest API: unexpected behavior of get with timestamp
[
https://issues.apache.org/jira/browse/HBASE-16106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354402#comment-15354402
]
Junegunn Choi commented on HBASE-16106:
---
[~Whitefret] To retrieve multiple versions, you have to set {{v}} field of the
query string to a number larger than 1. e.g. {{?v=3}}, and of course the column
family should be configured to hold more than one versions. The behavior is
consistent with Java client API where you only get one version unless otherwise
specified.
> HBase Rest API: unexpected behavior of get with timestamp
> -
>
> Key: HBASE-16106
> URL: https://issues.apache.org/jira/browse/HBASE-16106
> Project: HBase
> Issue Type: Bug
> Components: API, REST
>Affects Versions: 1.2.1
>Reporter: Blaye Nicolas
>Priority: Minor
> Labels: easyfix, newbie
>
> Issue seen there:
> http://stackoverflow.com/questions/37985426/hbase-get-request-for-row-data-with-timestamp?noredirect=1#comment63464266_37985426
>
> The *adress:port/table/row/column/timestamp* returns the first value
> *strictly inferior* to the timestamp provided.
> This behavior is not the one seen in bash and java as explained in the
> question.
> I haven't found the bug here but I may be wrong, and it may be fixed already.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16135) PeerClusterZnode under rs of removed peer may never be deleted
[ https://issues.apache.org/jira/browse/HBASE-16135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354203#comment-15354203 ] Duo Zhang commented on HBASE-16135: --- OK. Will fix in the next patch. Any questions on the fix? Thanks. > PeerClusterZnode under rs of removed peer may never be deleted > -- > > Key: HBASE-16135 > URL: https://issues.apache.org/jira/browse/HBASE-16135 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Attachments: HBASE-16135.patch > > > One of our cluster run out of space recently, and we found that the .oldlogs > directory had almost the same size as the data directory. > Finally we found the problem is that, we removed a peer abort 3 months ago, > but there are still some replication queue znode under some rs nodes. This > prevents the deletion of .oldlogs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-16115) Missing security context in RegionObserver coprocessor when a compaction/split is triggered manually
[ https://issues.apache.org/jira/browse/HBASE-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354186#comment-15354186 ] Andrew Purtell edited comment on HBASE-16115 at 6/29/16 1:44 AM: - Funny, I was just writing up commentary for our internal issue tracker. No [~lhofhansl], I think [~devaraj] and [~enis] are correct. I added debug logging around the coprocessor upcalls just to be sure. The upcalls are invoked with the credential of the requesting user, which is different from the locally logged in user. That's what we want for security when the AccessController is installed; and we don't want any other coprocessor naively attempting RPC with this security context. The semantics of the upcall seem correct. Phoenix isn't taking this into account. I think the fix is for Phoenix to save off the result of User.getCurrent() at init time and do a doAs() with that UGI whenever attempting RPC. At the same time this is very poorly documented, so that's an action item for HBase too: All coprocessor upcall javadoc should be annotated with the expected security context. This obviously doesn't impact compactions normally. Core code is executed in the context of the logged in user. We switch contexts only for the upcalls. When scheduled compactions take place there is no request user, so the logged in user context is used, so you won't see this unless and until requests are made from another host. was (Author: apurtell): Funny, I was just writing up commentary for our internal issue tracker. No [~lhofhansl], I think [~devaraj] and [~enis] are correct. I added debug logging around the coprocessor upcalls just to be sure. The upcalls are invoked with the credential of the requesting user, which is different from the locally logged in user. That's what we want for security when the AccessController is installed; and we don't want any other coprocessor naively attempting RPC with this security context. The semantics of the upcall seem correct. Phoenix isn't taking this into account. I think the fix is for Phoenix to save off the result of User.getCurrent() at init time and do a doAs() with that UGI whenever attempting RPC. At the same time this is very poorly documented, so that's an action item for HBase too: All coprocessor upcall javadoc should be annotated with the expected security context. This obviously doesn't impact compactions normally. Core code is executed in the context of the logged in user. We switch contexts only for the upcalls. > Missing security context in RegionObserver coprocessor when a > compaction/split is triggered manually > > > Key: HBASE-16115 > URL: https://issues.apache.org/jira/browse/HBASE-16115 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.20 >Reporter: Lars Hofhansl > > We ran into an interesting phenomenon which can easily render a cluster > unusable. > We loaded some tests data into a test table and forced a manual compaction > through the UI. We have some compaction hooks implemented in a region > observer, which writes back to another HBase table when the compaction > finishes. We noticed that this coprocessor is not setup correctly, it seems > the security context is missing. > The interesting part is that this _only_ happens when the compaction is > triggere through the UI. Automatic compactions (major or minor) or when > triggered via the HBase shell (folling a kinit) work fine. Only the > UI-triggered compactions cause this issues and lead to essentially > neverending compactions, immovable regions, etc. > Not sure what exactly the issue is, but I wanted to make sure I capture this. > [~apurtell], [~ghelmling], FYI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-16115) Missing security context in RegionObserver coprocessor when a compaction/split is triggered manually
[ https://issues.apache.org/jira/browse/HBASE-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354186#comment-15354186 ] Andrew Purtell edited comment on HBASE-16115 at 6/29/16 1:44 AM: - Funny, I was just writing up commentary for our internal issue tracker. No [~lhofhansl], I think [~devaraj] and [~enis] are correct. I added debug logging around the coprocessor upcalls just to be sure. The upcalls are invoked with the credential of the requesting user, which is different from the locally logged in user. That's what we want for security when the AccessController is installed; and we don't want any other coprocessor naively attempting RPC with this security context. The semantics of the upcall seem correct. Phoenix isn't taking this into account. I think the fix is for Phoenix to save off the result of User.getCurrent() at init time and do a doAs() with that UGI whenever attempting RPC. At the same time this is very poorly documented, so that's an action item for HBase too: All coprocessor upcall javadoc should be annotated with the expected security context. This obviously doesn't impact compactions normally. Core code is executed in the context of the logged in user. We switch contexts only for the upcalls. was (Author: apurtell): Funny, I was just writing up commentary for our internal issue tracker. No [~lhofhansl], I think [~devaraj] and [~enis] are correct. I added debug logging around the coprocessor upcalls just to be sure. The upcalls are invoked with the credential of the requesting user, which is different from the locally logged in user. That's what we want for security when the AccessController is installed; and we don't want any other coprocessor naively attempting RPC with this security context. The semantics of the upcall seem correct. Phoenix isn't taking this into account. I think the fix is for Phoenix to save off the result of User.getCurrent() at init time and do a doAs() with that UGI whenever attempting RPC. At the same time this is very poorly documented, so that's an action item for HBase too: All coprocessor upcall javadoc should be annotated with the expected security context. > Missing security context in RegionObserver coprocessor when a > compaction/split is triggered manually > > > Key: HBASE-16115 > URL: https://issues.apache.org/jira/browse/HBASE-16115 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.20 >Reporter: Lars Hofhansl > > We ran into an interesting phenomenon which can easily render a cluster > unusable. > We loaded some tests data into a test table and forced a manual compaction > through the UI. We have some compaction hooks implemented in a region > observer, which writes back to another HBase table when the compaction > finishes. We noticed that this coprocessor is not setup correctly, it seems > the security context is missing. > The interesting part is that this _only_ happens when the compaction is > triggere through the UI. Automatic compactions (major or minor) or when > triggered via the HBase shell (folling a kinit) work fine. Only the > UI-triggered compactions cause this issues and lead to essentially > neverending compactions, immovable regions, etc. > Not sure what exactly the issue is, but I wanted to make sure I capture this. > [~apurtell], [~ghelmling], FYI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-16115) Missing security context in RegionObserver coprocessor when a compaction/split is triggered manually
[ https://issues.apache.org/jira/browse/HBASE-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354186#comment-15354186 ] Andrew Purtell edited comment on HBASE-16115 at 6/29/16 1:42 AM: - Funny, I was just writing up commentary for our internal issue tracker. No [~lhofhansl], I think [~devaraj] and [~enis] are correct. I added debug logging around the coprocessor upcalls just to be sure. The upcalls are invoked with the credential of the requesting user, which is different from the locally logged in user. That's what we want for security when the AccessController is installed; and we don't want any other coprocessor naively attempting RPC with this security context. The semantics of the upcall seem correct. Phoenix isn't taking this into account. I think the fix is for Phoenix to save off the result of User.getCurrent() at init time and do a doAs() with that UGI whenever attempting RPC. At the same time this is very poorly documented, so that's an action item for HBase too: All coprocessor upcall javadoc should be annotated with the expected security context. was (Author: apurtell): Funny, I was just writing up commentary for our internal issue tracker. No [~lhofhansl], I think [~devaraj] and [~enis] are correct. I added debug logging around the coprocessor upcalls just to be sure. The upcalls are invoked with the credential of the requesting user, which is different from the locally logged in user. That's what we want for security when the AccessController is installed; and we don't want any other coprocessor naively attempting RPC with this security context. The semantics of the upcall seem correct. Phoenix isn't taking this into account. I think the fix is for Phoenix to save off the result of User.getCurrent() at init time and do a doAs() with that UGI whenever attempting RPC. At the same time this is very poorly documented, so that's an action item for HBase too. > Missing security context in RegionObserver coprocessor when a > compaction/split is triggered manually > > > Key: HBASE-16115 > URL: https://issues.apache.org/jira/browse/HBASE-16115 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.20 >Reporter: Lars Hofhansl > > We ran into an interesting phenomenon which can easily render a cluster > unusable. > We loaded some tests data into a test table and forced a manual compaction > through the UI. We have some compaction hooks implemented in a region > observer, which writes back to another HBase table when the compaction > finishes. We noticed that this coprocessor is not setup correctly, it seems > the security context is missing. > The interesting part is that this _only_ happens when the compaction is > triggere through the UI. Automatic compactions (major or minor) or when > triggered via the HBase shell (folling a kinit) work fine. Only the > UI-triggered compactions cause this issues and lead to essentially > neverending compactions, immovable regions, etc. > Not sure what exactly the issue is, but I wanted to make sure I capture this. > [~apurtell], [~ghelmling], FYI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16115) Missing security context in RegionObserver coprocessor when a compaction/split is triggered manually
[ https://issues.apache.org/jira/browse/HBASE-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354186#comment-15354186 ] Andrew Purtell commented on HBASE-16115: Funny, I was just writing up commentary for our internal issue tracker. No [~lhofhansl], I think [~devaraj] and [~enis] are correct. I added debug logging around the coprocessor upcalls just to be sure. The upcalls are invoked with the credential of the requesting user, which is different from the locally logged in user. That's what we want for security when the AccessController is installed; and we don't want any other coprocessor naively attempting RPC with this security context. The semantics of the upcall seem correct. Phoenix isn't taking this into account. I think the fix is for Phoenix to save off the result of User.getCurrent() at init time and do a doAs() with that UGI whenever attempting RPC. At the same time this is very poorly documented, so that's an action item for HBase too. > Missing security context in RegionObserver coprocessor when a > compaction/split is triggered manually > > > Key: HBASE-16115 > URL: https://issues.apache.org/jira/browse/HBASE-16115 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.20 >Reporter: Lars Hofhansl > > We ran into an interesting phenomenon which can easily render a cluster > unusable. > We loaded some tests data into a test table and forced a manual compaction > through the UI. We have some compaction hooks implemented in a region > observer, which writes back to another HBase table when the compaction > finishes. We noticed that this coprocessor is not setup correctly, it seems > the security context is missing. > The interesting part is that this _only_ happens when the compaction is > triggere through the UI. Automatic compactions (major or minor) or when > triggered via the HBase shell (folling a kinit) work fine. Only the > UI-triggered compactions cause this issues and lead to essentially > neverending compactions, immovable regions, etc. > Not sure what exactly the issue is, but I wanted to make sure I capture this. > [~apurtell], [~ghelmling], FYI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16044) Fix 'hbase shell' output parsing in graceful_stop.sh
[
https://issues.apache.org/jira/browse/HBASE-16044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354089#comment-15354089
]
Appy commented on HBASE-16044:
--
Reverted those changes from branch-1.
Submitted patch to fix this issue in master.
Following output shows difference between interactive/non-interactive for a
simple command like balance_switch.
Interactive
{noformat}
hbase(main):003:0> balance_switch true
Previous balancer state : true
Took 0.0210 seconds
hbase(main):004:0> balance_switch false
Previous balancer state : true
Took 0.0130 seconds
{noformat}
Non-interactive output. Note the last line returning raw (unformatted) output
which in this case is simple true/false string.
{noformat}
~/apache/hbase (HBASE-16044) → echo "balance_switch true" | ./bin/hbase shell
-n
2016-06-28 18:23:12,930 WARN [main] util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
Previous balancer state : false
Took 0.3890 seconds
false
~/apache/hbase (HBASE-16044) → echo "balance_switch true" | ./bin/hbase shell
-n
2016-06-28 18:23:22,528 WARN [main] util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
Previous balancer state : true
Took 0.3830 seconds
true
{noformat}
> Fix 'hbase shell' output parsing in graceful_stop.sh
>
>
> Key: HBASE-16044
> URL: https://issues.apache.org/jira/browse/HBASE-16044
> Project: HBase
> Issue Type: Bug
> Components: scripts
>Affects Versions: 2.0.0
>Reporter: Samir Ahmic
>Assignee: Samir Ahmic
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-16044.master.001.patch
>
>
> In some of our bash scripts we are piping command in hbase shell and then
> parsing response to define variables. Since 'hbase shell' output format is
> changed we are picking wrong values from output Here is example form
> gracful_stop.sh:
> {code}
> HBASE_BALANCER_STATE=$(echo 'balance_switch false' | "$bin"/hbase --config
> "${HBASE_CONF_DIR}" shell | tail -3 | head -1)
> {code}
> this will return "balance_switch true" instead of previous balancer state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16139) Use CellUtil instead of KeyValueUtil in Import.java
[
https://issues.apache.org/jira/browse/HBASE-16139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354086#comment-15354086
]
Ted Yu commented on HBASE-16139:
Please fix formatting:
{code}
1511 public static Cell createFirstOnRow(final byte [] row, int roffset,
short rlength) {
1512 return new FirstOnRowCell(row, roffset, rlength);
1513 }
{code}
Indentation is two spaces.
> Use CellUtil instead of KeyValueUtil in Import.java
> ---
>
> Key: HBASE-16139
> URL: https://issues.apache.org/jira/browse/HBASE-16139
> Project: HBase
> Issue Type: Improvement
>Reporter: NIDHI GAMBHIR
>Assignee: NIDHI GAMBHIR
>Priority: Minor
> Attachments: HBASE-16139.patch
>
>
> Currently in Import.java, map function uses KeyValueUtil.createFirstOnRow.
> This method calls createByteArray which causes a lot of copying. Instead use
> CellUtil's createFirstOnRow method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16044) Fix 'hbase shell' output parsing in graceful_stop.sh
[
https://issues.apache.org/jira/browse/HBASE-16044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-16044:
-
Status: Patch Available (was: Open)
> Fix 'hbase shell' output parsing in graceful_stop.sh
>
>
> Key: HBASE-16044
> URL: https://issues.apache.org/jira/browse/HBASE-16044
> Project: HBase
> Issue Type: Bug
> Components: scripts
>Affects Versions: 2.0.0
>Reporter: Samir Ahmic
>Assignee: Samir Ahmic
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-16044.master.001.patch
>
>
> In some of our bash scripts we are piping command in hbase shell and then
> parsing response to define variables. Since 'hbase shell' output format is
> changed we are picking wrong values from output Here is example form
> gracful_stop.sh:
> {code}
> HBASE_BALANCER_STATE=$(echo 'balance_switch false' | "$bin"/hbase --config
> "${HBASE_CONF_DIR}" shell | tail -3 | head -1)
> {code}
> this will return "balance_switch true" instead of previous balancer state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16044) Fix 'hbase shell' output parsing in graceful_stop.sh
[
https://issues.apache.org/jira/browse/HBASE-16044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-16044:
-
Summary: Fix 'hbase shell' output parsing in graceful_stop.sh (was: Fix
'hbase shell' output parsing in bash scripts)
> Fix 'hbase shell' output parsing in graceful_stop.sh
>
>
> Key: HBASE-16044
> URL: https://issues.apache.org/jira/browse/HBASE-16044
> Project: HBase
> Issue Type: Bug
> Components: scripts
>Affects Versions: 2.0.0
>Reporter: Samir Ahmic
>Assignee: Samir Ahmic
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-16044.master.001.patch
>
>
> In some of our bash scripts we are piping command in hbase shell and then
> parsing response to define variables. Since 'hbase shell' output format is
> changed we are picking wrong values from output Here is example form
> gracful_stop.sh:
> {code}
> HBASE_BALANCER_STATE=$(echo 'balance_switch false' | "$bin"/hbase --config
> "${HBASE_CONF_DIR}" shell | tail -3 | head -1)
> {code}
> this will return "balance_switch true" instead of previous balancer state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16044) Fix 'hbase shell' output parsing in graceful_stop.sh
[
https://issues.apache.org/jira/browse/HBASE-16044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Appy updated HBASE-16044:
-
Attachment: HBASE-16044.master.001.patch
> Fix 'hbase shell' output parsing in graceful_stop.sh
>
>
> Key: HBASE-16044
> URL: https://issues.apache.org/jira/browse/HBASE-16044
> Project: HBase
> Issue Type: Bug
> Components: scripts
>Affects Versions: 2.0.0
>Reporter: Samir Ahmic
>Assignee: Samir Ahmic
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-16044.master.001.patch
>
>
> In some of our bash scripts we are piping command in hbase shell and then
> parsing response to define variables. Since 'hbase shell' output format is
> changed we are picking wrong values from output Here is example form
> gracful_stop.sh:
> {code}
> HBASE_BALANCER_STATE=$(echo 'balance_switch false' | "$bin"/hbase --config
> "${HBASE_CONF_DIR}" shell | tail -3 | head -1)
> {code}
> this will return "balance_switch true" instead of previous balancer state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16115) Missing security context in RegionObserver coprocessor when a compaction/split is triggered manually
[ https://issues.apache.org/jira/browse/HBASE-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354015#comment-15354015 ] Lars Hofhansl commented on HBASE-16115: --- What we've seen is that with a manual compaction the request comes in from the master, so as a user on the master machine. With this patch it then passes that user through to any action performed inside the compaction, which is - I think - not correct. The Phoenix coprocessor is not doing anything special, it uses the HBase tooling to write some data to another table when the compaction is finished. It shouldn't have to do anything specific, it should authenticated as the proper user on the local machine, no? > Missing security context in RegionObserver coprocessor when a > compaction/split is triggered manually > > > Key: HBASE-16115 > URL: https://issues.apache.org/jira/browse/HBASE-16115 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.20 >Reporter: Lars Hofhansl > > We ran into an interesting phenomenon which can easily render a cluster > unusable. > We loaded some tests data into a test table and forced a manual compaction > through the UI. We have some compaction hooks implemented in a region > observer, which writes back to another HBase table when the compaction > finishes. We noticed that this coprocessor is not setup correctly, it seems > the security context is missing. > The interesting part is that this _only_ happens when the compaction is > triggere through the UI. Automatic compactions (major or minor) or when > triggered via the HBase shell (folling a kinit) work fine. Only the > UI-triggered compactions cause this issues and lead to essentially > neverending compactions, immovable regions, etc. > Not sure what exactly the issue is, but I wanted to make sure I capture this. > [~apurtell], [~ghelmling], FYI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15849) Shell Cleanup: Simplify handling of commands' runtime
[ https://issues.apache.org/jira/browse/HBASE-15849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354013#comment-15354013 ] Appy commented on HBASE-15849: -- Reverted from branch-1. Discussion [here|https://issues.apache.org/jira/browse/HBASE-16044?focusedCommentId=15347364&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15347364]. > Shell Cleanup: Simplify handling of commands' runtime > - > > Key: HBASE-15849 > URL: https://issues.apache.org/jira/browse/HBASE-15849 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15849.master.001.patch > > > Functions format_simple_command and format_and_return_simple_command are used > to print runtimes right now. They are called from within every single command > and use Ruby's 'yield' magic. Instead, we can simplify it using > 'command_safe' function. Since command_safe wraps all commands, we can simply > time before and after we call individual command. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15849) Shell Cleanup: Simplify handling of commands' runtime
[ https://issues.apache.org/jira/browse/HBASE-15849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-15849: - Fix Version/s: (was: 1.4.0) > Shell Cleanup: Simplify handling of commands' runtime > - > > Key: HBASE-15849 > URL: https://issues.apache.org/jira/browse/HBASE-15849 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy >Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-15849.master.001.patch > > > Functions format_simple_command and format_and_return_simple_command are used > to print runtimes right now. They are called from within every single command > and use Ruby's 'yield' magic. Instead, we can simplify it using > 'command_safe' function. Since command_safe wraps all commands, we can simply > time before and after we call individual command. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15965) Shell test changes. Use @shell.command instead directly calling functions in admin.rb and other libraries.
[ https://issues.apache.org/jira/browse/HBASE-15965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-15965: - Fix Version/s: (was: 1.4.0) > Shell test changes. Use @shell.command instead directly calling functions in > admin.rb and other libraries. > -- > > Key: HBASE-15965 > URL: https://issues.apache.org/jira/browse/HBASE-15965 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Fix For: 2.0.0 > > Attachments: HBASE-15965.master.001.patch, > HBASE-15965.master.002.patch, HBASE-15965.master.003.patch > > > Testing by executing a command will cover the exact path users will trigger, > so its better then directly calling library functions in tests. Changing the > tests to use @shell.command(:, args) to execute them like it's a > command coming from shell. > Norm change: > Commands should print the output user would like to see, but in the end, > should also return the relevant value. This way: > - Tests can use returned value to check that functionality works > - Tests can capture stdout to assert particular kind of output user should > see. > - We do not print the return value in interactive mode and keep the output > clean. See Shell.command() function. > Bugs found due to this change: > - Uncovered bug in major_compact.rb with this approach. It was calling > admin.majorCompact() which doesn't exist but our tests didn't catch it since > they directly tested admin.major_compact() > - Enabled TestReplicationShell. If it's bad, flaky infra will take care of it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15965) Shell test changes. Use @shell.command instead directly calling functions in admin.rb and other libraries.
[ https://issues.apache.org/jira/browse/HBASE-15965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354012#comment-15354012 ] Appy commented on HBASE-15965: -- Reverted from branch-1. Discussion [here|https://issues.apache.org/jira/browse/HBASE-16044?focusedCommentId=15347364&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15347364]. > Shell test changes. Use @shell.command instead directly calling functions in > admin.rb and other libraries. > -- > > Key: HBASE-15965 > URL: https://issues.apache.org/jira/browse/HBASE-15965 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Fix For: 2.0.0 > > Attachments: HBASE-15965.master.001.patch, > HBASE-15965.master.002.patch, HBASE-15965.master.003.patch > > > Testing by executing a command will cover the exact path users will trigger, > so its better then directly calling library functions in tests. Changing the > tests to use @shell.command(:, args) to execute them like it's a > command coming from shell. > Norm change: > Commands should print the output user would like to see, but in the end, > should also return the relevant value. This way: > - Tests can use returned value to check that functionality works > - Tests can capture stdout to assert particular kind of output user should > see. > - We do not print the return value in interactive mode and keep the output > clean. See Shell.command() function. > Bugs found due to this change: > - Uncovered bug in major_compact.rb with this approach. It was calling > admin.majorCompact() which doesn't exist but our tests didn't catch it since > they directly tested admin.major_compact() > - Enabled TestReplicationShell. If it's bad, flaky infra will take care of it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16133) RSGroupBasedLoadBalancer.retainAssignment() might miss a region
[
https://issues.apache.org/jira/browse/HBASE-16133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354008#comment-15354008
]
Hadoop QA commented on HBASE-16133:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
12s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 20s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 13s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 28s
{color} | {color:red} hbase-rsgroup in master has 6 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 11s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 13s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 13s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 4s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 18s
{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 34m 27s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12814008/hbase-16133_v1.patch |
| JIRA Issue | HBASE-16133 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 8bc4d41 |
| Default Java | 1.7.0_80 |
| Multi-JDK versions | /home/jenkins/tools/java/jdk1.8.0:1.8.0
/home/jenk
[jira] [Commented] (HBASE-16115) Missing security context in RegionObserver coprocessor when a compaction/split is triggered manually
[ https://issues.apache.org/jira/browse/HBASE-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354005#comment-15354005 ] Enis Soztutar commented on HBASE-16115: --- bq. When I talked about the issue we earlier faced, the way we fixed was to simply run everything in the compaction as the login user (which is hbase regionserver user), but we somehow thought that HBASE-14655 would fix it in the long run, but let me check that hypothesis... Good catch. I think HBASE-14655 is the right fix in HBase because the coprocessor should run with the user context (for security). In Phoenix (or any other coprocessor) that wants to do RPC to another region server though, that has to be performed as the login user context. So we need a fix in Phoenix to switch back to the login context? > Missing security context in RegionObserver coprocessor when a > compaction/split is triggered manually > > > Key: HBASE-16115 > URL: https://issues.apache.org/jira/browse/HBASE-16115 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.20 >Reporter: Lars Hofhansl > > We ran into an interesting phenomenon which can easily render a cluster > unusable. > We loaded some tests data into a test table and forced a manual compaction > through the UI. We have some compaction hooks implemented in a region > observer, which writes back to another HBase table when the compaction > finishes. We noticed that this coprocessor is not setup correctly, it seems > the security context is missing. > The interesting part is that this _only_ happens when the compaction is > triggere through the UI. Automatic compactions (major or minor) or when > triggered via the HBase shell (folling a kinit) work fine. Only the > UI-triggered compactions cause this issues and lead to essentially > neverending compactions, immovable regions, etc. > Not sure what exactly the issue is, but I wanted to make sure I capture this. > [~apurtell], [~ghelmling], FYI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16044) Fix 'hbase shell' output parsing in bash scripts
[
https://issues.apache.org/jira/browse/HBASE-16044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354002#comment-15354002
]
Appy commented on HBASE-16044:
--
Discussed with [~stack] offline. I am fine with reverting the earlier changes
(HBASE-15965 and HBASE-15849) from branch-1 even though they are within our
operational compatibility guidelines. I think his concern for usability
triumphs here, lets make minor release (if 1.4 does comes out) easier for users.
These two changes are not critical and don't need to be necessarily backported.
It's better to do them only in major release (2.0). One of them is a step
towards putting a better system in place (interactive/non-interactive compat,
also discussed
[here|https://issues.apache.org/jira/browse/HBASE-16078?focusedCommentId=15353210&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15353210]).
> Fix 'hbase shell' output parsing in bash scripts
>
>
> Key: HBASE-16044
> URL: https://issues.apache.org/jira/browse/HBASE-16044
> Project: HBase
> Issue Type: Bug
> Components: scripts
>Affects Versions: 2.0.0
>Reporter: Samir Ahmic
>Assignee: Samir Ahmic
>Priority: Critical
> Fix For: 2.0.0
>
>
> In some of our bash scripts we are piping command in hbase shell and then
> parsing response to define variables. Since 'hbase shell' output format is
> changed we are picking wrong values from output Here is example form
> gracful_stop.sh:
> {code}
> HBASE_BALANCER_STATE=$(echo 'balance_switch false' | "$bin"/hbase --config
> "${HBASE_CONF_DIR}" shell | tail -3 | head -1)
> {code}
> this will return "balance_switch true" instead of previous balancer state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16122) PerformanceEvaluation should provide user friendly hint when client threads argument is missing
[
https://issues.apache.org/jira/browse/HBASE-16122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-16122:
---
Hadoop Flags: Reviewed
Fix Version/s: 2.0.0
> PerformanceEvaluation should provide user friendly hint when client threads
> argument is missing
> ---
>
> Key: HBASE-16122
> URL: https://issues.apache.org/jira/browse/HBASE-16122
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Konstantin Ryakhovskiy
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16122-v2.master.patch
>
>
> I tried to run this command:
> hbase pe --rows=1 --nomapred --valueSize=200 sequentialWrite
> {code}
> java.util.NoSuchElementException
> at java.util.LinkedList.removeFirst(LinkedList.java:270)
> at java.util.LinkedList.remove(LinkedList.java:685)
> at
> org.apache.hadoop.hbase.PerformanceEvaluation.parseOpts(PerformanceEvaluation.java:2077)
> at
> org.apache.hadoop.hbase.PerformanceEvaluation.run(PerformanceEvaluation.java:2122)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at
> org.apache.hadoop.hbase.PerformanceEvaluation.main(PerformanceEvaluation.java:2159)
> {code}
> Number of client threads argument was missing.
> PerformanceEvaluation should print user friendly message informing user of
> the missing argument.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16122) PerformanceEvaluation should provide user friendly hint when client threads argument is missing
[
https://issues.apache.org/jira/browse/HBASE-16122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353999#comment-15353999
]
Ted Yu commented on HBASE-16122:
I ran TestPerformanceEvaluation locally.
In
hbase-server/target/surefire-reports/org.apache.hadoop.hbase.TestPerformanceEvaluation-output.txt
, I can see the following line:
{code}
Command sequentialWrite does not have threads number
{code}
The test failures were not related to the patch.
> PerformanceEvaluation should provide user friendly hint when client threads
> argument is missing
> ---
>
> Key: HBASE-16122
> URL: https://issues.apache.org/jira/browse/HBASE-16122
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Konstantin Ryakhovskiy
>Priority: Minor
> Attachments: HBASE-16122-v2.master.patch
>
>
> I tried to run this command:
> hbase pe --rows=1 --nomapred --valueSize=200 sequentialWrite
> {code}
> java.util.NoSuchElementException
> at java.util.LinkedList.removeFirst(LinkedList.java:270)
> at java.util.LinkedList.remove(LinkedList.java:685)
> at
> org.apache.hadoop.hbase.PerformanceEvaluation.parseOpts(PerformanceEvaluation.java:2077)
> at
> org.apache.hadoop.hbase.PerformanceEvaluation.run(PerformanceEvaluation.java:2122)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at
> org.apache.hadoop.hbase.PerformanceEvaluation.main(PerformanceEvaluation.java:2159)
> {code}
> Number of client threads argument was missing.
> PerformanceEvaluation should print user friendly message informing user of
> the missing argument.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16115) Missing security context in RegionObserver coprocessor when a compaction/split is triggered manually
[ https://issues.apache.org/jira/browse/HBASE-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353994#comment-15353994 ] Devaraj Das commented on HBASE-16115: - Yeah, HBASE-14655 might be the cause of the issue at hand. Thinking about it, one regionserver wouldn't be able to communicate with another without valid credentials. HBASE-14655 makes it so that the preCompact hook would run as the end user submitting the compaction request. That wouldn't work for authentication purposes. When I talked about the issue we earlier faced, the way we fixed was to simply run everything in the compaction as the login user (which is hbase regionserver user), but we somehow thought that HBASE-14655 would fix it in the long run, but let me check that hypothesis... > Missing security context in RegionObserver coprocessor when a > compaction/split is triggered manually > > > Key: HBASE-16115 > URL: https://issues.apache.org/jira/browse/HBASE-16115 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.20 >Reporter: Lars Hofhansl > > We ran into an interesting phenomenon which can easily render a cluster > unusable. > We loaded some tests data into a test table and forced a manual compaction > through the UI. We have some compaction hooks implemented in a region > observer, which writes back to another HBase table when the compaction > finishes. We noticed that this coprocessor is not setup correctly, it seems > the security context is missing. > The interesting part is that this _only_ happens when the compaction is > triggere through the UI. Automatic compactions (major or minor) or when > triggered via the HBase shell (folling a kinit) work fine. Only the > UI-triggered compactions cause this issues and lead to essentially > neverending compactions, immovable regions, etc. > Not sure what exactly the issue is, but I wanted to make sure I capture this. > [~apurtell], [~ghelmling], FYI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16136) Check components are alive when closing RegionServer
[ https://issues.apache.org/jira/browse/HBASE-16136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353982#comment-15353982 ] Enis Soztutar commented on HBASE-16136: --- What is the goal of this? > Check components are alive when closing RegionServer > > > Key: HBASE-16136 > URL: https://issues.apache.org/jira/browse/HBASE-16136 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 0.98.18, 0.98.19, 0.98.20 >Reporter: darion yaphet > Labels: regionserver > Attachments: HBASE-16136.patch > > > Check components are alive when closing RegionServer . -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16132) Scan does not return all the result when regionserver is busy
[
https://issues.apache.org/jira/browse/HBASE-16132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
binlijin updated HBASE-16132:
-
Attachment: HBASE-16132_v3.patch
> Scan does not return all the result when regionserver is busy
> -
>
> Key: HBASE-16132
> URL: https://issues.apache.org/jira/browse/HBASE-16132
> Project: HBase
> Issue Type: Bug
>Reporter: binlijin
> Attachments: HBASE-16132.patch, HBASE-16132_v2.patch,
> HBASE-16132_v3.patch
>
>
> We have find some corner case, when regionserver is busy and last a long
> time. Some scanner may return null even if they do not scan all data.
> We find in ScannerCallableWithReplicas there is a case do not handler
> correct, when cs.poll timeout and do not return any result , it is will
> return a null result, so scan get null result, and end the scan.
> {code}
> try {
> Future> f = cs.poll(timeout,
> TimeUnit.MILLISECONDS);
> if (f != null) {
> Pair r = f.get(timeout,
> TimeUnit.MILLISECONDS);
> if (r != null && r.getSecond() != null) {
> updateCurrentlyServingReplica(r.getSecond(), r.getFirst(), done,
> pool);
> }
> return r == null ? null : r.getFirst(); // great we got an answer
> }
> } catch (ExecutionException e) {
> RpcRetryingCallerWithReadReplicas.throwEnrichedException(e, retries);
> } catch (CancellationException e) {
> throw new InterruptedIOException(e.getMessage());
> } catch (InterruptedException e) {
> throw new InterruptedIOException(e.getMessage());
> } catch (TimeoutException e) {
> throw new InterruptedIOException(e.getMessage());
> } finally {
> // We get there because we were interrupted or because one or more of
> the
> // calls succeeded or failed. In all case, we stop all our tasks.
> cs.cancelAll();
> }
> return null; // unreachable
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16134) Introduce InternalCell
[ https://issues.apache.org/jira/browse/HBASE-16134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353978#comment-15353978 ] Enis Soztutar commented on HBASE-16134: --- bq. On the patch, seems odd having a Cell named InternalCell in hbase-common. Should it be in hbase-server? Hmm.. I see you can't do that because its used by KV, etc. which is ok. So, it should have a name like SuperCell because it is Cell with extra info? ServersideCell? ExtendedCell? Agreed that we should remove SettableSeqId and timestamp if we can. > Introduce InternalCell > -- > > Key: HBASE-16134 > URL: https://issues.apache.org/jira/browse/HBASE-16134 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0 > > Attachments: HBASE-16134.patch, HBASE-16134.patch > > > Came after the discussion under HBASE-15721 and HBASE-15879. > InternalCell is a Cell extension. We do have Cell extensions across different > interfaces now. > SettableSeqId > SettableTimestamp > Streamable. > And demand for this keep growing. > So let us include everything into one Interface. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16133) RSGroupBasedLoadBalancer.retainAssignment() might miss a region
[
https://issues.apache.org/jira/browse/HBASE-16133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-16133:
--
Status: Patch Available (was: Open)
> RSGroupBasedLoadBalancer.retainAssignment() might miss a region
> ---
>
> Key: HBASE-16133
> URL: https://issues.apache.org/jira/browse/HBASE-16133
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 2.0.0
>
> Attachments: hbase-16133_v1.patch
>
>
> We have seen in the tests through the IntegrationTestRSGroup that we may miss
> assigning a region.
> It is a simple logic error here:
> {code}
> if (server != null && !assignments.containsKey(server)) {
> assignments.put(server, new ArrayList());
> } else if (server != null) {
>assignments.get(server).add(region);
> } else {
> {code}
> in the first condition, we are not adding the region to the newly created
> ArrayList.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16115) Missing security context in RegionObserver coprocessor when a compaction/split is triggered manually
[ https://issues.apache.org/jira/browse/HBASE-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353948#comment-15353948 ] Lars Hofhansl commented on HBASE-16115: --- bq. at org.apache.hadoop.hbase.regionserver.compactions.Compactor$3.run(Compactor.java:360) At least we know the passed user is not null (we'd see line 357 instead) > Missing security context in RegionObserver coprocessor when a > compaction/split is triggered manually > > > Key: HBASE-16115 > URL: https://issues.apache.org/jira/browse/HBASE-16115 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.20 >Reporter: Lars Hofhansl > > We ran into an interesting phenomenon which can easily render a cluster > unusable. > We loaded some tests data into a test table and forced a manual compaction > through the UI. We have some compaction hooks implemented in a region > observer, which writes back to another HBase table when the compaction > finishes. We noticed that this coprocessor is not setup correctly, it seems > the security context is missing. > The interesting part is that this _only_ happens when the compaction is > triggere through the UI. Automatic compactions (major or minor) or when > triggered via the HBase shell (folling a kinit) work fine. Only the > UI-triggered compactions cause this issues and lead to essentially > neverending compactions, immovable regions, etc. > Not sure what exactly the issue is, but I wanted to make sure I capture this. > [~apurtell], [~ghelmling], FYI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16140) bump owasp.esapi from 2.1.0 to 2.1.0.1
[
https://issues.apache.org/jira/browse/HBASE-16140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353946#comment-15353946
]
Hadoop QA commented on HBASE-16140:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 42s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 34s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 6s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 90m 4s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 125m 19s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12814191/hbase-16140.patch |
| JIRA Issue | HBASE-16140 |
| Optional Tests | asflicense javac javadoc unit xml compile |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 8bc4d41 |
| Default Java | 1.7.0_80 |
| Multi-JDK versions | /home/jenkins/tools/java/jdk1.8.0:1.8.0
/home/jenkins/jenkins-slave/tools/hudson.model.JDK/JDK_1.7_latest_:1.7.0_80 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/2407/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/2407/console |
| Powered by | Apache Yetus 0.2.1 http://yetus.apache.org |
This message was automatically generated.
> bump owasp.esapi from 2.1.0 to 2.1.0.1
> --
>
> Key: HBASE-16140
> URL: https://issues.apache.org/jira/browse/HBASE-16140
> Project: H
[jira] [Commented] (HBASE-16139) Use CellUtil instead of KeyValueUtil in Import.java
[
https://issues.apache.org/jira/browse/HBASE-16139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353944#comment-15353944
]
Hadoop QA commented on HBASE-16139:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 19s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
56s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 7s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 59s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
32s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
19s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
3s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 13s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 13s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
25s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 2 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
30m 9s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 0s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 59s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 0s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 97m 44s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 153m 59s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12814186/HBASE-16139.patch |
[jira] [Commented] (HBASE-16115) Missing security context in RegionObserver coprocessor when a compaction/split is triggered manually
[ https://issues.apache.org/jira/browse/HBASE-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353942#comment-15353942 ] Andrew Purtell commented on HBASE-16115: We've traced occurrences of this back to at least 0.98.16, which may implicate HBASE-14655 but not later changes. We didn't recognize the problem at the time. We don't normally force major compaction from either UI or shell. Timed major compactions work normally. > Missing security context in RegionObserver coprocessor when a > compaction/split is triggered manually > > > Key: HBASE-16115 > URL: https://issues.apache.org/jira/browse/HBASE-16115 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.20 >Reporter: Lars Hofhansl > > We ran into an interesting phenomenon which can easily render a cluster > unusable. > We loaded some tests data into a test table and forced a manual compaction > through the UI. We have some compaction hooks implemented in a region > observer, which writes back to another HBase table when the compaction > finishes. We noticed that this coprocessor is not setup correctly, it seems > the security context is missing. > The interesting part is that this _only_ happens when the compaction is > triggere through the UI. Automatic compactions (major or minor) or when > triggered via the HBase shell (folling a kinit) work fine. Only the > UI-triggered compactions cause this issues and lead to essentially > neverending compactions, immovable regions, etc. > Not sure what exactly the issue is, but I wanted to make sure I capture this. > [~apurtell], [~ghelmling], FYI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16115) Missing security context in RegionObserver coprocessor when a compaction/split is triggered manually
[ https://issues.apache.org/jira/browse/HBASE-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-16115: -- Summary: Missing security context in RegionObserver coprocessor when a compaction/split is triggered manually (was: Missing security context in RegionObserver coprocessor when a compaction is triggered through the UI) > Missing security context in RegionObserver coprocessor when a > compaction/split is triggered manually > > > Key: HBASE-16115 > URL: https://issues.apache.org/jira/browse/HBASE-16115 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.20 >Reporter: Lars Hofhansl > > We ran into an interesting phenomenon which can easily render a cluster > unusable. > We loaded some tests data into a test table and forced a manual compaction > through the UI. We have some compaction hooks implemented in a region > observer, which writes back to another HBase table when the compaction > finishes. We noticed that this coprocessor is not setup correctly, it seems > the security context is missing. > The interesting part is that this _only_ happens when the compaction is > triggere through the UI. Automatic compactions (major or minor) or when > triggered via the HBase shell (folling a kinit) work fine. Only the > UI-triggered compactions cause this issues and lead to essentially > neverending compactions, immovable regions, etc. > Not sure what exactly the issue is, but I wanted to make sure I capture this. > [~apurtell], [~ghelmling], FYI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16130) Add comments to ProcedureStoreTracker
[
https://issues.apache.org/jira/browse/HBASE-16130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353922#comment-15353922
]
Hadoop QA commented on HBASE-16130:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 24s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 11s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
30s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 20s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 25s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 25s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 11s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 11s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
9s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 11s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
35s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 9s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 41s
{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 39m 28s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12814198/HBASE-16130.master.002.patch
|
| JIRA Issue | HBASE-16130 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 8bc4d41 |
| Default Java | 1.7.0_80 |
| Multi-JDK versions | /home/jenkins/tools/java/jdk1.8.0:1.8.0
/home/jenkins/jenkins-slave/tools/h
[jira] [Commented] (HBASE-16078) Create java cli tool for managing balancer states for scripts usage
[ https://issues.apache.org/jira/browse/HBASE-16078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353921#comment-15353921 ] Appy commented on HBASE-16078: -- Just spent longer than an hour reading the link in your prev post. It felt so good understanding git internals. Thanks for the pointer. bq. The idea there is that -procelain- *interactive* user-facing things might change substantially between releases to improve the user experience. In contrast, -plumbing- *non-interactive* command are meant to be used when building on top of things while scripting. Yeah, that's the core idea. Digressing a bit (explaining my use of strikethroughs above). It does look a bit like Git's porcelain and plumbing. It's just that porcelain in git (whole VCS functionality) is much more than what we'll have (simple formatter). :-) And while we'll be able to change our formatter as we wish, i don't think git can change its commands (commit, cherry-pick, etc) easily. :) > Create java cli tool for managing balancer states for scripts usage > --- > > Key: HBASE-16078 > URL: https://issues.apache.org/jira/browse/HBASE-16078 > Project: HBase > Issue Type: Improvement > Components: scripts, util >Affects Versions: 2.0.0 >Reporter: Samir Ahmic >Assignee: Samir Ahmic > Fix For: 2.0.0 > > Attachments: HBASE-16078_v1.patch, HBASE-16078_v2.patch > > > This ticket is result of discussion in > [HBASE16044|https://issues.apache.org/jira/browse/HBASE-16044] to avoid > "hbase shell" output parsing hacks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16130) Add comments to ProcedureStoreTracker
[ https://issues.apache.org/jira/browse/HBASE-16130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-16130: - Attachment: HBASE-16130.master.002.patch > Add comments to ProcedureStoreTracker > - > > Key: HBASE-16130 > URL: https://issues.apache.org/jira/browse/HBASE-16130 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy >Priority: Minor > Attachments: HBASE-16130.master.001.patch, > HBASE-16130.master.002.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16140) bump owasp.esapi from 2.1.0 to 2.1.0.1
[ https://issues.apache.org/jira/browse/HBASE-16140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-16140: --- Attachment: hbase-16140.patch > bump owasp.esapi from 2.1.0 to 2.1.0.1 > -- > > Key: HBASE-16140 > URL: https://issues.apache.org/jira/browse/HBASE-16140 > Project: HBase > Issue Type: Improvement > Components: build >Affects Versions: 2.0.0, 1.2.0, 1.3.0, 1.1.4, 1.0.4 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Attachments: hbase-16140.patch > > > A small pom change to upgrade the library. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16140) bump owasp.esapi from 2.1.0 to 2.1.0.1
[ https://issues.apache.org/jira/browse/HBASE-16140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-16140: --- Status: Patch Available (was: Open) > bump owasp.esapi from 2.1.0 to 2.1.0.1 > -- > > Key: HBASE-16140 > URL: https://issues.apache.org/jira/browse/HBASE-16140 > Project: HBase > Issue Type: Improvement > Components: build >Affects Versions: 1.1.4, 1.2.0, 2.0.0, 1.3.0, 1.0.4 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Attachments: hbase-16140.patch > > > A small pom change to upgrade the library. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16140) bump owasp.esapi from 2.1.0 to 2.1.0.1
[ https://issues.apache.org/jira/browse/HBASE-16140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-16140: --- Affects Version/s: 1.0.4 1.3.0 1.2.0 1.1.4 > bump owasp.esapi from 2.1.0 to 2.1.0.1 > -- > > Key: HBASE-16140 > URL: https://issues.apache.org/jira/browse/HBASE-16140 > Project: HBase > Issue Type: Improvement > Components: build >Affects Versions: 2.0.0, 1.2.0, 1.3.0, 1.1.4, 1.0.4 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > > A small pom change to upgrade the library. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-16140) bump owasp.esapi from 2.1.0 to 2.1.0.1
[ https://issues.apache.org/jira/browse/HBASE-16140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh reassigned HBASE-16140: -- Assignee: Jonathan Hsieh > bump owasp.esapi from 2.1.0 to 2.1.0.1 > -- > > Key: HBASE-16140 > URL: https://issues.apache.org/jira/browse/HBASE-16140 > Project: HBase > Issue Type: Improvement > Components: build >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > > A small pom change to upgrade the library. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16140) bump owasp.esapi from 2.1.0 to 2.1.0.1
Jonathan Hsieh created HBASE-16140: -- Summary: bump owasp.esapi from 2.1.0 to 2.1.0.1 Key: HBASE-16140 URL: https://issues.apache.org/jira/browse/HBASE-16140 Project: HBase Issue Type: Improvement Components: build Affects Versions: 2.0.0 Reporter: Jonathan Hsieh A small pom change to upgrade the library. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16115) Missing security context in RegionObserver coprocessor when a compaction is triggered through the UI
[
https://issues.apache.org/jira/browse/HBASE-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353754#comment-15353754
]
Mujtaba Chohan commented on HBASE-16115:
{noformat}
Stack:
FATAL [ctions-1466815775283] ipc.RpcClient - SASL authentication failed. The
most likely cause is missing or invalid credentials. Consider 'kinit'.
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException:
No valid credentials provided (Mechanism level: Failed to find any Kerberos
tgt)]
at
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
at
org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
at
org.apache.hadoop.hbase.ipc.RpcClient$Connection.setupSaslConnection(RpcClient.java:774)
at
org.apache.hadoop.hbase.ipc.RpcClient$Connection.access$600(RpcClient.java:360)
at
org.apache.hadoop.hbase.ipc.RpcClient$Connection$2.run(RpcClient.java:895)
at
org.apache.hadoop.hbase.ipc.RpcClient$Connection$2.run(RpcClient.java:892)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1706)
at
org.apache.hadoop.hbase.ipc.RpcClient$Connection.setupIOstreams(RpcClient.java:892)
at
org.apache.hadoop.hbase.ipc.RpcClient.getConnection(RpcClient.java:1577)
at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1476)
at
org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1693)
at
org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1760)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.get(ClientProtos.java:32914)
at
org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRowOrBefore(ProtobufUtil.java:1559)
at org.apache.hadoop.hbase.client.HTable$2.call(HTable.java:747)
at org.apache.hadoop.hbase.client.HTable$2.call(HTable.java:745)
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:115)
at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:751)
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:144)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:1261)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1323)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1179)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1136)
at
org.apache.hadoop.hbase.client.AsyncProcess.findDestLocation(AsyncProcess.java:390)
at
org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:335)
at
org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:287)
at
org.apache.hadoop.hbase.client.HTable.backgroundFlushCommits(HTable.java:1019)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1395)
at org.apache.hadoop.hbase.client.HTable.put(HTable.java:965)
at
org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment$HTableWrapper.put(CoprocessorHost.java:478)
at
org.apache.phoenix.schema.stats.StatisticsWriter.commitLastStatsUpdatedTime(StatisticsWriter.java:227)
at
org.apache.phoenix.schema.stats.StatisticsWriter.newWriter(StatisticsWriter.java:83)
at
org.apache.phoenix.schema.stats.DefaultStatisticsCollector.(DefaultStatisticsCollector.java:85)
at
org.apache.phoenix.schema.stats.StatisticsCollectorFactory.createStatisticsCollector(StatisticsCollectorFactory.java:51)
at
org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.preCompact(UngroupedAggregateRegionObserver.java:614)
at
org.apache.hadoop.hbase.coprocessor.BaseRegionObserver.preCompact(BaseRegionObserver.java:197)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$9.call(RegionCoprocessorHost.java:584)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionOperation.call(RegionCoprocessorHost.java:1621)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1697)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperationWithResult(RegionCoprocessorHost.java:1670)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preCompact(RegionCoprocessorHost.java:579)
at
org.apache.hadoop.hbase.regionserver.co
[jira] [Commented] (HBASE-15716) HRegion#RegionScannerImpl scannerReadPoints synchronization constrains random read
[
https://issues.apache.org/jira/browse/HBASE-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353752#comment-15353752
]
Hadoop QA commented on HBASE-15716:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
54s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} branch-1 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 31s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
59s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} branch-1 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 53s
{color} | {color:red} hbase-server in branch-1 has 1 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} branch-1 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s
{color} | {color:green} branch-1 passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 17s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 4s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 78m 30s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 107m 34s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12814171/HBASE-15716.branch-1.004.patch
|
| JIRA Issue | HBASE-15716 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-1 / 7a78d87 |
| Default Java | 1.7.0_80 |
| Multi-JDK versions | /home/jenkins/tools/j
[jira] [Updated] (HBASE-16139) Use CellUtil instead of KeyValueUtil in Import.java
[ https://issues.apache.org/jira/browse/HBASE-16139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] NIDHI GAMBHIR updated HBASE-16139: -- Attachment: HBASE-16139.patch > Use CellUtil instead of KeyValueUtil in Import.java > --- > > Key: HBASE-16139 > URL: https://issues.apache.org/jira/browse/HBASE-16139 > Project: HBase > Issue Type: Improvement >Reporter: NIDHI GAMBHIR >Assignee: NIDHI GAMBHIR >Priority: Minor > Attachments: HBASE-16139.patch > > > Currently in Import.java, map function uses KeyValueUtil.createFirstOnRow. > This method calls createByteArray which causes a lot of copying. Instead use > CellUtil's createFirstOnRow method. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16139) Use CellUtil instead of KeyValueUtil in Import.java
[ https://issues.apache.org/jira/browse/HBASE-16139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] NIDHI GAMBHIR updated HBASE-16139: -- Status: Patch Available (was: Open) patch available for review > Use CellUtil instead of KeyValueUtil in Import.java > --- > > Key: HBASE-16139 > URL: https://issues.apache.org/jira/browse/HBASE-16139 > Project: HBase > Issue Type: Improvement >Reporter: NIDHI GAMBHIR >Assignee: NIDHI GAMBHIR >Priority: Minor > Attachments: HBASE-16139.patch > > > Currently in Import.java, map function uses KeyValueUtil.createFirstOnRow. > This method calls createByteArray which causes a lot of copying. Instead use > CellUtil's createFirstOnRow method. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16115) Missing security context in RegionObserver coprocessor when a compaction is triggered through the UI
[ https://issues.apache.org/jira/browse/HBASE-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353705#comment-15353705 ] Andrew Purtell commented on HBASE-16115: Let's turn on JRE level kerberos debug logging and see what's going on when the compaction request fails. > Missing security context in RegionObserver coprocessor when a compaction is > triggered through the UI > > > Key: HBASE-16115 > URL: https://issues.apache.org/jira/browse/HBASE-16115 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.20 >Reporter: Lars Hofhansl > > We ran into an interesting phenomenon which can easily render a cluster > unusable. > We loaded some tests data into a test table and forced a manual compaction > through the UI. We have some compaction hooks implemented in a region > observer, which writes back to another HBase table when the compaction > finishes. We noticed that this coprocessor is not setup correctly, it seems > the security context is missing. > The interesting part is that this _only_ happens when the compaction is > triggere through the UI. Automatic compactions (major or minor) or when > triggered via the HBase shell (folling a kinit) work fine. Only the > UI-triggered compactions cause this issues and lead to essentially > neverending compactions, immovable regions, etc. > Not sure what exactly the issue is, but I wanted to make sure I capture this. > [~apurtell], [~ghelmling], FYI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16115) Missing security context in RegionObserver coprocessor when a compaction is triggered through the UI
[ https://issues.apache.org/jira/browse/HBASE-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353700#comment-15353700 ] Lars Hofhansl commented on HBASE-16115: --- [~apurtell] > Missing security context in RegionObserver coprocessor when a compaction is > triggered through the UI > > > Key: HBASE-16115 > URL: https://issues.apache.org/jira/browse/HBASE-16115 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.20 >Reporter: Lars Hofhansl > > We ran into an interesting phenomenon which can easily render a cluster > unusable. > We loaded some tests data into a test table and forced a manual compaction > through the UI. We have some compaction hooks implemented in a region > observer, which writes back to another HBase table when the compaction > finishes. We noticed that this coprocessor is not setup correctly, it seems > the security context is missing. > The interesting part is that this _only_ happens when the compaction is > triggere through the UI. Automatic compactions (major or minor) or when > triggered via the HBase shell (folling a kinit) work fine. Only the > UI-triggered compactions cause this issues and lead to essentially > neverending compactions, immovable regions, etc. > Not sure what exactly the issue is, but I wanted to make sure I capture this. > [~apurtell], [~ghelmling], FYI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14422) Fix TestFastFailWithoutTestUtil
[ https://issues.apache.org/jira/browse/HBASE-14422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Ryakhovskiy updated HBASE-14422: --- Attachment: trace.log HBASE-14422.master.002.patch [~bstack], ok, then I would like to attach another patch.002 (and log produced by the patch with test failure -- trace.log). the difference: new patch uses commons-logging instead of std-out. > Fix TestFastFailWithoutTestUtil > --- > > Key: HBASE-14422 > URL: https://issues.apache.org/jira/browse/HBASE-14422 > Project: HBase > Issue Type: Task > Components: test >Reporter: stack >Assignee: Konstantin Ryakhovskiy >Priority: Minor > Labels: beginner > Attachments: HBASE-14422.master.001.patch, > HBASE-14422.master.002.patch, log.txt, trace.log > > > TestFastFailWithoutTestUtil has a unit test that does > testInterceptorIntercept50Times Usually it passes but on occasion, the > latching between thread 1 and thread 2 goes awry and the test hangs and the > test hangs out. Depends on the hardware but it seems to happen about one in > four runs here on an internal rig. > HBASE-14421 changed the wait-on-latch to timeout and do a thread dump and > just let the test keep going. > This issue is about digging in on figuring why the hang up on latches and > then fixing it so the test doesn't have to have the latch timeout. Hopefully > the threaddump helps. > This one could be hard to fix since it not easy to reproduce. Marking it > beginner anyways. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16115) Missing security context in RegionObserver coprocessor when a compaction is triggered through the UI
[ https://issues.apache.org/jira/browse/HBASE-16115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353688#comment-15353688 ] Lars Hofhansl commented on HBASE-16115: --- We've now also seen the same with a compactions triggered from the HBase shell, so the UI was likely dud. :( What was reported now is that this appears to work when the cluster/server was recently rebooted. [~mujtabachohan], can you attach the stack trace you have? > Missing security context in RegionObserver coprocessor when a compaction is > triggered through the UI > > > Key: HBASE-16115 > URL: https://issues.apache.org/jira/browse/HBASE-16115 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.20 >Reporter: Lars Hofhansl > > We ran into an interesting phenomenon which can easily render a cluster > unusable. > We loaded some tests data into a test table and forced a manual compaction > through the UI. We have some compaction hooks implemented in a region > observer, which writes back to another HBase table when the compaction > finishes. We noticed that this coprocessor is not setup correctly, it seems > the security context is missing. > The interesting part is that this _only_ happens when the compaction is > triggere through the UI. Automatic compactions (major or minor) or when > triggered via the HBase shell (folling a kinit) work fine. Only the > UI-triggered compactions cause this issues and lead to essentially > neverending compactions, immovable regions, etc. > Not sure what exactly the issue is, but I wanted to make sure I capture this. > [~apurtell], [~ghelmling], FYI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
