[jira] [Commented] (HBASE-18588) Verify we're using netty .so epolling on linux post HBASE-18271
[
https://issues.apache.org/jira/browse/HBASE-18588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126819#comment-16126819
]
Guanghao Zhang commented on HBASE-18588:
OS: Ubuntu.
java version "1.8.0_131".
Apache Maven 3.3.9.
I run "mvn clean test -Dtest=TestXX", then get a error. Is it related?
{code}
Discovery starting.
Discovery completed in 1 second, 161 milliseconds.
Run starting. Expected test count is: 79
HBaseDStreamFunctionsSuite:
Formatting using clusterid: testClusterID
*** RUN ABORTED ***
java.io.IOException: Shutting down
at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:232)
at org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1065)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:936)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:930)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)
at
org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll(HBaseDStreamFunctionsSuite.scala:41)
at
org.scalatest.BeforeAndAfterAll$class.beforeAll(BeforeAndAfterAll.scala:187)
at
org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll(HBaseDStreamFunctionsSuite.scala:30)
...
Cause: java.lang.RuntimeException: Failed construction of Master: class
org.apache.hadoop.hbase.master.HMasterorg.apache.hadoop.hbase.shaded.io.netty.channel.epoll.NativeStaticallyReferencedJniMethods.epollin()I
at
org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:145)
at
org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:217)
at
org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:152)
at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:214)
at org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:94)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1065)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:936)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:930)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)
...
Cause: java.lang.UnsatisfiedLinkError: failed to load the required native
library
at
org.apache.hadoop.hbase.shaded.io.netty.channel.epoll.Epoll.ensureAvailability(Epoll.java:78)
at
org.apache.hadoop.hbase.shaded.io.netty.channel.epoll.EpollEventLoopGroup.(EpollEventLoopGroup.java:38)
at
org.apache.hadoop.hbase.util.NettyEventLoopGroupConfig.(NettyEventLoopGroupConfig.java:61)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:553)
at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:469)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:140)
...
Cause: java.lang.UnsatisfiedLinkError:
org.apache.hadoop.hbase.shaded.io.netty.channel.epoll.NativeStaticallyReferencedJniMethods.epollin()I
at
org.apache.hadoop.hbase.shaded.io.netty.channel.epoll.NativeStaticallyReferencedJniMethods.epollin(Native
Method)
at
org.apache.hadoop.hbase.shaded.io.netty.channel.epoll.Native.(Native.java:66)
at
org.apache.hadoop.hbase.shaded.io.netty.channel.epoll.Epoll.(Epoll.java:33)
at
org.apache.hadoop.hbase.shaded.io.netty.channel.epoll.EpollEventLoopGroup.(EpollEventLoopGroup.java:38)
at
org.apache.hadoop.hbase.util.NettyEventLoopGroupConfig.(NettyEventLoopGroupConfig.java:61)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:553)
at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:469)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
...
{code}
> Verify we're using netty .so epolling on linux post HBASE-18271
> ---
>
> Key: HBASE-18588
> URL: https://issues.apache.org/jira/browse/HBASE-18588
[jira] [Commented] (HBASE-7266) [89-fb] Using pread for non-compaction read request
[ https://issues.apache.org/jira/browse/HBASE-7266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126804#comment-16126804 ] vinisha commented on HBASE-7266: Which HBase versions does it affect? > [89-fb] Using pread for non-compaction read request > --- > > Key: HBASE-7266 > URL: https://issues.apache.org/jira/browse/HBASE-7266 > Project: HBase > Issue Type: Improvement > Components: Performance >Reporter: Liyin Tang > > There are 2 kinds of read operations in HBase: pread and seek+read. > Pread, positional read, is stateless and create a new connection between the > DFSClient and DataNode for each operation. While seek+read is to seek to a > specific postion and prefetch blocks from data nodes. The benefit of > seek+read is that it will cache the prefetch result but the downside is it is > stateful and needs to synchronized. > So far, both compaction and scan are using seek+read, which caused some > resource contention. So using the pread for the scan request can avoid the > resource contention. In addition, the region server is able to do the > prefetch for the scan request (HBASE-6874) so that it won't be necessary to > let the DFSClient to prefetch the data any more. > I will run through the scan benchmark (with no block cache) with verify the > performance. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[
https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126789#comment-16126789
]
Hadoop QA commented on HBASE-18508:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 12m
59s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
9s{color} | {color:green} HBASE-14070.HLC passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
47s{color} | {color:green} HBASE-14070.HLC passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s{color} | {color:green} HBASE-14070.HLC passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s{color} | {color:green} HBASE-14070.HLC passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
57s{color} | {color:red} hbase-server in HBASE-14070.HLC has 9 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} HBASE-14070.HLC passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
31m 36s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 92m 33s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}154m 9s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.master.procedure.TestDisableTableProcedure |
| | org.apache.hadoop.hbase.master.procedure.TestModifyTableProcedure |
| | org.apache.hadoop.hbase.master.procedure.TestCreateTableProcedure |
| | org.apache.hadoop.hbase.master.procedure.TestEnableTableProcedure |
| | org.apache.hadoop.hbase.master.procedure.TestDeleteTableProcedure |
| | org.apache.hadoop.hbase.client.TestSnapshotCloneIndependence |
| | org.apache.hadoop.hbase.master.TestGetLastFlushedSequenceId |
| | org.apache.hadoop.hbase.client.TestFromClientSide |
| | org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer2 |
| | org.apache.hadoop.hbase.master.TestAssignmentManagerMetrics |
| | org.apache.hadoop.hbase.namespace.TestNamespaceAuditor |
| | org.apache.hadoop.hbase.TestHBaseTestingUtility |
| | org.apache.hadoop.hbase.mapred.TestTableInputFormat |
| | org.apache.hadoop.hbase.client.TestAsyncNamespaceAdminApi |
| | org.apache.hadoop.hbase.client.TestAsyncTableBatch |
| | org.apache.hadoop.hbase.client.TestAsyncProcedureAdminApi |
| | org.apache.hadoop.hbase.TestMultiVersions |
| | org.apache.hadoop.hbase.quotas.TestQuotaStatusRPCs |
| |
org.apache.hadoop.hbase.security.visibility.TestVisibilityLablesWithGroups |
| |
org.apache.hadoop.hbase.security.visibility.TestEnforcingScanLabelGenerator |
| |
[jira] [Resolved] (HBASE-18524) Test reenabling Master carrying regions
[ https://issues.apache.org/jira/browse/HBASE-18524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-18524. --- Resolution: Not A Problem Assignee: stack Resovling as not-a-problem. Over in HBASE-18511 we add tests that show master can carry any region, not just system tables. > Test reenabling Master carrying regions > --- > > Key: HBASE-18524 > URL: https://issues.apache.org/jira/browse/HBASE-18524 > Project: HBase > Issue Type: Sub-task > Components: master >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > > This issue is to ensure that it is still possible to configure Master to > carry tables. Further, it should be possible to configure Master to carry > system tables only with Master doing short-circuit ops -- going directly to > the internal RS methods w/o going via RPC -- when accessing system tables. > We also need a mechanism for specifying that the Master can carry any region, > not just system table regions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-18154) Verify Master carrying system-tables only with short-circuit RPC
[ https://issues.apache.org/jira/browse/HBASE-18154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-18154. --- Resolution: Not A Problem Assignee: stack Resolving as not a problem. Over in HBASE-18511 I verified we do short-circuit RPC if we've enabled master-carrying-regions. It also adds the called for tests and figures how we are going to deliver master-with-tables vs master-w/o-tables. > Verify Master carrying system-tables only with short-circuit RPC > > > Key: HBASE-18154 > URL: https://issues.apache.org/jira/browse/HBASE-18154 > Project: HBase > Issue Type: Sub-task > Components: Region Assignment >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > > This sort-of-works in new AMv2 given this the current default. Spend some > time on it. Ensure that when Master goes down, on relaunch, that it gets the > system tables back. Ensure we are doing the short-circuit RPC. > RegionServer Groups might be the right conduit for this sort of deploy type. > Review. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18511) Default no regions on master
[ https://issues.apache.org/jira/browse/HBASE-18511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126642#comment-16126642 ] stack edited comment on HBASE-18511 at 8/15/17 4:17 AM: .007 changes hbase.balancer.tablesOnMaster to be boolean. If enabled, master is like any other regionserver. If false -- the default -- then the master carries no tables. A new config, hbase.balancer.tablesOnMaster.systemTablesOnly, when enabled makes it so master carries system tables. Adds a test that tries the three configs along w/ some master killing to make sure the deploy-type prevails past master kill. A bunch of other tests are changed. The old deploy mode of master exclusively carrying system tables was around so long, a bunch of tests came to depend on this layout. Coarse test changes were made so deal w/ the new deploy mode. For a few tests, they don't make sense in the new deploy mode. On why I changed the type of the flag, as is, the implication was that you could specify any set of tables for Master to host -- it did not have to be system tables only. Also you had to hand specify the system table set. On the former, already have a well-developed mechanism for matching tables to server sets, it is called rsgroup. This mechanism on master was an implementation of a subset of rsgroup. HBase is hard enough already w/o two means to same end. Given system tables on master was a config some of us require, a simple flag to enable this exception seemed way to go. Oh, as part of testing I verified that we are indeed doing short-circuit RPC if we find a region colocated. [~zyork] You ok w/ this sir? Anyone up for a review? This is last one before alpha2. Thanks. was (Author: stack): .007 changes hbase.balancer.tablesOnMaster to be boolean. If enabled, master is like any other regionserver. If false -- the default -- then the master carries no tables. A new config, hbase.balancer.tablesOnMaster.systemTablesOnly, when enabled makes it so master carries system tables. Adds a test that tries the three configs along w/ some master killing to make sure the deploy-type prevails past master kill. A bunch of other tests are changed. The old deploy mode of master exclusively carrying system tables was around so long, a bunch of tests came to depend on this layout. Coarse test changes were made so deal w/ the new deploy mode. For a few tests, they don't make sense in the new deploy mode. [~zyork] You ok w/ this sir? Anyone up for a review? This is last one before alpha2. Thanks. > Default no regions on master > > > Key: HBASE-18511 > URL: https://issues.apache.org/jira/browse/HBASE-18511 > Project: HBase > Issue Type: Task > Components: master >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-18511.master.001.patch, > HBASE-18511.master.002.patch, HBASE-18511.master.003.patch, > HBASE-18511.master.004.patch, HBASE-18511.master.005.patch, > HBASE-18511.master.006.patch, HBASE-18511.master.007.patch > > > Let this be umbrella issue for no-regions-on-master as default deploy (as it > was in branch-1). > Also need to make sure we can run WITH regions on master; in particular > system tables with RPC short-circuit as it is now in hbase master. > Background is that master branch carried a change that allowed Master carry > regions. On top of this improvement on branch-1, Master defaulted to carry > system tables only. No release was made with this configuration. Now we are > going to cut the 2.0.0 release, the decision is that hbase-2 should have the > same layout as hbase-1 so this issue implements the undoing of Master > carrying system tables by default (though the capability remains). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-15511) ClusterStatus should be able to return responses by scope
[ https://issues.apache.org/jira/browse/HBASE-15511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan updated HBASE-15511: -- Release Note: Provide a new way to get ClusterStatus with a filter to filter out unwanted information by using ClusterStatus.Options, such that the response back to client can be limited. Note that, the constructor way to new a ClusterStatus will be no longer support after 2.0.0, and use ClusterStatus.Builder instead. > ClusterStatus should be able to return responses by scope > - > > Key: HBASE-15511 > URL: https://issues.apache.org/jira/browse/HBASE-15511 > Project: HBase > Issue Type: Improvement >Reporter: Esteban Gutierrez >Assignee: Reid Chan > Fix For: 2.0.0 > > Attachments: HBASE-15511.master.001.patch, > HBASE-15511.master.002.patch, HBASE-15511.master.003.patch, > HBASE-15511.master.004.patch, HBASE-15511.master.005.patch, > HBASE-15511.master.006.patch, HBASE-15511.master.007.patch, > HBASE-15511.master.008.patch, HBASE-15511.master.009.patch, > HBASE-15511.master.010.patch, HBASE-15511.master.011.patch, > HBASE-15511.master.012.patch, HBASE-15511.master.013.patch, > HBASE-15511.master.014.patch > > > The current ClusterStatus response returns too much information about the > load per region and replication cluster wide. Sometimes that response can be > quite large (10s or 100s of MBs) and methods like getServerSize() or > getRegionsCount() don't really need the full response. One possibility is to > provide a scope (or filter) for the ClusterStatus requests to limit the > response back to the client. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18588) Verify we're using netty .so epolling on linux post HBASE-18271
[ https://issues.apache.org/jira/browse/HBASE-18588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126769#comment-16126769 ] Duo Zhang commented on HBASE-18588: --- [~stack] Thanks sir, will try it soon and report back here. > Verify we're using netty .so epolling on linux post HBASE-18271 > --- > > Key: HBASE-18588 > URL: https://issues.apache.org/jira/browse/HBASE-18588 > Project: HBase > Issue Type: Task > Components: test >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > > This is a task to verify that indeed we are using .so native epoll on linux. > This task is probably unnecessary since we'd fail on the linux build box if > this was not in place but verify that our relocation is indeed finding the > native code. Assigned myself. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HBASE-17064) Add TaskMonitor#getTasks() variant which accepts type selection
[
https://issues.apache.org/jira/browse/HBASE-17064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan reassigned HBASE-17064:
-
Assignee: Reid Chan
> Add TaskMonitor#getTasks() variant which accepts type selection
> ---
>
> Key: HBASE-17064
> URL: https://issues.apache.org/jira/browse/HBASE-17064
> Project: HBase
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: Reid Chan
>Priority: Minor
> Labels: ui
>
> In TaskMonitorTmpl.jamon :
> {code}
> List tasks = taskMonitor.getTasks();
> Iterator iter = tasks.iterator();
> // apply requested filter
> while (iter.hasNext()) {
> MonitoredTask t = iter.next();
> if (filter.equals("general")) {
> if (t instanceof MonitoredRPCHandler)
> iter.remove();
> {code}
> This means when user refreshes rs-status page, regardless of the type of
> filter, we always traverse and clone MonitoredTask's.
> getTasks() is synchronized :
> {code}
> public synchronized List getTasks() {
> {code}
> A variant of getTasks() can be added which takes the type of filter so that
> unnecessary cloning is avoided.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18546) Append/Increment a cell with custom timestamp
[
https://issues.apache.org/jira/browse/HBASE-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126758#comment-16126758
]
Phil Yang commented on HBASE-18546:
---
IIRC, for append/increment we will set a ts which must be larger than the
previous one in WAL. Does we really need setting timestamp for append/increment
at client?
> Append/Increment a cell with custom timestamp
> -
>
> Key: HBASE-18546
> URL: https://issues.apache.org/jira/browse/HBASE-18546
> Project: HBase
> Issue Type: New Feature
> Components: API, Client
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Critical
> Labels: incompatibleChange
> Fix For: 2.0.0
>
> Attachments: HBASE-18546.v0.patch, HBASE-18546.v1.patch
>
>
> We don't pass the custom timestamp for Increment, and the increment's
> timestamp always be rewrite. Hence, user can't increment a cell with custom
> timestamp.
> {code:title=ProtobufUtil.java}
> if (values != null && values.size() > 0) {
> for (Cell cell: values) {
> valueBuilder.clear();
> valueBuilder.setQualifier(UnsafeByteOperations.unsafeWrap(
> cell.getQualifierArray(), cell.getQualifierOffset(),
> cell.getQualifierLength()));
> valueBuilder.setValue(UnsafeByteOperations.unsafeWrap(
> cell.getValueArray(), cell.getValueOffset(),
> cell.getValueLength()));
> if (cell.getTagsLength() > 0) {
>
> valueBuilder.setTags(UnsafeByteOperations.unsafeWrap(cell.getTagsArray(),
> cell.getTagsOffset(), cell.getTagsLength()));
> }
> columnBuilder.addQualifierValue(valueBuilder.build());
> }
> }
> {code}
> In contrast to Increment, user can append the cell with custom timestamp. It
> would be better that make their behavior consistent.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18508: --- Status: Patch Available (was: In Progress) > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18510) Update clock on replaying recovered edits
[ https://issues.apache.org/jira/browse/HBASE-18510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126701#comment-16126701 ] Hudson commented on HBASE-18510: FAILURE: Integrated in Jenkins build HBASE-14070.HLC #213 (See [https://builds.apache.org/job/HBASE-14070.HLC/213/]) HBASE-18510 Update clock on replaying recovered edits (Sai Teja Ranuva) (appy: rev 58a51bcb72250c3173d418a6d3fffb0c1f0d446c) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java * (edit) hbase-common/src/main/java/org/apache/hadoop/hbase/TimestampType.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java > Update clock on replaying recovered edits > - > > Key: HBASE-18510 > URL: https://issues.apache.org/jira/browse/HBASE-18510 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Fix For: HBASE-14070 > > Attachments: HBASE-18510.HBASE-14070.HLC.001.patch, > HBASE-18510.HBASE-14070.HLC.002.patch > > > This task covers updating the clock on recovery in > HRegion#replayRecoveredEdits and includes region-level tests. > Credit for the baseline of this work all goes to our [~saitejar]. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18497) Add clock type to proto message for HLC region open/close
[ https://issues.apache.org/jira/browse/HBASE-18497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126700#comment-16126700 ] Hudson commented on HBASE-18497: FAILURE: Integrated in Jenkins build HBASE-14070.HLC #213 (See [https://builds.apache.org/job/HBASE-14070.HLC/213/]) HBASE-18497 Add clock type to proto message for HLC region open/close (appy: rev 93c0d71c47ba850b4353872082ddc1ffa76dd30d) * (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/AdminProtos.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/MockRegionServerServices.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/RequestConverter.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterServices.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * (edit) hbase-protocol-shaded/src/main/protobuf/Admin.proto * (edit) hbase-protocol-shaded/src/main/protobuf/HBase.proto * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/RSProcedureDispatcher.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockNoopMasterServices.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java * (edit) hbase-protocol-shaded/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/generated/HBaseProtos.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java > Add clock type to proto message for HLC region open/close > - > > Key: HBASE-18497 > URL: https://issues.apache.org/jira/browse/HBASE-18497 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Fix For: HBASE-14070 > > Attachments: HBASE-18497.HBASE-14070.HLC.001.patch, > HBASE-18497.HBASE-14070.HLC.002.patch, HBASE-18497.HBASE-14070.HLC.003.patch, > HBASE-18497.HBASE-14070.HLC.004.patch > > > This task covers the work for adding clock type to the protobuf message > NodeTime used for sending timestamps of each clock during region open and > close requests/responses. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18303) Clean up some parameterized test declarations
[ https://issues.apache.org/jira/browse/HBASE-18303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1612#comment-1612 ] Hudson commented on HBASE-18303: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3533 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3533/]) HBASE-18303 Clean up @Parameter boilerplate (mdrob: rev 0b26ccdaa1b8700e7958aeebbaf9cad81e737dd0) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockIndex.java * (edit) hbase-prefix-tree/src/test/java/org/apache/hadoop/hbase/codec/keyvalue/TestKeyValueTool.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java * (edit) hbase-common/src/test/java/org/apache/hadoop/hbase/HBaseCommonTestingUtility.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/encoding/TestSeekToBlockWithEncoders.java * (edit) hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestByteBufferUtils.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV3.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java * (edit) hbase-client/src/test/java/org/apache/hadoop/hbase/filter/TestKeyOnlyFilter.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * (edit) hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/TestMultiRowResource.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestCoprocessorScanPolicy.java * (edit) hbase-prefix-tree/src/test/java/org/apache/hadoop/hbase/codec/prefixtree/row/TestRowEncoder.java * (edit) hbase-common/src/test/java/org/apache/hadoop/hbase/types/TestStruct.java > Clean up some parameterized test declarations > - > > Key: HBASE-18303 > URL: https://issues.apache.org/jira/browse/HBASE-18303 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Minor > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18303.patch, HBASE-18303.patch > > > While debugging something unrelated, I noticed that we use the constructor > form of junit parameterized tests, instead of the annotated members form. > I personally find using the @Parameter annotation more clear. > Also, we can move the parameter generator to hbase-common so that it is > accessible in more modules. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18238) Address ruby static analysis for bin directory
[ https://issues.apache.org/jira/browse/HBASE-18238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126665#comment-16126665 ] Hudson commented on HBASE-18238: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3533 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3533/]) HBASE-18238 rubocop autocorrect for bin/ (mdrob: rev ea8fa59a4c2fe7633ebe70df622098bfe36b5df9) * (edit) bin/replication/copy_tables_desc.rb * (edit) bin/shutdown_regionserver.rb * (edit) bin/region_mover.rb * (edit) bin/get-active-master.rb * (edit) bin/draining_servers.rb * (edit) bin/hirb.rb * (edit) bin/region_status.rb > Address ruby static analysis for bin directory > -- > > Key: HBASE-18238 > URL: https://issues.apache.org/jira/browse/HBASE-18238 > Project: HBase > Issue Type: Sub-task >Reporter: Mike Drob >Assignee: Mike Drob > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18238.patch, HBASE-18238.v2.patch, report.txt > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18522) Add RowMutations support to Batch
[ https://issues.apache.org/jira/browse/HBASE-18522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126664#comment-16126664 ] Hudson commented on HBASE-18522: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3533 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3533/]) HBASE-18522 Add RowMutations support to Batch (jerryjch: rev 096dac2e83c675f212bad4f91888d8440ba152ca) * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ResponseConverter.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/MultiServerCallable.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Table.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/RequestConverter.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java > Add RowMutations support to Batch > - > > Key: HBASE-18522 > URL: https://issues.apache.org/jira/browse/HBASE-18522 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.2.6 >Reporter: Jerry He >Assignee: Jerry He > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0 > > Attachments: HBASE-18522-branch-1.patch, > HBASE-18522-branch-1-v2.patch, HBASE-18522-master-v2.patch > > > RowMutations is multiple Puts and/or Deletes atomically on a single row. > Current Batch call does not support RowMutations as part of the batch. > We should add this missing part. We should be able to batch RowMutations. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18533) Expose BucketCache values to be configured
[ https://issues.apache.org/jira/browse/HBASE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126667#comment-16126667 ] Hudson commented on HBASE-18533: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3533 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3533/]) HBASE-18533 Expose BucketCache values to be configured (tedyu: rev 0e32869f01697abf29292aa786d0cdcca10213c6) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestBucketWriterThread.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestBucketCache.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java > Expose BucketCache values to be configured > -- > > Key: HBASE-18533 > URL: https://issues.apache.org/jira/browse/HBASE-18533 > Project: HBase > Issue Type: Improvement > Components: BucketCache >Reporter: Zach York >Assignee: Zach York > Fix For: 3.0.0, 1.4.0, 1.5.0, 2.0.0-alpha-2 > > Attachments: HBASE-18533.branch-1.001.patch, > HBASE-18533.branch-1.002.patch, HBASE-18533.branch-1.003.patch, > HBASE-18533.master.001.patch, HBASE-18533.master.002.patch, > HBASE-18533.master.003.patch > > > BucketCache always uses the default values for all cache configuration. > However, this doesn't work for all use cases. In particular, users want to be > able to configure the percentage of the cache that is single access, multi > access, and in-memory access. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18511) Default no regions on master
[ https://issues.apache.org/jira/browse/HBASE-18511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18511: -- Hadoop Flags: Incompatible change Release Note: Changes the configuration hbase.balancer.tablesOnMaster from list of table names that the can carry (with 'none' meaning no tables on the master) to instead be a boolean that is set to true if master carries tables/regions and false if it does not. If true, the master acts like any regionserver. If false, then the master carries no tables. This is the default for hbase-2.0.0. Another boolean configuration, hbase.balancer.tablesOnMaster.systemTablesOnly, when set to true, enables hbase.balancer.tablesOnMaster and makes it so the master hosts system tables exclusively (the long-time deploy mode of master branch and branch-2 up until this commit). .007 changes hbase.balancer.tablesOnMaster to be boolean. If enabled, master is like any other regionserver. If false -- the default -- then the master carries no tables. A new config, hbase.balancer.tablesOnMaster.systemTablesOnly, when enabled makes it so master carries system tables. Adds a test that tries the three configs along w/ some master killing to make sure the deploy-type prevails past master kill. A bunch of other tests are changed. The old deploy mode of master exclusively carrying system tables was around so long, a bunch of tests came to depend on this layout. Coarse test changes were made so deal w/ the new deploy mode. For a few tests, they don't make sense in the new deploy mode. [~zyork] You ok w/ this sir? Anyone up for a review? This is last one before alpha2. Thanks. > Default no regions on master > > > Key: HBASE-18511 > URL: https://issues.apache.org/jira/browse/HBASE-18511 > Project: HBase > Issue Type: Task > Components: master >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-18511.master.001.patch, > HBASE-18511.master.002.patch, HBASE-18511.master.003.patch, > HBASE-18511.master.004.patch, HBASE-18511.master.005.patch, > HBASE-18511.master.006.patch, HBASE-18511.master.007.patch > > > Let this be umbrella issue for no-regions-on-master as default deploy (as it > was in branch-1). > Also need to make sure we can run WITH regions on master; in particular > system tables with RPC short-circuit as it is now in hbase master. > Background is that master branch carried a change that allowed Master carry > regions. On top of this improvement on branch-1, Master defaulted to carry > system tables only. No release was made with this configuration. Now we are > going to cut the 2.0.0 release, the decision is that hbase-2 should have the > same layout as hbase-1 so this issue implements the undoing of Master > carrying system tables by default (though the capability remains). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18547) Provide better cleanup of WAL related entries in backup table
[ https://issues.apache.org/jira/browse/HBASE-18547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126637#comment-16126637 ] Ted Yu commented on HBASE-18547: One potential solution is for cleaner chore to notify BackupLogCleaner which files have been successfully deleted. BackupLogCleaner can respond and clean up related entries in backup table. > Provide better cleanup of WAL related entries in backup table > - > > Key: HBASE-18547 > URL: https://issues.apache.org/jira/browse/HBASE-18547 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu > > I was discussing the design around incremental backup with Vladimir. > Currently if WAL file is recorded in backup table, BackupLogCleaner would > signal that the file can be cleaned up. > However, cleaner chore doesn't notify BackupLogCleaner whether the file is > actually deleted. > This means that potentially large number of WAL file entries would stay in > backup table. > This issue is to investigate better cleanup strategy for the WAL entries. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18511) Default no regions on master
[ https://issues.apache.org/jira/browse/HBASE-18511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18511: -- Attachment: HBASE-18511.master.007.patch > Default no regions on master > > > Key: HBASE-18511 > URL: https://issues.apache.org/jira/browse/HBASE-18511 > Project: HBase > Issue Type: Task > Components: master >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-18511.master.001.patch, > HBASE-18511.master.002.patch, HBASE-18511.master.003.patch, > HBASE-18511.master.004.patch, HBASE-18511.master.005.patch, > HBASE-18511.master.006.patch, HBASE-18511.master.007.patch > > > Let this be umbrella issue for no-regions-on-master as default deploy (as it > was in branch-1). > Also need to make sure we can run WITH regions on master; in particular > system tables with RPC short-circuit as it is now in hbase master. > Background is that master branch carried a change that allowed Master carry > regions. On top of this improvement on branch-1, Master defaulted to carry > system tables only. No release was made with this configuration. Now we are > going to cut the 2.0.0 release, the decision is that hbase-2 should have the > same layout as hbase-1 so this issue implements the undoing of Master > carrying system tables by default (though the capability remains). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16338) update jackson to 2.y
[
https://issues.apache.org/jira/browse/HBASE-16338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126625#comment-16126625
]
Hadoop QA commented on HBASE-16338:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m
2s{color} | {color:blue} The patch file was not named according to hbase's
naming conventions. Please see
https://yetus.apache.org/documentation/0.4.0/precommit-patchnames for
instructions. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
45s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
7s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
32m 0s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
26s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}137m
25s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
30s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 4s{color}
| {color:red} hbase-rest in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}177m 10s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black}
[jira] [Reopened] (HBASE-18579) Enable core dump by default for docker
[ https://issues.apache.org/jira/browse/HBASE-18579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu reopened HBASE-18579: > Enable core dump by default for docker > -- > > Key: HBASE-18579 > URL: https://issues.apache.org/jira/browse/HBASE-18579 > Project: HBase > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Ted Yu >Priority: Minor > Fix For: HBASE-14850 > > Attachments: 18579.v1.txt, 18579.v2.txt > > > In recent debugging experience, I found that by default the ulimit value > prohibits the generation of core dump. This makes debugging difficult. > We should enable core dump generation by default. > https://www.akadia.com/services/ora_enable_core.html -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18533) Expose BucketCache values to be configured
[ https://issues.apache.org/jira/browse/HBASE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126556#comment-16126556 ] Hudson commented on HBASE-18533: FAILURE: Integrated in Jenkins build HBase-1.4 #856 (See [https://builds.apache.org/job/HBase-1.4/856/]) HBASE-18533 Expose BucketCache values to be configured (tedyu: rev 81f5da7af8525c34018df345eb8936b1b9851cc3) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestBucketWriterThread.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestBucketCache.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java > Expose BucketCache values to be configured > -- > > Key: HBASE-18533 > URL: https://issues.apache.org/jira/browse/HBASE-18533 > Project: HBase > Issue Type: Improvement > Components: BucketCache >Reporter: Zach York >Assignee: Zach York > Fix For: 3.0.0, 1.4.0, 1.5.0, 2.0.0-alpha-2 > > Attachments: HBASE-18533.branch-1.001.patch, > HBASE-18533.branch-1.002.patch, HBASE-18533.branch-1.003.patch, > HBASE-18533.master.001.patch, HBASE-18533.master.002.patch, > HBASE-18533.master.003.patch > > > BucketCache always uses the default values for all cache configuration. > However, this doesn't work for all use cases. In particular, users want to be > able to configure the percentage of the cache that is single access, multi > access, and in-memory access. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18573) Update Append and Delete to use Mutation#getCellList(family)
[
https://issues.apache.org/jira/browse/HBASE-18573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126549#comment-16126549
]
Jerry He commented on HBASE-18573:
--
There is no guarantee that initializing it with 1 will do any good. No
assumption can be made on the number of columns that will be added to a family.
> Update Append and Delete to use Mutation#getCellList(family)
>
>
> Key: HBASE-18573
> URL: https://issues.apache.org/jira/browse/HBASE-18573
> Project: HBase
> Issue Type: Improvement
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
>
> In addxxx() of Put and Increment, Mutation#getCellList(family) is called to
> get cell list from familyMap. But in the other 2 sub-class of Mutation:
> Append and Delete, the logic like Mutation#getCellList(family) is used, like
> {code}
> List list = familyMap.get(family);
> if(list == null) {
> list = new ArrayList<>(1);
> }
> {code}
> in
> {code}
> public Delete addColumn(byte [] family, byte [] qualifier, long timestamp)
> {code}
> of Delete
> We could make them to call Mutation#getCellList(family) to get better
> encapsulation
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18533) Expose BucketCache values to be configured
[ https://issues.apache.org/jira/browse/HBASE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126550#comment-16126550 ] Hudson commented on HBASE-18533: FAILURE: Integrated in Jenkins build HBase-1.5 #3 (See [https://builds.apache.org/job/HBase-1.5/3/]) HBASE-18533 Expose BucketCache values to be configured (tedyu: rev d6a781cf0823b5051d929455218d6a4720150394) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestBucketCache.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestBucketWriterThread.java > Expose BucketCache values to be configured > -- > > Key: HBASE-18533 > URL: https://issues.apache.org/jira/browse/HBASE-18533 > Project: HBase > Issue Type: Improvement > Components: BucketCache >Reporter: Zach York >Assignee: Zach York > Fix For: 3.0.0, 1.4.0, 1.5.0, 2.0.0-alpha-2 > > Attachments: HBASE-18533.branch-1.001.patch, > HBASE-18533.branch-1.002.patch, HBASE-18533.branch-1.003.patch, > HBASE-18533.master.001.patch, HBASE-18533.master.002.patch, > HBASE-18533.master.003.patch > > > BucketCache always uses the default values for all cache configuration. > However, this doesn't work for all use cases. In particular, users want to be > able to configure the percentage of the cache that is single access, multi > access, and in-memory access. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126519#comment-16126519 ] Amit Patel edited comment on HBASE-18508 at 8/14/17 10:34 PM: -- Added patch [^HBASE-18508.HBASE-14070.HLC.001.patch] that may fix the timed out test by resetting the EEM in TestIncrementTimeRange and TestCellACLWithMultipleVersions. If the EEM is set and then a mini cluster is started, the test will time out and these tests do not reset the EEM at the end so it is likely that this is causing other tests to timeout. was (Author: amit.patel): Added patch that may fix the timed out test by resetting the EEM in TestIncrementTimeRange and TestCellACLWithMultipleVersions. If the EEM is set and then a mini cluster is started, the test will time out and these tests do not reset the EEM at the end so it is likely that this is causing other tests to timeout. > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18508: --- Attachment: HBASE-18508.HBASE-14070.HLC.001.patch > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18508: --- Status: Open (was: Patch Available) > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18508: --- Attachment: (was: HBASE-18508.HBASE-14070.HLC.001.patch) > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18508: --- Status: Patch Available (was: In Progress) > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Work started] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-18508 started by Amit Patel. -- > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126519#comment-16126519 ] Amit Patel commented on HBASE-18508: Added patch that may fix the timed out test by resetting the EEM in TestIncrementTimeRange and TestCellACLWithMultipleVersions. If the EEM is set and then a mini cluster is started, the test will time out and these tests do not reset the EEM at the end so it is likely that this is causing other tests to timeout. > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Work started] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-18508 started by Amit Patel. -- > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18303) Clean up some parameterized test declarations
[ https://issues.apache.org/jira/browse/HBASE-18303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126514#comment-16126514 ] Hudson commented on HBASE-18303: FAILURE: Integrated in Jenkins build HBase-2.0 #330 (See [https://builds.apache.org/job/HBase-2.0/330/]) HBASE-18303 Clean up @Parameter boilerplate (mdrob: rev 0ded122b1ebaa006e84f12a7a051eb8eaca571c3) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV3.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/encoding/TestSeekToBlockWithEncoders.java * (edit) hbase-client/src/test/java/org/apache/hadoop/hbase/filter/TestKeyOnlyFilter.java * (edit) hbase-common/src/test/java/org/apache/hadoop/hbase/HBaseCommonTestingUtility.java * (edit) hbase-prefix-tree/src/test/java/org/apache/hadoop/hbase/codec/prefixtree/row/TestRowEncoder.java * (edit) hbase-common/src/test/java/org/apache/hadoop/hbase/types/TestStruct.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestCoprocessorScanPolicy.java * (edit) hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/TestMultiRowResource.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockIndex.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java * (edit) hbase-prefix-tree/src/test/java/org/apache/hadoop/hbase/codec/keyvalue/TestKeyValueTool.java * (edit) hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestByteBufferUtils.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java > Clean up some parameterized test declarations > - > > Key: HBASE-18303 > URL: https://issues.apache.org/jira/browse/HBASE-18303 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Minor > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18303.patch, HBASE-18303.patch > > > While debugging something unrelated, I noticed that we use the constructor > form of junit parameterized tests, instead of the annotated members form. > I personally find using the @Parameter annotation more clear. > Also, we can move the parameter generator to hbase-common so that it is > accessible in more modules. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18508) [HLC] Fix timing out tests in HBASE-14070.HLC branch
[ https://issues.apache.org/jira/browse/HBASE-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18508: --- Attachment: HBASE-18508.HBASE-14070.HLC.001.patch > [HLC] Fix timing out tests in HBASE-14070.HLC branch > > > Key: HBASE-18508 > URL: https://issues.apache.org/jira/browse/HBASE-18508 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Amit Patel > Attachments: HBASE-18508.HBASE-14070.HLC.001.patch > > > Pre-commit runs for this branch output a huge list of timed out tests. Not > seeing those in master branch. > Needs to be fixed before merging. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18533) Expose BucketCache values to be configured
[ https://issues.apache.org/jira/browse/HBASE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126515#comment-16126515 ] Hudson commented on HBASE-18533: FAILURE: Integrated in Jenkins build HBase-2.0 #330 (See [https://builds.apache.org/job/HBase-2.0/330/]) HBASE-18533 Expose BucketCache values to be configured (tedyu: rev 26bbc8ad6c683a9965f61e2bce0e7caf5fd2a37e) * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestBucketCache.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestBucketWriterThread.java > Expose BucketCache values to be configured > -- > > Key: HBASE-18533 > URL: https://issues.apache.org/jira/browse/HBASE-18533 > Project: HBase > Issue Type: Improvement > Components: BucketCache >Reporter: Zach York >Assignee: Zach York > Fix For: 3.0.0, 1.4.0, 1.5.0, 2.0.0-alpha-2 > > Attachments: HBASE-18533.branch-1.001.patch, > HBASE-18533.branch-1.002.patch, HBASE-18533.branch-1.003.patch, > HBASE-18533.master.001.patch, HBASE-18533.master.002.patch, > HBASE-18533.master.003.patch > > > BucketCache always uses the default values for all cache configuration. > However, this doesn't work for all use cases. In particular, users want to be > able to configure the percentage of the cache that is single access, multi > access, and in-memory access. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18595) Set version in branch-2 from 2.0.0-alpha2-SNAPSHOT to 2.0.0-alpha2
[ https://issues.apache.org/jira/browse/HBASE-18595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126511#comment-16126511 ] Hudson commented on HBASE-18595: FAILURE: Integrated in Jenkins build HBase-2.0 #330 (See [https://builds.apache.org/job/HBase-2.0/330/]) HBASE-18595 Set version in branch-2 from 2.0.0-alpha2-SNAPSHOT to (stack: rev add99745155dec52a11e2fbd88497fcda88a39b9) * (edit) hbase-prefix-tree/pom.xml * (edit) hbase-shaded/pom.xml * (edit) hbase-archetypes/hbase-client-project/pom.xml * (edit) hbase-server/pom.xml * (edit) hbase-hadoop2-compat/pom.xml * (edit) hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/RESTServer.java * (edit) hbase-metrics/pom.xml * (edit) hbase-checkstyle/pom.xml * (edit) hbase-archetypes/hbase-shaded-client-project/pom.xml * (edit) hbase-examples/pom.xml * (edit) hbase-protocol/pom.xml * (edit) hbase-assembly/pom.xml * (edit) hbase-common/pom.xml * (edit) hbase-thrift/pom.xml * (edit) hbase-client/pom.xml * (edit) hbase-rsgroup/pom.xml * (edit) hbase-shaded/hbase-shaded-server/pom.xml * (edit) hbase-shaded/hbase-shaded-client/pom.xml * (edit) pom.xml * (edit) hbase-shell/pom.xml * (edit) hbase-archetypes/pom.xml * (edit) hbase-external-blockcache/pom.xml * (edit) hbase-annotations/pom.xml * (edit) hbase-procedure/pom.xml * (edit) hbase-archetypes/hbase-archetype-builder/pom.xml * (edit) hbase-it/pom.xml * (edit) hbase-protocol-shaded/pom.xml * (edit) hbase-resource-bundle/pom.xml * (edit) hbase-rest/pom.xml * (edit) hbase-spark-it/pom.xml * (edit) hbase-testing-util/pom.xml * (edit) hbase-hadoop-compat/pom.xml * (edit) hbase-metrics-api/pom.xml * (edit) hbase-endpoint/pom.xml * (edit) hbase-spark/pom.xml * (edit) hbase-rest/src/test/java/org/apache/hadoop/hbase/rest/HBaseRESTTestingUtility.java > Set version in branch-2 from 2.0.0-alpha2-SNAPSHOT to 2.0.0-alpha2 > -- > > Key: HBASE-18595 > URL: https://issues.apache.org/jira/browse/HBASE-18595 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack > Fix For: 2.0.0-alpha-2 > > > Set version for alpha2 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18522) Add RowMutations support to Batch
[ https://issues.apache.org/jira/browse/HBASE-18522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126512#comment-16126512 ] Hudson commented on HBASE-18522: FAILURE: Integrated in Jenkins build HBase-2.0 #330 (See [https://builds.apache.org/job/HBase-2.0/330/]) HBASE-18522 Add RowMutations support to Batch (jerryjch: rev cf050de9172a36b767eb7e4700787b9d6a32a1b9) * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ResponseConverter.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/MultiServerCallable.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Table.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/RequestConverter.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java > Add RowMutations support to Batch > - > > Key: HBASE-18522 > URL: https://issues.apache.org/jira/browse/HBASE-18522 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.2.6 >Reporter: Jerry He >Assignee: Jerry He > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0 > > Attachments: HBASE-18522-branch-1.patch, > HBASE-18522-branch-1-v2.patch, HBASE-18522-master-v2.patch > > > RowMutations is multiple Puts and/or Deletes atomically on a single row. > Current Batch call does not support RowMutations as part of the batch. > We should add this missing part. We should be able to batch RowMutations. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18238) Address ruby static analysis for bin directory
[ https://issues.apache.org/jira/browse/HBASE-18238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126513#comment-16126513 ] Hudson commented on HBASE-18238: FAILURE: Integrated in Jenkins build HBase-2.0 #330 (See [https://builds.apache.org/job/HBase-2.0/330/]) HBASE-18238 rubocop autocorrect for bin/ (mdrob: rev 4e9961b4fcb198a74d292e6a5405755dce261dcc) * (edit) bin/get-active-master.rb * (edit) bin/hirb.rb * (edit) bin/region_mover.rb * (edit) bin/replication/copy_tables_desc.rb * (edit) bin/shutdown_regionserver.rb * (edit) bin/draining_servers.rb * (edit) bin/region_status.rb > Address ruby static analysis for bin directory > -- > > Key: HBASE-18238 > URL: https://issues.apache.org/jira/browse/HBASE-18238 > Project: HBase > Issue Type: Sub-task >Reporter: Mike Drob >Assignee: Mike Drob > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18238.patch, HBASE-18238.v2.patch, report.txt > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18532) Improve cache related stats rendered on RS UI
[
https://issues.apache.org/jira/browse/HBASE-18532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Biju Nair updated HBASE-18532:
--
Attachment: HBASE-18532.WIP
Attaching a WIP patch. Will update it based on the feedback for COUNT/SIZE
stats on {{DATA BLOCKS}}.
> Improve cache related stats rendered on RS UI
> -
>
> Key: HBASE-18532
> URL: https://issues.apache.org/jira/browse/HBASE-18532
> Project: HBase
> Issue Type: Improvement
> Components: regionserver, UI
>Affects Versions: 1.1.2
>Reporter: Biju Nair
>Assignee: Biju Nair
> Attachments: COMBINED-STATS.PNG, HBASE-18532.WIP, L1-STATS.PNG,
> L2-STATS.PNG
>
>
> The stats currently rendered for L1 and L2 cache are incorrect. Refer to the
> attached screenshots of stats from a cluster showing the combined cache
> stats, L1 stats and L2 stats. For e.g. the combined stats shows 38 GB used
> for cache while if we sum size of L1 and L2 cache the value is way less. One
> way we can improve this is to use the same stats used to populate the
> combined stats to render the values of L1 & L2 cache. Also for usability we
> can remove the table with details with BucketCache buckets from the L2 cache
> stats since this is going to be long for any installation using L2 cache.
> This will help in understanding the cache usage better. Thoughts? If there
> are no concerns will submit a patch.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18587) Fix Flaky TestFileIOEngine
[
https://issues.apache.org/jira/browse/HBASE-18587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126464#comment-16126464
]
Hadoop QA commented on HBASE-18587:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
10s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} branch-1 passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
42s{color} | {color:green} branch-1 passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
3s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
20s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
19s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} branch-1 passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} branch-1 passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} the patch passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} the patch passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
18m 2s{color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} the patch passed with JDK v1.8.0_144 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} the patch passed with JDK v1.7.0_131 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}108m 25s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}143m 25s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestClientScannerRPCTimeout |
| | hadoop.hbase.TestZooKeeper |
| | hadoop.hbase.regionserver.TestCompactionInDeadRegionServer |
| | hadoop.hbase.master.TestMasterFailover |
| Timed out junit tests | org.apache.hadoop.hbase.security.access.TestCellACLs |
| | org.apache.hadoop.hbase.mapreduce.TestTableSnapshotInputFormat |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:6f1cc2c |
| JIRA Issue | HBASE-18587 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12881809/HBASE-18587.branch-1.002.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs
[jira] [Commented] (HBASE-17614) Move Backup/Restore into separate module
[ https://issues.apache.org/jira/browse/HBASE-17614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126434#comment-16126434 ] Josh Elser commented on HBASE-17614: Just had a chat with Vlad offline. He already has this one on his plate for this week so I assigned it accordingly. > Move Backup/Restore into separate module > - > > Key: HBASE-17614 > URL: https://issues.apache.org/jira/browse/HBASE-17614 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Blocker > Fix For: 2.0.0, HBASE-7912 > > > Move all the backup code into separate hbase-backup module. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HBASE-17614) Move Backup/Restore into separate module
[ https://issues.apache.org/jira/browse/HBASE-17614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser reassigned HBASE-17614: -- Assignee: Vladimir Rodionov > Move Backup/Restore into separate module > - > > Key: HBASE-17614 > URL: https://issues.apache.org/jira/browse/HBASE-17614 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Blocker > Fix For: 2.0.0, HBASE-7912 > > > Move all the backup code into separate hbase-backup module. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HBASE-18532) Improve cache related stats rendered on RS UI
[ https://issues.apache.org/jira/browse/HBASE-18532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Biju Nair reassigned HBASE-18532: - Assignee: Biju Nair > Improve cache related stats rendered on RS UI > - > > Key: HBASE-18532 > URL: https://issues.apache.org/jira/browse/HBASE-18532 > Project: HBase > Issue Type: Improvement > Components: regionserver, UI >Affects Versions: 1.1.2 >Reporter: Biju Nair >Assignee: Biju Nair > Attachments: COMBINED-STATS.PNG, L1-STATS.PNG, L2-STATS.PNG > > > The stats currently rendered for L1 and L2 cache are incorrect. Refer to the > attached screenshots of stats from a cluster showing the combined cache > stats, L1 stats and L2 stats. For e.g. the combined stats shows 38 GB used > for cache while if we sum size of L1 and L2 cache the value is way less. One > way we can improve this is to use the same stats used to populate the > combined stats to render the values of L1 & L2 cache. Also for usability we > can remove the table with details with BucketCache buckets from the L2 cache > stats since this is going to be long for any installation using L2 cache. > This will help in understanding the cache usage better. Thoughts? If there > are no concerns will submit a patch. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18532) Improve cache related stats rendered on RS UI
[
https://issues.apache.org/jira/browse/HBASE-18532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126399#comment-16126399
]
Biju Nair commented on HBASE-18532:
---
You are correct. Sure can provide a patch. The {{count}} and {{size}} of DATA
BLOCKS currently rendered in the L1 & L2 cache UI is from {{cbsbf}}.
{{BlockCache}} doesn't seem to keep track of DATA BLOCKS. Either we need to
keep track of it to provide the actual value or remove it from the UI.
> Improve cache related stats rendered on RS UI
> -
>
> Key: HBASE-18532
> URL: https://issues.apache.org/jira/browse/HBASE-18532
> Project: HBase
> Issue Type: Improvement
> Components: regionserver, UI
>Affects Versions: 1.1.2
>Reporter: Biju Nair
> Attachments: COMBINED-STATS.PNG, L1-STATS.PNG, L2-STATS.PNG
>
>
> The stats currently rendered for L1 and L2 cache are incorrect. Refer to the
> attached screenshots of stats from a cluster showing the combined cache
> stats, L1 stats and L2 stats. For e.g. the combined stats shows 38 GB used
> for cache while if we sum size of L1 and L2 cache the value is way less. One
> way we can improve this is to use the same stats used to populate the
> combined stats to render the values of L1 & L2 cache. Also for usability we
> can remove the table with details with BucketCache buckets from the L2 cache
> stats since this is going to be long for any installation using L2 cache.
> This will help in understanding the cache usage better. Thoughts? If there
> are no concerns will submit a patch.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18526) FIFOCompactionPolicy pre-check uses wrong scope
[
https://issues.apache.org/jira/browse/HBASE-18526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126386#comment-16126386
]
Ted Yu commented on HBASE-18526:
Patch for master cannot be applied on branch-1:
{code}
patching file
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
Reversed (or previously applied) patch detected! Assume -R? [n]
Apply anyway? [n]
Skipping patch.
5 out of 5 hunks ignored -- saving rejects to file
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java.rej
{code}
> FIFOCompactionPolicy pre-check uses wrong scope
> ---
>
> Key: HBASE-18526
> URL: https://issues.apache.org/jira/browse/HBASE-18526
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 1.3.1
>Reporter: Lars George
>Assignee: Vladimir Rodionov
> Attachments: HBASE-18526-v1.patch
>
>
> See https://issues.apache.org/jira/browse/HBASE-14468
> It adds this check to {{HMaster.checkCompactionPolicy()}}:
> {code}
> // 1. Check TTL
> if (hcd.getTimeToLive() == HColumnDescriptor.DEFAULT_TTL) {
> message = "Default TTL is not supported for FIFO compaction";
> throw new IOException(message);
> }
> // 2. Check min versions
> if (hcd.getMinVersions() > 0) {
> message = "MIN_VERSION > 0 is not supported for FIFO compaction";
> throw new IOException(message);
> }
> // 3. blocking file count
> String sbfc = htd.getConfigurationValue(HStore.BLOCKING_STOREFILES_KEY);
> if (sbfc != null) {
> blockingFileCount = Integer.parseInt(sbfc);
> }
> if (blockingFileCount < 1000) {
> message =
> "blocking file count '" + HStore.BLOCKING_STOREFILES_KEY + "' "
> + blockingFileCount
> + " is below recommended minimum of 1000";
> throw new IOException(message);
> }
> {code}
> Why does it only check the blocking file count on the HTD level, while
> others are check on the HCD level? Doing this for example fails
> because of it:
> {noformat}
> hbase(main):008:0> create 'ttltable', { NAME => 'cf1', TTL => 300,
> CONFIGURATION => { 'hbase.hstore.defaultengine.compactionpolicy.class'
> => 'org.apache.hadoop.hbase.regionserver.compactions.FIFOCompactionPolicy',
> 'hbase.hstore.blockingStoreFiles' => 2000 } }
> ERROR: org.apache.hadoop.hbase.DoNotRetryIOException: blocking file
> count 'hbase.hstore.blockingStoreFiles' 10 is below recommended
> minimum of 1000 Set hbase.table.sanity.checks to false at conf or
> table descriptor if you want to bypass sanity checks
> at
> org.apache.hadoop.hbase.master.HMaster.warnOrThrowExceptionForFailure(HMaster.java:1782)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1663)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1545)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:469)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:58549)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2339)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168)
> Caused by: java.io.IOException: blocking file count
> 'hbase.hstore.blockingStoreFiles' 10 is below recommended minimum of
> 1000
> at
> org.apache.hadoop.hbase.master.HMaster.checkCompactionPolicy(HMaster.java:1773)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1661)
> ... 7 more
> {noformat}
> The check should be performed on the column family level instead.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18587) Fix Flaky TestFileIOEngine
[ https://issues.apache.org/jira/browse/HBASE-18587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126379#comment-16126379 ] Chia-Ping Tsai commented on HBASE-18587: +1 > Fix Flaky TestFileIOEngine > -- > > Key: HBASE-18587 > URL: https://issues.apache.org/jira/browse/HBASE-18587 > Project: HBase > Issue Type: Bug > Components: BucketCache, test >Reporter: Zach York >Assignee: Zach York > Attachments: HBASE-18587.branch-1.001.patch, > HBASE-18587.branch-1.002.patch, HBASE-18587.master.001.patch, > HBASE-18587.master.001.patch, HBASE-18587.master.002.patch > > > As a part of HBASE-18533, the Jenkins report said that TestFileIOEngine > failed. I investigated and noticed that there are two cases where this test > is flaky: > len = 0 and offset 0; it will fail for > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/FileIOEngine.java#L198 > will pass in -1 for offset which is invalid. > len = 0 and i = 13; it will try to specify an offset of the totalCapacity > which is not allowed. > > This patch fixes this test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18526) FIFOCompactionPolicy pre-check uses wrong scope
[
https://issues.apache.org/jira/browse/HBASE-18526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126377#comment-16126377
]
Vladimir Rodionov commented on HBASE-18526:
---
Should apply for all branches.
> FIFOCompactionPolicy pre-check uses wrong scope
> ---
>
> Key: HBASE-18526
> URL: https://issues.apache.org/jira/browse/HBASE-18526
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 1.3.1
>Reporter: Lars George
>Assignee: Vladimir Rodionov
> Attachments: HBASE-18526-v1.patch
>
>
> See https://issues.apache.org/jira/browse/HBASE-14468
> It adds this check to {{HMaster.checkCompactionPolicy()}}:
> {code}
> // 1. Check TTL
> if (hcd.getTimeToLive() == HColumnDescriptor.DEFAULT_TTL) {
> message = "Default TTL is not supported for FIFO compaction";
> throw new IOException(message);
> }
> // 2. Check min versions
> if (hcd.getMinVersions() > 0) {
> message = "MIN_VERSION > 0 is not supported for FIFO compaction";
> throw new IOException(message);
> }
> // 3. blocking file count
> String sbfc = htd.getConfigurationValue(HStore.BLOCKING_STOREFILES_KEY);
> if (sbfc != null) {
> blockingFileCount = Integer.parseInt(sbfc);
> }
> if (blockingFileCount < 1000) {
> message =
> "blocking file count '" + HStore.BLOCKING_STOREFILES_KEY + "' "
> + blockingFileCount
> + " is below recommended minimum of 1000";
> throw new IOException(message);
> }
> {code}
> Why does it only check the blocking file count on the HTD level, while
> others are check on the HCD level? Doing this for example fails
> because of it:
> {noformat}
> hbase(main):008:0> create 'ttltable', { NAME => 'cf1', TTL => 300,
> CONFIGURATION => { 'hbase.hstore.defaultengine.compactionpolicy.class'
> => 'org.apache.hadoop.hbase.regionserver.compactions.FIFOCompactionPolicy',
> 'hbase.hstore.blockingStoreFiles' => 2000 } }
> ERROR: org.apache.hadoop.hbase.DoNotRetryIOException: blocking file
> count 'hbase.hstore.blockingStoreFiles' 10 is below recommended
> minimum of 1000 Set hbase.table.sanity.checks to false at conf or
> table descriptor if you want to bypass sanity checks
> at
> org.apache.hadoop.hbase.master.HMaster.warnOrThrowExceptionForFailure(HMaster.java:1782)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1663)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1545)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:469)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:58549)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2339)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168)
> Caused by: java.io.IOException: blocking file count
> 'hbase.hstore.blockingStoreFiles' 10 is below recommended minimum of
> 1000
> at
> org.apache.hadoop.hbase.master.HMaster.checkCompactionPolicy(HMaster.java:1773)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1661)
> ... 7 more
> {noformat}
> The check should be performed on the column family level instead.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18251) Remove unnecessary traversing to the first and last keys in the CellSet
[
https://issues.apache.org/jira/browse/HBASE-18251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126378#comment-16126378
]
Hadoop QA commented on HBASE-18251:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
33m 38s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}120m
23s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}171m 47s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:bdc94b1 |
| JIRA Issue | HBASE-18251 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12881800/HBASE-18251-v2.patch |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux f973063b5001 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 096dac2 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/8081/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8081/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Remove unnecessary traversing to the first and last keys in the CellSet
> ---
>
> Key: HBASE-18251
> URL: https://issues.apache.org/jira/browse/HBASE-18251
> Project: HBase
> Issue Type: Bug
>Reporter: Anastasia Braginsky
>
[jira] [Commented] (HBASE-18526) FIFOCompactionPolicy pre-check uses wrong scope
[
https://issues.apache.org/jira/browse/HBASE-18526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126374#comment-16126374
]
Ted Yu commented on HBASE-18526:
Patch for master branch has been committed.
Waiting for branch-1 patch.
> FIFOCompactionPolicy pre-check uses wrong scope
> ---
>
> Key: HBASE-18526
> URL: https://issues.apache.org/jira/browse/HBASE-18526
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 1.3.1
>Reporter: Lars George
>Assignee: Vladimir Rodionov
> Attachments: HBASE-18526-v1.patch
>
>
> See https://issues.apache.org/jira/browse/HBASE-14468
> It adds this check to {{HMaster.checkCompactionPolicy()}}:
> {code}
> // 1. Check TTL
> if (hcd.getTimeToLive() == HColumnDescriptor.DEFAULT_TTL) {
> message = "Default TTL is not supported for FIFO compaction";
> throw new IOException(message);
> }
> // 2. Check min versions
> if (hcd.getMinVersions() > 0) {
> message = "MIN_VERSION > 0 is not supported for FIFO compaction";
> throw new IOException(message);
> }
> // 3. blocking file count
> String sbfc = htd.getConfigurationValue(HStore.BLOCKING_STOREFILES_KEY);
> if (sbfc != null) {
> blockingFileCount = Integer.parseInt(sbfc);
> }
> if (blockingFileCount < 1000) {
> message =
> "blocking file count '" + HStore.BLOCKING_STOREFILES_KEY + "' "
> + blockingFileCount
> + " is below recommended minimum of 1000";
> throw new IOException(message);
> }
> {code}
> Why does it only check the blocking file count on the HTD level, while
> others are check on the HCD level? Doing this for example fails
> because of it:
> {noformat}
> hbase(main):008:0> create 'ttltable', { NAME => 'cf1', TTL => 300,
> CONFIGURATION => { 'hbase.hstore.defaultengine.compactionpolicy.class'
> => 'org.apache.hadoop.hbase.regionserver.compactions.FIFOCompactionPolicy',
> 'hbase.hstore.blockingStoreFiles' => 2000 } }
> ERROR: org.apache.hadoop.hbase.DoNotRetryIOException: blocking file
> count 'hbase.hstore.blockingStoreFiles' 10 is below recommended
> minimum of 1000 Set hbase.table.sanity.checks to false at conf or
> table descriptor if you want to bypass sanity checks
> at
> org.apache.hadoop.hbase.master.HMaster.warnOrThrowExceptionForFailure(HMaster.java:1782)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1663)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1545)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:469)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:58549)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2339)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168)
> Caused by: java.io.IOException: blocking file count
> 'hbase.hstore.blockingStoreFiles' 10 is below recommended minimum of
> 1000
> at
> org.apache.hadoop.hbase.master.HMaster.checkCompactionPolicy(HMaster.java:1773)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1661)
> ... 7 more
> {noformat}
> The check should be performed on the column family level instead.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18424) Fix TestAsyncTableGetMultiThreaded
[ https://issues.apache.org/jira/browse/HBASE-18424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126373#comment-16126373 ] Vladimir Rodionov commented on HBASE-18424: --- [~syuanjiang], [~Apache9], the patch is ready to be committed. > Fix TestAsyncTableGetMultiThreaded > -- > > Key: HBASE-18424 > URL: https://issues.apache.org/jira/browse/HBASE-18424 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Attachments: HBASE-18424-v1.patch, HBASE-18424-v2.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18526) FIFOCompactionPolicy pre-check uses wrong scope
[
https://issues.apache.org/jira/browse/HBASE-18526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126372#comment-16126372
]
Vladimir Rodionov commented on HBASE-18526:
---
[~tedyu] can you commit this one?
> FIFOCompactionPolicy pre-check uses wrong scope
> ---
>
> Key: HBASE-18526
> URL: https://issues.apache.org/jira/browse/HBASE-18526
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 1.3.1
>Reporter: Lars George
>Assignee: Vladimir Rodionov
> Attachments: HBASE-18526-v1.patch
>
>
> See https://issues.apache.org/jira/browse/HBASE-14468
> It adds this check to {{HMaster.checkCompactionPolicy()}}:
> {code}
> // 1. Check TTL
> if (hcd.getTimeToLive() == HColumnDescriptor.DEFAULT_TTL) {
> message = "Default TTL is not supported for FIFO compaction";
> throw new IOException(message);
> }
> // 2. Check min versions
> if (hcd.getMinVersions() > 0) {
> message = "MIN_VERSION > 0 is not supported for FIFO compaction";
> throw new IOException(message);
> }
> // 3. blocking file count
> String sbfc = htd.getConfigurationValue(HStore.BLOCKING_STOREFILES_KEY);
> if (sbfc != null) {
> blockingFileCount = Integer.parseInt(sbfc);
> }
> if (blockingFileCount < 1000) {
> message =
> "blocking file count '" + HStore.BLOCKING_STOREFILES_KEY + "' "
> + blockingFileCount
> + " is below recommended minimum of 1000";
> throw new IOException(message);
> }
> {code}
> Why does it only check the blocking file count on the HTD level, while
> others are check on the HCD level? Doing this for example fails
> because of it:
> {noformat}
> hbase(main):008:0> create 'ttltable', { NAME => 'cf1', TTL => 300,
> CONFIGURATION => { 'hbase.hstore.defaultengine.compactionpolicy.class'
> => 'org.apache.hadoop.hbase.regionserver.compactions.FIFOCompactionPolicy',
> 'hbase.hstore.blockingStoreFiles' => 2000 } }
> ERROR: org.apache.hadoop.hbase.DoNotRetryIOException: blocking file
> count 'hbase.hstore.blockingStoreFiles' 10 is below recommended
> minimum of 1000 Set hbase.table.sanity.checks to false at conf or
> table descriptor if you want to bypass sanity checks
> at
> org.apache.hadoop.hbase.master.HMaster.warnOrThrowExceptionForFailure(HMaster.java:1782)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1663)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1545)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:469)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:58549)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2339)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168)
> Caused by: java.io.IOException: blocking file count
> 'hbase.hstore.blockingStoreFiles' 10 is below recommended minimum of
> 1000
> at
> org.apache.hadoop.hbase.master.HMaster.checkCompactionPolicy(HMaster.java:1773)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1661)
> ... 7 more
> {noformat}
> The check should be performed on the column family level instead.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18533) Expose BucketCache values to be configured
[ https://issues.apache.org/jira/browse/HBASE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-18533: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: (was: 2.0.0) 2.0.0-alpha-2 3.0.0 Status: Resolved (was: Patch Available) Thanks for the patch, Zach. > Expose BucketCache values to be configured > -- > > Key: HBASE-18533 > URL: https://issues.apache.org/jira/browse/HBASE-18533 > Project: HBase > Issue Type: Improvement > Components: BucketCache >Reporter: Zach York >Assignee: Zach York > Fix For: 3.0.0, 1.4.0, 1.5.0, 2.0.0-alpha-2 > > Attachments: HBASE-18533.branch-1.001.patch, > HBASE-18533.branch-1.002.patch, HBASE-18533.branch-1.003.patch, > HBASE-18533.master.001.patch, HBASE-18533.master.002.patch, > HBASE-18533.master.003.patch > > > BucketCache always uses the default values for all cache configuration. > However, this doesn't work for all use cases. In particular, users want to be > able to configure the percentage of the cache that is single access, multi > access, and in-memory access. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18593) Tell m2eclipse what to do w/ replacer plugin
[
https://issues.apache.org/jira/browse/HBASE-18593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126351#comment-16126351
]
Hudson commented on HBASE-18593:
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3532 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/3532/])
HBASE-18593 Tell m2eclipse what to do w/ replacer plugin (stack: rev
bd40073094b248f74ac9a3c0fff7ef6668265feb)
* (edit) hbase-protocol-shaded/pom.xml
> Tell m2eclipse what to do w/ replacer plugin
>
>
> Key: HBASE-18593
> URL: https://issues.apache.org/jira/browse/HBASE-18593
> Project: HBase
> Issue Type: Sub-task
> Components: build
>Reporter: stack
>Assignee: stack
>Priority: Trivial
> Fix For: 2.0.0
>
>
> Tripped over this playing w/ eclipse today. The mvn maven-replacer-plugin
> plugin that we added to hbase-protocol-shaded triggers outside usual
> lifecycle. The newer m2eclipse plugins need to be told explicitly what to do
> w/ plugins of this type. Let me commit this:
> {code}
> diff --git a/hbase-protocol-shaded/pom.xml b/hbase-protocol-shaded/pom.xml
> index b28c03e740..4c72ecaa93 100644
> --- a/hbase-protocol-shaded/pom.xml
> +++ b/hbase-protocol-shaded/pom.xml
> @@ -192,6 +192,23 @@
>
>
>
> +
> +
> +
> + com.google.code.maven-replacer-plugin
> +
> +replacer
> +[1.5.3,)
> +
> + replace
> +
> +
> +
> +
> + false
> +
> +
> +
>
>
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18588) Verify we're using netty .so epolling on linux post HBASE-18271
[ https://issues.apache.org/jira/browse/HBASE-18588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126350#comment-16126350 ] Hudson commented on HBASE-18588: FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3532 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3532/]) HBASE-18588 Verify we're using netty .so epolling on linux post (stack: rev ddbaf56ca8c712dc44608d3323280f578c56aed2) * (edit) hbase-protocol-shaded/pom.xml Revert "HBASE-18588 Verify we're using netty .so epolling on linux post (stack: rev 424dff20607577901c06cb40b1293ea5051ec5c5) * (edit) hbase-protocol-shaded/pom.xml > Verify we're using netty .so epolling on linux post HBASE-18271 > --- > > Key: HBASE-18588 > URL: https://issues.apache.org/jira/browse/HBASE-18588 > Project: HBase > Issue Type: Task > Components: test >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > > This is a task to verify that indeed we are using .so native epoll on linux. > This task is probably unnecessary since we'd fail on the linux build box if > this was not in place but verify that our relocation is indeed finding the > native code. Assigned myself. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18522) Add RowMutations support to Batch
[ https://issues.apache.org/jira/browse/HBASE-18522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126349#comment-16126349 ] Hudson commented on HBASE-18522: FAILURE: Integrated in Jenkins build HBase-1.4 #855 (See [https://builds.apache.org/job/HBase-1.4/855/]) HBASE-18522 Add RowMutations support to Batch (jerryjch: rev c6f57e0f382e9dcef48f05da087d12eb0e47e9ad) * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Table.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/MultiServerCallable.java > Add RowMutations support to Batch > - > > Key: HBASE-18522 > URL: https://issues.apache.org/jira/browse/HBASE-18522 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.2.6 >Reporter: Jerry He >Assignee: Jerry He > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0 > > Attachments: HBASE-18522-branch-1.patch, > HBASE-18522-branch-1-v2.patch, HBASE-18522-master-v2.patch > > > RowMutations is multiple Puts and/or Deletes atomically on a single row. > Current Batch call does not support RowMutations as part of the batch. > We should add this missing part. We should be able to batch RowMutations. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18588) Verify we're using netty .so epolling on linux post HBASE-18271
[ https://issues.apache.org/jira/browse/HBASE-18588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126346#comment-16126346 ] Hudson commented on HBASE-18588: FAILURE: Integrated in Jenkins build HBase-2.0 #329 (See [https://builds.apache.org/job/HBase-2.0/329/]) HBASE-18588 Verify we're using netty .so epolling on linux post (stack: rev b4793a03554c08a9e33b76457a1c1fe347aa0f2f) * (edit) hbase-protocol-shaded/pom.xml Revert "HBASE-18588 Verify we're using netty .so epolling on linux post (stack: rev c8d56bb13e68680613121df5e5de4e4d197211e2) * (edit) hbase-protocol-shaded/pom.xml > Verify we're using netty .so epolling on linux post HBASE-18271 > --- > > Key: HBASE-18588 > URL: https://issues.apache.org/jira/browse/HBASE-18588 > Project: HBase > Issue Type: Task > Components: test >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > > This is a task to verify that indeed we are using .so native epoll on linux. > This task is probably unnecessary since we'd fail on the linux build box if > this was not in place but verify that our relocation is indeed finding the > native code. Assigned myself. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18593) Tell m2eclipse what to do w/ replacer plugin
[
https://issues.apache.org/jira/browse/HBASE-18593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126347#comment-16126347
]
Hudson commented on HBASE-18593:
FAILURE: Integrated in Jenkins build HBase-2.0 #329 (See
[https://builds.apache.org/job/HBase-2.0/329/])
HBASE-18593 Tell m2eclipse what to do w/ replacer plugin (stack: rev
c20ce21fe800289ed902cacbf3d72ad614264442)
* (edit) hbase-protocol-shaded/pom.xml
> Tell m2eclipse what to do w/ replacer plugin
>
>
> Key: HBASE-18593
> URL: https://issues.apache.org/jira/browse/HBASE-18593
> Project: HBase
> Issue Type: Sub-task
> Components: build
>Reporter: stack
>Assignee: stack
>Priority: Trivial
> Fix For: 2.0.0
>
>
> Tripped over this playing w/ eclipse today. The mvn maven-replacer-plugin
> plugin that we added to hbase-protocol-shaded triggers outside usual
> lifecycle. The newer m2eclipse plugins need to be told explicitly what to do
> w/ plugins of this type. Let me commit this:
> {code}
> diff --git a/hbase-protocol-shaded/pom.xml b/hbase-protocol-shaded/pom.xml
> index b28c03e740..4c72ecaa93 100644
> --- a/hbase-protocol-shaded/pom.xml
> +++ b/hbase-protocol-shaded/pom.xml
> @@ -192,6 +192,23 @@
>
>
>
> +
> +
> +
> + com.google.code.maven-replacer-plugin
> +
> +replacer
> +[1.5.3,)
> +
> + replace
> +
> +
> +
> +
> + false
> +
> +
> +
>
>
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18587) Fix Flaky TestFileIOEngine
[ https://issues.apache.org/jira/browse/HBASE-18587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126344#comment-16126344 ] Ted Yu commented on HBASE-18587: Good by me. See if Chia-ping has other comments. > Fix Flaky TestFileIOEngine > -- > > Key: HBASE-18587 > URL: https://issues.apache.org/jira/browse/HBASE-18587 > Project: HBase > Issue Type: Bug > Components: BucketCache, test >Reporter: Zach York >Assignee: Zach York > Attachments: HBASE-18587.branch-1.001.patch, > HBASE-18587.branch-1.002.patch, HBASE-18587.master.001.patch, > HBASE-18587.master.001.patch, HBASE-18587.master.002.patch > > > As a part of HBASE-18533, the Jenkins report said that TestFileIOEngine > failed. I investigated and noticed that there are two cases where this test > is flaky: > len = 0 and offset 0; it will fail for > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/FileIOEngine.java#L198 > will pass in -1 for offset which is invalid. > len = 0 and i = 13; it will try to specify an offset of the totalCapacity > which is not allowed. > > This patch fixes this test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18457) [HLC] Follow up work of updating clocks on assign/unassign
[ https://issues.apache.org/jira/browse/HBASE-18457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-18457: - Summary: [HLC] Follow up work of updating clocks on assign/unassign (was: [HLC] Follow up work of updating clocks (from HBASE-18395)) > [HLC] Follow up work of updating clocks on assign/unassign > -- > > Key: HBASE-18457 > URL: https://issues.apache.org/jira/browse/HBASE-18457 > Project: HBase > Issue Type: Sub-task >Reporter: Appy > > So list of things that need to be considered in future > - Mocking clocks in HMaster/RegionServer to test that clocks get updated on > assign/unassign > - Discuss the idea of adding timestamp type to NodeTime proto --> Get rid of > isLikelyOfType() fn > - In assign/unassign RPCs, either update the clock which matches table's > clock type OR all clocks. Updating only meta clock every time looks kind of > weird right now (and this called out explicitly in comments too). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-18457) [HLC] Follow up work of updating clocks (from HBASE-18395)
[ https://issues.apache.org/jira/browse/HBASE-18457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy resolved HBASE-18457. -- Resolution: Duplicate > [HLC] Follow up work of updating clocks (from HBASE-18395) > -- > > Key: HBASE-18457 > URL: https://issues.apache.org/jira/browse/HBASE-18457 > Project: HBase > Issue Type: Sub-task >Reporter: Appy > > So list of things that need to be considered in future > - Mocking clocks in HMaster/RegionServer to test that clocks get updated on > assign/unassign > - Discuss the idea of adding timestamp type to NodeTime proto --> Get rid of > isLikelyOfType() fn > - In assign/unassign RPCs, either update the clock which matches table's > clock type OR all clocks. Updating only meta clock every time looks kind of > weird right now (and this called out explicitly in comments too). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18457) [HLC] Follow up work of updating clocks (from HBASE-18395)
[ https://issues.apache.org/jira/browse/HBASE-18457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126311#comment-16126311 ] Appy commented on HBASE-18457: -- All of the work got done as part of other jiras: - Mocking clocks in HMaster/RegionServer to test that clocks get updated on assign/unassign - HBASE-18395 - Discuss the idea of adding timestamp type to NodeTime proto - HBASE-18497 - In assign/unassign RPCs, either update the clock which matches table's clock type OR all clocks. Updating only meta clock every time looks kind of weird right now (and this called out explicitly in comments too). - HBASE-18497 makes the change to send all clocks with the RPC. Closing this jira as duplicate. > [HLC] Follow up work of updating clocks (from HBASE-18395) > -- > > Key: HBASE-18457 > URL: https://issues.apache.org/jira/browse/HBASE-18457 > Project: HBase > Issue Type: Sub-task >Reporter: Appy > > So list of things that need to be considered in future > - Mocking clocks in HMaster/RegionServer to test that clocks get updated on > assign/unassign > - Discuss the idea of adding timestamp type to NodeTime proto --> Get rid of > isLikelyOfType() fn > - In assign/unassign RPCs, either update the clock which matches table's > clock type OR all clocks. Updating only meta clock every time looks kind of > weird right now (and this called out explicitly in comments too). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18510) Update clock on replaying recovered edits
[ https://issues.apache.org/jira/browse/HBASE-18510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126299#comment-16126299 ] Appy commented on HBASE-18510: -- Pushed to brach HBASE-14070.HLC. Thanks [~amit.patel]. > Update clock on replaying recovered edits > - > > Key: HBASE-18510 > URL: https://issues.apache.org/jira/browse/HBASE-18510 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Fix For: HBASE-14070 > > Attachments: HBASE-18510.HBASE-14070.HLC.001.patch, > HBASE-18510.HBASE-14070.HLC.002.patch > > > This task covers updating the clock on recovery in > HRegion#replayRecoveredEdits and includes region-level tests. > Credit for the baseline of this work all goes to our [~saitejar]. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18539) Remove usages of masterSystemTime
[
https://issues.apache.org/jira/browse/HBASE-18539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126293#comment-16126293
]
Hadoop QA commented on HBASE-18539:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 16m
47s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
15s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
29s{color} | {color:green} HBASE-14070.HLC passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
55s{color} | {color:green} HBASE-14070.HLC passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 14m
56s{color} | {color:green} HBASE-14070.HLC passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
57s{color} | {color:green} HBASE-14070.HLC passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
2s{color} | {color:red} hbase-protocol-shaded in HBASE-14070.HLC has 27 extant
Findbugs warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
3s{color} | {color:red} hbase-server in HBASE-14070.HLC has 10 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} HBASE-14070.HLC passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 14m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 43s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
1m 27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
32s{color} | {color:green} hbase-protocol-shaded in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
19s{color} | {color:green} hbase-protocol in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
55s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}106m 49s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
45s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}219m 28s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests |
org.apache.hadoop.hbase.master.procedure.TestDisableTableProcedure |
| | org.apache.hadoop.hbase.master.procedure.TestDeleteTableProcedure |
| |
[jira] [Assigned] (HBASE-17921) TestBlockReorder always fails against hadoop3.0.0-alpha2
[
https://issues.apache.org/jira/browse/HBASE-17921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Drob reassigned HBASE-17921:
-
Assignee: Mike Drob
> TestBlockReorder always fails against hadoop3.0.0-alpha2
>
>
> Key: HBASE-17921
> URL: https://issues.apache.org/jira/browse/HBASE-17921
> Project: HBase
> Issue Type: Sub-task
> Components: hadoop3
>Reporter: Jonathan Hsieh
>Assignee: Mike Drob
>
> {code}
> java.lang.ExceptionInInitializerError: null
> at
> org.apache.hadoop.hbase.fs.TestBlockReorder.(TestBlockReorder.java:83)
> testBlockLocation(org.apache.hadoop.hbase.fs.TestBlockReorder) Time elapsed:
> 0.023 sec <<< ERROR!
> java.lang.NoClassDefFoundError: Could not initialize class
> org.apache.hadoop.hbase.fs.TestBlockReorder
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.junit.runners.BlockJUnit4ClassRunner.createTest(BlockJUnit4ClassRunner.java:217)
> at
> org.junit.runners.BlockJUnit4ClassRunner$1.runReflectiveCall(BlockJUnit4ClassRunner.java:266)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.BlockJUnit4ClassRunner.methodBlock(BlockJUnit4ClassRunner.java:263)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
> testHBaseCluster(org.apache.hadoop.hbase.fs.TestBlockReorder) Time elapsed:
> 0.011 sec <<< ERROR!
> java.lang.NoClassDefFoundError: Could not initialize class
> org.apache.hadoop.hbase.fs.TestBlockReorder
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.junit.runners.BlockJUnit4ClassRunner.createTest(BlockJUnit4ClassRunner.java:217)
> at
> org.junit.runners.BlockJUnit4ClassRunner$1.runReflectiveCall(BlockJUnit4ClassRunner.java:266)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.BlockJUnit4ClassRunner.methodBlock(BlockJUnit4ClassRunner.java:263)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173)
> at
>
[jira] [Commented] (HBASE-18440) ITs and Actions modify immutable TableDescriptors
[ https://issues.apache.org/jira/browse/HBASE-18440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126280#comment-16126280 ] Mike Drob commented on HBASE-18440: --- Not yet, I got distracted with some other stuff, will hopefully come back to this soon. > ITs and Actions modify immutable TableDescriptors > - > > Key: HBASE-18440 > URL: https://issues.apache.org/jira/browse/HBASE-18440 > Project: HBase > Issue Type: Sub-task > Components: integration tests >Reporter: Mike Drob >Assignee: Mike Drob > Fix For: 2.0.0 > > Attachments: HBASE-18440.patch, HBASE-18440.v2.patch, > HBASE-18440.v3.patch, HBASE-18440.v3.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18303) Clean up some parameterized test declarations
[ https://issues.apache.org/jira/browse/HBASE-18303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated HBASE-18303: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Clean up some parameterized test declarations > - > > Key: HBASE-18303 > URL: https://issues.apache.org/jira/browse/HBASE-18303 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Minor > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18303.patch, HBASE-18303.patch > > > While debugging something unrelated, I noticed that we use the constructor > form of junit parameterized tests, instead of the annotated members form. > I personally find using the @Parameter annotation more clear. > Also, we can move the parameter generator to hbase-common so that it is > accessible in more modules. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18496) Fix TestTimestampType failing on isLikelyOfType tests
[ https://issues.apache.org/jira/browse/HBASE-18496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-18496: - Fix Version/s: HBASE-14070 > Fix TestTimestampType failing on isLikelyOfType tests > - > > Key: HBASE-18496 > URL: https://issues.apache.org/jira/browse/HBASE-18496 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Trivial > Fix For: HBASE-14070 > > Attachments: HBASE-18496.HBASE-14070.HLC.001.patch > > > The tests testPhysicalIsLikelyOfType and testHybridIsLikelyOfType were > failing after the initial commit of core HLC so this patch fixes that. Note > that both Physical#isLikelyOfType and Hybrid#isLikelyOfType will return false > if the input timestamp is from before the first half of the first day of 1970. > Changes > * Offset date by second day of year > * Fixed instance of not properly setting ZonedDateTime in > TestTimestampType#testPhysicalIsLikelyOfType > * Changed timestamp for year 2016 in TimestampType to be UTC+0 instead of > UTC+8 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18498) Design improvements to Clock.java
[ https://issues.apache.org/jira/browse/HBASE-18498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-18498: - Fix Version/s: HBASE-14070 > Design improvements to Clock.java > - > > Key: HBASE-18498 > URL: https://issues.apache.org/jira/browse/HBASE-18498 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Appy > Fix For: HBASE-14070 > > Attachments: HBASE-18498.HBASE-14070.HLC.001.patch > > > - Delete PhysicalClock interface which seems useless give we have SYSTEM > implementation of Clock which is basically the same. > - Embed systemMonotonic clock into HLC to get rid of redundancy in management > of physical component, esp logic around ensuring monotonicity. > - Make max_skew in clocks configurable > - Remove isMonotonicallyIncreasing() which is not used. > - update logical overflow test to not use hooks but prod code path > - Added lots of comments. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18395) Update clock on region open and close
[ https://issues.apache.org/jira/browse/HBASE-18395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-18395: - Fix Version/s: HBASE-14070 > Update clock on region open and close > - > > Key: HBASE-18395 > URL: https://issues.apache.org/jira/browse/HBASE-18395 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Fix For: HBASE-14070 > > Attachments: HBASE-18395.HBASE-14070.HLC.001.patch, > HBASE-18395.HBASE-14070.HLC.002.patch, HBASE-18395.HBASE-14070.HLC.003.patch, > HBASE-18395.HBASE-14070.HLC.004.patch, HBASE-18395.HBASE-14070.HLC.005.patch, > HBASE-18395.HBASE-14070.HLC.006.patch > > > This task covers the patch for updating the clock on region opening and > closing. > The patch would include the following: > * Addition of a new protobuf message type that contains a field for a > timestamp > * Setting of timestamp field in building region open/close request and > response messages > * Updating the clock upon receiving message > The patch for this task will be going against the > [HBASE-14070.HLC|https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=shortlog;h=refs/heads/HBASE-14070.HLC] > branch with the intent that it would be added as a commit. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18510) Update clock on replaying recovered edits
[ https://issues.apache.org/jira/browse/HBASE-18510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-18510: - Fix Version/s: HBASE-14070 > Update clock on replaying recovered edits > - > > Key: HBASE-18510 > URL: https://issues.apache.org/jira/browse/HBASE-18510 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Fix For: HBASE-14070 > > Attachments: HBASE-18510.HBASE-14070.HLC.001.patch, > HBASE-18510.HBASE-14070.HLC.002.patch > > > This task covers updating the clock on recovery in > HRegion#replayRecoveredEdits and includes region-level tests. > Credit for the baseline of this work all goes to our [~saitejar]. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18497) Add clock type to proto message for HLC region open/close
[ https://issues.apache.org/jira/browse/HBASE-18497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-18497: - Fix Version/s: HBASE-14070 > Add clock type to proto message for HLC region open/close > - > > Key: HBASE-18497 > URL: https://issues.apache.org/jira/browse/HBASE-18497 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Fix For: HBASE-14070 > > Attachments: HBASE-18497.HBASE-14070.HLC.001.patch, > HBASE-18497.HBASE-14070.HLC.002.patch, HBASE-18497.HBASE-14070.HLC.003.patch, > HBASE-18497.HBASE-14070.HLC.004.patch > > > This task covers the work for adding clock type to the protobuf message > NodeTime used for sending timestamps of each clock during region open and > close requests/responses. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18497) Add clock type to proto message for HLC region open/close
[ https://issues.apache.org/jira/browse/HBASE-18497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-18497: - Resolution: Fixed Status: Resolved (was: Patch Available) > Add clock type to proto message for HLC region open/close > - > > Key: HBASE-18497 > URL: https://issues.apache.org/jira/browse/HBASE-18497 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Attachments: HBASE-18497.HBASE-14070.HLC.001.patch, > HBASE-18497.HBASE-14070.HLC.002.patch, HBASE-18497.HBASE-14070.HLC.003.patch, > HBASE-18497.HBASE-14070.HLC.004.patch > > > This task covers the work for adding clock type to the protobuf message > NodeTime used for sending timestamps of each clock during region open and > close requests/responses. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18497) Add clock type to proto message for HLC region open/close
[ https://issues.apache.org/jira/browse/HBASE-18497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126275#comment-16126275 ] Appy commented on HBASE-18497: -- Pushed to brach HBASE-14070.HLC. Great work sir [~amit.patel]. :) > Add clock type to proto message for HLC region open/close > - > > Key: HBASE-18497 > URL: https://issues.apache.org/jira/browse/HBASE-18497 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Fix For: HBASE-14070 > > Attachments: HBASE-18497.HBASE-14070.HLC.001.patch, > HBASE-18497.HBASE-14070.HLC.002.patch, HBASE-18497.HBASE-14070.HLC.003.patch, > HBASE-18497.HBASE-14070.HLC.004.patch > > > This task covers the work for adding clock type to the protobuf message > NodeTime used for sending timestamps of each clock during region open and > close requests/responses. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18303) Clean up some parameterized test declarations
[ https://issues.apache.org/jira/browse/HBASE-18303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated HBASE-18303: -- Fix Version/s: (was: 2.0.0) 2.0.0-alpha-2 3.0.0 > Clean up some parameterized test declarations > - > > Key: HBASE-18303 > URL: https://issues.apache.org/jira/browse/HBASE-18303 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Minor > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18303.patch, HBASE-18303.patch > > > While debugging something unrelated, I noticed that we use the constructor > form of junit parameterized tests, instead of the annotated members form. > I personally find using the @Parameter annotation more clear. > Also, we can move the parameter generator to hbase-common so that it is > accessible in more modules. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18510) Update clock on replaying recovered edits
[ https://issues.apache.org/jira/browse/HBASE-18510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Patel updated HBASE-18510: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Update clock on replaying recovered edits > - > > Key: HBASE-18510 > URL: https://issues.apache.org/jira/browse/HBASE-18510 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Attachments: HBASE-18510.HBASE-14070.HLC.001.patch, > HBASE-18510.HBASE-14070.HLC.002.patch > > > This task covers updating the clock on recovery in > HRegion#replayRecoveredEdits and includes region-level tests. > Credit for the baseline of this work all goes to our [~saitejar]. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18587) Fix Flaky TestFileIOEngine
[ https://issues.apache.org/jira/browse/HBASE-18587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126260#comment-16126260 ] Zach York commented on HBASE-18587: --- [~tedyu] I have updated the patch to remove the constant naming for boundaryStartPositions and boundaryStopPositions. > Fix Flaky TestFileIOEngine > -- > > Key: HBASE-18587 > URL: https://issues.apache.org/jira/browse/HBASE-18587 > Project: HBase > Issue Type: Bug > Components: BucketCache, test >Reporter: Zach York >Assignee: Zach York > Attachments: HBASE-18587.branch-1.001.patch, > HBASE-18587.branch-1.002.patch, HBASE-18587.master.001.patch, > HBASE-18587.master.001.patch, HBASE-18587.master.002.patch > > > As a part of HBASE-18533, the Jenkins report said that TestFileIOEngine > failed. I investigated and noticed that there are two cases where this test > is flaky: > len = 0 and offset 0; it will fail for > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/FileIOEngine.java#L198 > will pass in -1 for offset which is invalid. > len = 0 and i = 13; it will try to specify an offset of the totalCapacity > which is not allowed. > > This patch fixes this test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18587) Fix Flaky TestFileIOEngine
[ https://issues.apache.org/jira/browse/HBASE-18587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-18587: -- Attachment: HBASE-18587.branch-1.002.patch > Fix Flaky TestFileIOEngine > -- > > Key: HBASE-18587 > URL: https://issues.apache.org/jira/browse/HBASE-18587 > Project: HBase > Issue Type: Bug > Components: BucketCache, test >Reporter: Zach York >Assignee: Zach York > Attachments: HBASE-18587.branch-1.001.patch, > HBASE-18587.branch-1.002.patch, HBASE-18587.master.001.patch, > HBASE-18587.master.001.patch, HBASE-18587.master.002.patch > > > As a part of HBASE-18533, the Jenkins report said that TestFileIOEngine > failed. I investigated and noticed that there are two cases where this test > is flaky: > len = 0 and offset 0; it will fail for > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/FileIOEngine.java#L198 > will pass in -1 for offset which is invalid. > len = 0 and i = 13; it will try to specify an offset of the totalCapacity > which is not allowed. > > This patch fixes this test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18587) Fix Flaky TestFileIOEngine
[ https://issues.apache.org/jira/browse/HBASE-18587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-18587: -- Attachment: HBASE-18587.master.002.patch > Fix Flaky TestFileIOEngine > -- > > Key: HBASE-18587 > URL: https://issues.apache.org/jira/browse/HBASE-18587 > Project: HBase > Issue Type: Bug > Components: BucketCache, test >Reporter: Zach York >Assignee: Zach York > Attachments: HBASE-18587.branch-1.001.patch, > HBASE-18587.master.001.patch, HBASE-18587.master.001.patch, > HBASE-18587.master.002.patch > > > As a part of HBASE-18533, the Jenkins report said that TestFileIOEngine > failed. I investigated and noticed that there are two cases where this test > is flaky: > len = 0 and offset 0; it will fail for > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/FileIOEngine.java#L198 > will pass in -1 for offset which is invalid. > len = 0 and i = 13; it will try to specify an offset of the totalCapacity > which is not allowed. > > This patch fixes this test. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18238) Address ruby static analysis for bin directory
[ https://issues.apache.org/jira/browse/HBASE-18238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated HBASE-18238: -- Resolution: Fixed Status: Resolved (was: Patch Available) Thanks for review, Stack! > Address ruby static analysis for bin directory > -- > > Key: HBASE-18238 > URL: https://issues.apache.org/jira/browse/HBASE-18238 > Project: HBase > Issue Type: Sub-task >Reporter: Mike Drob >Assignee: Mike Drob > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18238.patch, HBASE-18238.v2.patch, report.txt > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18238) Address ruby static analysis for bin directory
[ https://issues.apache.org/jira/browse/HBASE-18238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated HBASE-18238: -- Fix Version/s: (was: 2.0.0) 2.0.0-alpha-2 3.0.0 > Address ruby static analysis for bin directory > -- > > Key: HBASE-18238 > URL: https://issues.apache.org/jira/browse/HBASE-18238 > Project: HBase > Issue Type: Sub-task >Reporter: Mike Drob >Assignee: Mike Drob > Fix For: 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18238.patch, HBASE-18238.v2.patch, report.txt > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18497) Add clock type to proto message for HLC region open/close
[ https://issues.apache.org/jira/browse/HBASE-18497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126240#comment-16126240 ] stack commented on HBASE-18497: --- Yes [~appy] > Add clock type to proto message for HLC region open/close > - > > Key: HBASE-18497 > URL: https://issues.apache.org/jira/browse/HBASE-18497 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Attachments: HBASE-18497.HBASE-14070.HLC.001.patch, > HBASE-18497.HBASE-14070.HLC.002.patch, HBASE-18497.HBASE-14070.HLC.003.patch, > HBASE-18497.HBASE-14070.HLC.004.patch > > > This task covers the work for adding clock type to the protobuf message > NodeTime used for sending timestamps of each clock during region open and > close requests/responses. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18533) Expose BucketCache values to be configured
[ https://issues.apache.org/jira/browse/HBASE-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126239#comment-16126239 ] Zach York commented on HBASE-18533: --- [~tedyu] Any more issues with this patch? The test run is clean (with the exception of HBASE-18587) > Expose BucketCache values to be configured > -- > > Key: HBASE-18533 > URL: https://issues.apache.org/jira/browse/HBASE-18533 > Project: HBase > Issue Type: Improvement > Components: BucketCache >Reporter: Zach York >Assignee: Zach York > Fix For: 2.0.0, 1.4.0, 1.5.0 > > Attachments: HBASE-18533.branch-1.001.patch, > HBASE-18533.branch-1.002.patch, HBASE-18533.branch-1.003.patch, > HBASE-18533.master.001.patch, HBASE-18533.master.002.patch, > HBASE-18533.master.003.patch > > > BucketCache always uses the default values for all cache configuration. > However, this doesn't work for all use cases. In particular, users want to be > able to configure the percentage of the cache that is single access, multi > access, and in-memory access. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18591) Upgrade thrift from 0.9.3 to 0.10.0
[ https://issues.apache.org/jira/browse/HBASE-18591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126227#comment-16126227 ] stack commented on HBASE-18591: --- [~dimaspivak] happybase seems pegged on 0.9.3. Not breaking happybase would be good reason to stay on 0.9.3 I'm thinking. This update is not needed for jetty. It is a standalone item. I am here just because trying to close out general directive that we upgrade everything before hbase2 release. Thanks boss. > Upgrade thrift from 0.9.3 to 0.10.0 > --- > > Key: HBASE-18591 > URL: https://issues.apache.org/jira/browse/HBASE-18591 > Project: HBase > Issue Type: Bug > Components: Thrift >Reporter: stack >Assignee: stack > Fix For: 2.0.0 > > Attachments: 18591.txt > > > Maybe we don't want to do this for 2.0.0. Maybe we want to let it up to the > implementor. Going to 0.10.0 will probably break any 0.9.3 clients. I could > try it I suppose but if past experience is anything to go by...it'll break > (There is this note that it is broke up on happybase > https://github.com/wbolster/happybase/issues/154). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18224) Upgrade jetty
[ https://issues.apache.org/jira/browse/HBASE-18224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126221#comment-16126221 ] stack commented on HBASE-18224: --- Made this standalone, non-critical issue for 2.0. Lets come back here when we need the upgrade. Can also shade it if needs be, adding it to hbase-thirdparty. > Upgrade jetty > - > > Key: HBASE-18224 > URL: https://issues.apache.org/jira/browse/HBASE-18224 > Project: HBase > Issue Type: Bug >Reporter: Balazs Meszaros > Fix For: 2.0.0 > > Attachments: HBASE-18224.branch-2.001.patch > > > Jetty can be updated to 9.4.6 and thrift can be updated to 0.10.0. I tried to > update them in HBASE-17898 but some unit tests failed, so created a sub-task > for them. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18224) Upgrade jetty
[ https://issues.apache.org/jira/browse/HBASE-18224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18224: -- Issue Type: Bug (was: Sub-task) Parent: (was: HBASE-17898) > Upgrade jetty > - > > Key: HBASE-18224 > URL: https://issues.apache.org/jira/browse/HBASE-18224 > Project: HBase > Issue Type: Bug >Reporter: Balazs Meszaros > Fix For: 2.0.0 > > Attachments: HBASE-18224.branch-2.001.patch > > > Jetty can be updated to 9.4.6 and thrift can be updated to 0.10.0. I tried to > update them in HBASE-17898 but some unit tests failed, so created a sub-task > for them. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18224) Upgrade jetty
[ https://issues.apache.org/jira/browse/HBASE-18224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126219#comment-16126219 ] stack commented on HBASE-18224: --- It fails the hadoop check because of this [INFO] - [ERROR] COMPILATION ERROR : [INFO] - [ERROR] /testptch/hbase/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/metrics/impl/GlobalMetricRegistriesAdapter.java:[160,42] cannot find symbol symbol: method unregisterSource(java.lang.String) location: class org.apache.hadoop.metrics2.MetricsSystem Thats probably ok for a hbase2; i.e. not working w/ hadoop 2.5.2 or earlier. The unit tests are complaining: java.io.IOException: Problem starting http server at org.apache.hadoop.hbase.http.TestSSLHttpServer.setup(TestSSLHttpServer.java:94) Caused by: java.lang.IllegalStateException: Insufficient threads: max=10 < needed(acceptors=2 + selectors=8 + request=1) at org.apache.hadoop.hbase.http.TestSSLHttpServer.setup(TestSSLHttpServer.java:94) .. which is probably a symptom of something else. Let me put this aside until we need it. Going to make it a non-critical issue for hbase2. > Upgrade jetty > - > > Key: HBASE-18224 > URL: https://issues.apache.org/jira/browse/HBASE-18224 > Project: HBase > Issue Type: Sub-task >Reporter: Balazs Meszaros > Fix For: 2.0.0 > > Attachments: HBASE-18224.branch-2.001.patch > > > Jetty can be updated to 9.4.6 and thrift can be updated to 0.10.0. I tried to > update them in HBASE-17898 but some unit tests failed, so created a sub-task > for them. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18522) Add RowMutations support to Batch
[ https://issues.apache.org/jira/browse/HBASE-18522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126209#comment-16126209 ] Hudson commented on HBASE-18522: FAILURE: Integrated in Jenkins build HBase-1.5 #2 (See [https://builds.apache.org/job/HBase-1.5/2/]) HBASE-18522 Add RowMutations support to Batch (jerryjch: rev 9078a034c410d53800e656d6a19f810c30fc102f) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Table.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/MultiServerCallable.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java > Add RowMutations support to Batch > - > > Key: HBASE-18522 > URL: https://issues.apache.org/jira/browse/HBASE-18522 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.2.6 >Reporter: Jerry He >Assignee: Jerry He > Fix For: 2.0.0, 3.0.0, 1.4.0, 1.5.0 > > Attachments: HBASE-18522-branch-1.patch, > HBASE-18522-branch-1-v2.patch, HBASE-18522-master-v2.patch > > > RowMutations is multiple Puts and/or Deletes atomically on a single row. > Current Batch call does not support RowMutations as part of the batch. > We should add this missing part. We should be able to batch RowMutations. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18224) Upgrade jetty
[
https://issues.apache.org/jira/browse/HBASE-18224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126206#comment-16126206
]
Hadoop QA commented on HBASE-18224:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
37s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
44s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
9s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
1s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m
0s{color} | {color:red} The patch causes 16 errors with Hadoop v2.4.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m
58s{color} | {color:red} The patch causes 16 errors with Hadoop v2.4.1. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m
57s{color} | {color:red} The patch causes 16 errors with Hadoop v2.5.0. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m
57s{color} | {color:red} The patch causes 16 errors with Hadoop v2.5.1. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m
55s{color} | {color:red} The patch causes 16 errors with Hadoop v2.5.2. {color}
|
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
57s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 37m 15s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}105m 48s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.http.TestHttpServer |
| | hadoop.hbase.http.TestSpnegoHttpServer |
| | hadoop.hbase.http.TestSSLHttpServer |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:3c8b364 |
| JIRA Issue | HBASE-18224 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12881662/HBASE-18224.branch-2.001.patch
|
| Optional Tests | asflicense javac javadoc unit xml compile |
| uname | Linux f07851d7e86c 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | branch-2 / c20ce21 |
| Default Java | 1.8.0_144 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/8080/artifact/patchprocess/patch-unit-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/8080/testReport/ |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/8080/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Upgrade jetty
> -
>
> Key: HBASE-18224
> URL:
[jira] [Updated] (HBASE-18503) Change ***Util and Master to use TableDescriptor and ColumnFamilyDescriptor
[ https://issues.apache.org/jira/browse/HBASE-18503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated HBASE-18503: --- Fix Version/s: 2.0.0 > Change ***Util and Master to use TableDescriptor and ColumnFamilyDescriptor > --- > > Key: HBASE-18503 > URL: https://issues.apache.org/jira/browse/HBASE-18503 > Project: HBase > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 2.0.0 > > > These helper classes accept the HTD and HCD as argument. We need to make some > changes for them, otherwise we will be forced to use HTD and HCD. > # SecureTestUtil > # MobSnapshotTestingUtils -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18542) [HLC] Performance microbenchmarks
[ https://issues.apache.org/jira/browse/HBASE-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-18542: - Description: Need tests to benchmark performance of Clock#now() and update() functions (for all types of clocks). If update() is too costly, we can do optimizations in ExecuteProceduresRemoteCall#call(), HRegion#replayRecoveredEdits() and other places where we call update() in loop. Instead, it might be faster to calculate max timestamp in loop and call update() just once. was: Need tests to benchmark performance of Clock#now() and update() functions (for all types of clocks). If update() is too costly, we can do optimizations in ExecuteProceduresRemoteCall#call() and other places where we call update() in loop. Instead, it might be faster to calculate max timestamp in loop and call update() just once. > [HLC] Performance microbenchmarks > - > > Key: HBASE-18542 > URL: https://issues.apache.org/jira/browse/HBASE-18542 > Project: HBase > Issue Type: Sub-task >Reporter: Appy > > Need tests to benchmark performance of Clock#now() and update() functions > (for all types of clocks). > If update() is too costly, we can do optimizations in > ExecuteProceduresRemoteCall#call(), HRegion#replayRecoveredEdits() and other > places where we call update() in loop. Instead, it might be faster to > calculate max timestamp in loop and call update() just once. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-16338) update jackson to 2.y
[ https://issues.apache.org/jira/browse/HBASE-16338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126166#comment-16126166 ] Andrew Purtell commented on HBASE-16338: I think maybe consistent ordering of fields in JSON serialization? > update jackson to 2.y > - > > Key: HBASE-16338 > URL: https://issues.apache.org/jira/browse/HBASE-16338 > Project: HBase > Issue Type: Task > Components: dependencies >Reporter: Sean Busbey >Assignee: stack >Priority: Minor > Fix For: 2.0.0 > > Attachments: 16338.txt > > > Our jackson dependency is from ~3 years ago. Update to the jackson 2.y line, > using 2.7.0+. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18497) Add clock type to proto message for HLC region open/close
[ https://issues.apache.org/jira/browse/HBASE-18497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126137#comment-16126137 ] Appy edited comment on HBASE-18497 at 8/14/17 6:20 PM: --- It's v4. I uploaded it here few days ago, but RB was updated today because only amit can update diffs there. The last QA run is for v4 and it doesn't show Test*Clock tests, so i'll commit this one. To fix the timeout failures in this branch, there's HBASE-18508. sg [~stack]? was (Author: appy): It's v4. I uploaded it here few days ago, but RB was updated today coz only amit can update diffs there. Strange though that there was no run. Let me trigger one manually. > Add clock type to proto message for HLC region open/close > - > > Key: HBASE-18497 > URL: https://issues.apache.org/jira/browse/HBASE-18497 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Attachments: HBASE-18497.HBASE-14070.HLC.001.patch, > HBASE-18497.HBASE-14070.HLC.002.patch, HBASE-18497.HBASE-14070.HLC.003.patch, > HBASE-18497.HBASE-14070.HLC.004.patch > > > This task covers the work for adding clock type to the protobuf message > NodeTime used for sending timestamps of each clock during region open and > close requests/responses. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18497) Add clock type to proto message for HLC region open/close
[ https://issues.apache.org/jira/browse/HBASE-18497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126137#comment-16126137 ] Appy commented on HBASE-18497: -- It's v4. I uploaded it here few days ago, but RB was updated today coz only amit can update diffs there. Strange though that there was no run. Let me trigger one manually. > Add clock type to proto message for HLC region open/close > - > > Key: HBASE-18497 > URL: https://issues.apache.org/jira/browse/HBASE-18497 > Project: HBase > Issue Type: Sub-task >Reporter: Amit Patel >Assignee: Amit Patel >Priority: Minor > Attachments: HBASE-18497.HBASE-14070.HLC.001.patch, > HBASE-18497.HBASE-14070.HLC.002.patch, HBASE-18497.HBASE-14070.HLC.003.patch, > HBASE-18497.HBASE-14070.HLC.004.patch > > > This task covers the work for adding clock type to the protobuf message > NodeTime used for sending timestamps of each clock during region open and > close requests/responses. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18597) ExportSnapshot tool closes FileSystems objects obtained from the cache
[
https://issues.apache.org/jira/browse/HBASE-18597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ben Borchard updated HBASE-18597:
-
Summary: ExportSnapshot tool closes FileSystems objects obtained from the
cache (was: ExportSnapshot tool is closing a FileSystem object obtained from
the cache)
> ExportSnapshot tool closes FileSystems objects obtained from the cache
> --
>
> Key: HBASE-18597
> URL: https://issues.apache.org/jira/browse/HBASE-18597
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Reporter: Ben Borchard
> Labels: snapshot, snapshots
> Original Estimate: 0.25h
> Remaining Estimate: 0.25h
>
> In the ExportSnapshot tool the input and output FileSystem objects are being
> obtained using the method _FileSystem.get_ which will either create a new
> FileSystem object or simply take an existing object out of the cache if a
> suitable one exists.
> After it is done with the these FileSystem objects it will close them.
> The issue here is that the ExportSnapshot tool will potentially remove
> preexisting FileSystem objects from the cache by closing said objects. This
> will break any apis running in the same process that are relying on these
> cached FileSystem objects.
> A simple solution could, I believe, fix this problem without affecting the
> functionality of the ExportSnapshot tool:
> Change ExportSnapshot.java lines 943:
> {{FileSystem inputFs = FileSystem.get(inputRoot.toUri(), srcConf);}}
> and 947:
> {{FileSystem outputFs = FileSystem.get(outputRoot.toUri(), destConf);}}
> to use _FileSystem.newInstance_ instead of _FileSystem.get_:
> 943
> {{FileSystem inputFs = FileSystem.newInstance(inputRoot.toUri(), srcConf);}}
> 947
> {{FileSystem outputFs = FileSystem.newInstance(outputRoot.toUri(),
> destConf);}}
> This will create a unique entry in the cache and in this way prevent the
> closure of these FileSystem objects from wiping out any preexisting
> FileSystem objects. It will also ensure that no unused FileSystem objects
> created by the ExportSnapshot tool will be left taking up heap space and
> potentially causing memory issues.
> I am happy to submit a fix for this, but figured I would open an issue first
> so the issue can be properly discussed and tracked.
> Also note this is the first issue I have opened so I apologize in advance for
> any standard procedures and/or best practices I haven't followed.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18597) ExportSnapshot tool closes FileSystem objects obtained from the cache
[
https://issues.apache.org/jira/browse/HBASE-18597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ben Borchard updated HBASE-18597:
-
Summary: ExportSnapshot tool closes FileSystem objects obtained from the
cache (was: ExportSnapshot tool closes FileSystems objects obtained from the
cache)
> ExportSnapshot tool closes FileSystem objects obtained from the cache
> -
>
> Key: HBASE-18597
> URL: https://issues.apache.org/jira/browse/HBASE-18597
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Reporter: Ben Borchard
> Labels: snapshot, snapshots
> Original Estimate: 0.25h
> Remaining Estimate: 0.25h
>
> In the ExportSnapshot tool the input and output FileSystem objects are being
> obtained using the method _FileSystem.get_ which will either create a new
> FileSystem object or simply take an existing object out of the cache if a
> suitable one exists.
> After it is done with the these FileSystem objects it will close them.
> The issue here is that the ExportSnapshot tool will potentially remove
> preexisting FileSystem objects from the cache by closing said objects. This
> will break any apis running in the same process that are relying on these
> cached FileSystem objects.
> A simple solution could, I believe, fix this problem without affecting the
> functionality of the ExportSnapshot tool:
> Change ExportSnapshot.java lines 943:
> {{FileSystem inputFs = FileSystem.get(inputRoot.toUri(), srcConf);}}
> and 947:
> {{FileSystem outputFs = FileSystem.get(outputRoot.toUri(), destConf);}}
> to use _FileSystem.newInstance_ instead of _FileSystem.get_:
> 943
> {{FileSystem inputFs = FileSystem.newInstance(inputRoot.toUri(), srcConf);}}
> 947
> {{FileSystem outputFs = FileSystem.newInstance(outputRoot.toUri(),
> destConf);}}
> This will create a unique entry in the cache and in this way prevent the
> closure of these FileSystem objects from wiping out any preexisting
> FileSystem objects. It will also ensure that no unused FileSystem objects
> created by the ExportSnapshot tool will be left taking up heap space and
> potentially causing memory issues.
> I am happy to submit a fix for this, but figured I would open an issue first
> so the issue can be properly discussed and tracked.
> Also note this is the first issue I have opened so I apologize in advance for
> any standard procedures and/or best practices I haven't followed.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-16338) update jackson to 2.y
[ https://issues.apache.org/jira/browse/HBASE-16338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126134#comment-16126134 ] stack commented on HBASE-16338: --- [~apurtell] Any chance you remember what the 'feature' provided.. something to do w/ JSON serialization. > update jackson to 2.y > - > > Key: HBASE-16338 > URL: https://issues.apache.org/jira/browse/HBASE-16338 > Project: HBase > Issue Type: Task > Components: dependencies >Reporter: Sean Busbey >Assignee: stack >Priority: Minor > Fix For: 2.0.0 > > Attachments: 16338.txt > > > Our jackson dependency is from ~3 years ago. Update to the jackson 2.y line, > using 2.7.0+. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-15676) FuzzyRowFilter fails and matches all the rows in the table if the mask consists of all 0s
[
https://issues.apache.org/jira/browse/HBASE-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126129#comment-16126129
]
Biju Nair commented on HBASE-15676:
---
Thanks [~elserj] .
> FuzzyRowFilter fails and matches all the rows in the table if the mask
> consists of all 0s
> -
>
> Key: HBASE-15676
> URL: https://issues.apache.org/jira/browse/HBASE-15676
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 2.0.0, 0.98.13, 1.0.2, 1.2.0, 1.1.1
>Reporter: Rohit Sinha
>Assignee: Matt Warhaftig
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.5, 1.2.2, 0.98.20
>
> Attachments: hbase-15287-0.98-v1.patch, hbase-15676-v1.patch,
> hbase-15676-v2.patch, hbase-15676-v3.patch, hbase-15676-v4.patch
>
>
> While using FuzzyRowFilter we noticed that if the mask array consists of all
> 0s (fixed) the FuzzyRowFilter matches all the rows in the table. We noticed
> this on HBase 1.1, 1.2 and higher.
> After some digging we suspect that this is because of isPreprocessedMask()
> check which is used in preprocessMask() which was added here:
> https://issues.apache.org/jira/browse/HBASE-13761
> If the mask consists of all 0s then the isPreprocessedMask() returns true and
> the preprocessing which responsible for changing 0s to -1 doesn't happen and
> hence all rows are matched in scan.
> This scenario can be tested in TestFuzzyRowFilterEndToEnd#testHBASE14782() If
> we change the
> byte[] fuzzyKey = Bytes.toBytesBinary("\\x00\\x00\\x044");
> byte[] mask = new byte[] {1,0,0,0};
> to
> byte[] fuzzyKey = Bytes.toBytesBinary("\\x9B\\x00\\x044e");
> byte[] mask = new byte[] {0,0,0,0,0};
> We expect one match but this will match all the rows in the table.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-18597) ExportSnapshot tool is closing a FileSystem object obtained from the cache
Ben Borchard created HBASE-18597:
Summary: ExportSnapshot tool is closing a FileSystem object
obtained from the cache
Key: HBASE-18597
URL: https://issues.apache.org/jira/browse/HBASE-18597
Project: HBase
Issue Type: Bug
Components: snapshots
Reporter: Ben Borchard
In the ExportSnapshot tool the input and output FileSystem objects are being
obtained using the method _FileSystem.get_ which will either create a new
FileSystem object or simply take an existing object out of the cache if a
suitable one exists.
After it is done with the these FileSystem objects it will close them.
The issue here is that the ExportSnapshot tool will potentially remove
preexisting FileSystem objects from the cache by closing said objects. This
will break any apis running in the same process that are relying on these
cached FileSystem objects.
A simple solution could, I believe, fix this problem without affecting the
functionality of the ExportSnapshot tool:
Change ExportSnapshot.java lines 943:
{{FileSystem inputFs = FileSystem.get(inputRoot.toUri(), srcConf);}}
and 947:
{{FileSystem outputFs = FileSystem.get(outputRoot.toUri(), destConf);}}
to use _FileSystem.newInstance_ instead of _FileSystem.get_:
943
{{FileSystem inputFs = FileSystem.newInstance(inputRoot.toUri(), srcConf);}}
947
{{FileSystem outputFs = FileSystem.newInstance(outputRoot.toUri(), destConf);}}
This will create a unique entry in the cache and in this way prevent the
closure of these FileSystem objects from wiping out any preexisting FileSystem
objects. It will also ensure that no unused FileSystem objects created by the
ExportSnapshot tool will be left taking up heap space and potentially causing
memory issues.
I am happy to submit a fix for this, but figured I would open an issue first so
the issue can be properly discussed and tracked.
Also note this is the first issue I have opened so I apologize in advance for
any standard procedures and/or best practices I haven't followed.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-16338) update jackson to 2.y
[ https://issues.apache.org/jira/browse/HBASE-16338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126121#comment-16126121 ] Andrew Purtell commented on HBASE-16338: Sure, that's fine. Whatever we need > update jackson to 2.y > - > > Key: HBASE-16338 > URL: https://issues.apache.org/jira/browse/HBASE-16338 > Project: HBase > Issue Type: Task > Components: dependencies >Reporter: Sean Busbey >Assignee: stack >Priority: Minor > Fix For: 2.0.0 > > Attachments: 16338.txt > > > Our jackson dependency is from ~3 years ago. Update to the jackson 2.y line, > using 2.7.0+. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
