[jira] [Created] (HBASE-21208) Bytes#toShort doesn't work without unsafe
Chia-Ping Tsai created HBASE-21208:
--
Summary: Bytes#toShort doesn't work without unsafe
Key: HBASE-21208
URL: https://issues.apache.org/jira/browse/HBASE-21208
Project: HBase
Issue Type: Bug
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai
Fix For: 2.2.0, 2.1.1, 2.0.3
seems we put the brackets in the wrong place :)
{code}
short n = 0;
n = (short) ((n ^ bytes[offset]) & 0xFF);
n = (short) (n << 8);
n = (short) ((n ^ bytes[offset+1]) & 0xFF); // this one
return n;
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21013) Backport "read part" of HBASE-18754 to all active 1.x branches
Chia-Ping Tsai created HBASE-21013: -- Summary: Backport "read part" of HBASE-18754 to all active 1.x branches Key: HBASE-21013 URL: https://issues.apache.org/jira/browse/HBASE-21013 Project: HBase Issue Type: Sub-task Reporter: Chia-Ping Tsai Fix For: 1.5.0, 1.2.7, 1.3.3, 1.4.7 The hfiles impacted by HBASE-18754 will have bytes of proto.TimeRangeTracker. It makes all 1.x branches failed to read the hfile since all 1.x branches can't deserialize the proto.TimeRangeTracker. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21012) Revert the use of proto.TimeRangeTracker when writing hfile
Chia-Ping Tsai created HBASE-21012: -- Summary: Revert the use of proto.TimeRangeTracker when writing hfile Key: HBASE-21012 URL: https://issues.apache.org/jira/browse/HBASE-21012 Project: HBase Issue Type: Sub-task Reporter: Chia-Ping Tsai HBASE-18754 change the serialization of TimeRangeTracker from "manual way" to protobuf. However, the change breaks the backward compatibility of hfile. We should revert the change ASAP. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20807) Complete the test and document for all public tools
Chia-Ping Tsai created HBASE-20807: -- Summary: Complete the test and document for all public tools Key: HBASE-20807 URL: https://issues.apache.org/jira/browse/HBASE-20807 Project: HBase Issue Type: Umbrella Components: tooling Reporter: Chia-Ping Tsai There are many tools having no tests and document. That may make the tools unstable and stale as time goes by. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20623) Introduce the getCellBuilder to Mutation
Chia-Ping Tsai created HBASE-20623:
--
Summary: Introduce the getCellBuilder to Mutation
Key: HBASE-20623
URL: https://issues.apache.org/jira/browse/HBASE-20623
Project: HBase
Issue Type: Task
Reporter: Chia-Ping Tsai
Fix For: 3.0.0, 2.1.0
see
[https://lists.apache.org/thread.html/d05bfaa0134502a47f6e1aca56cb0b096d4dd32ddefbbdf28db4952a@%3Cdev.hbase.apache.org%3E]
for more details.
{code:java}
How about a "getCellBuilder" or "getCellBuilderFactory" method for
Mutation implementations that gives you a CellBuilder instance that
already has relevant parts set? Like for a Put instance it should be
able to already have the Type and Row set.{code}
mentioned a day or so ago by [~busbey]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-20622) Revise the default IA rule of nested class - the default IA is same with their enclosing class
Chia-Ping Tsai created HBASE-20622:
--
Summary: Revise the default IA rule of nested class - the default
IA is same with their enclosing class
Key: HBASE-20622
URL: https://issues.apache.org/jira/browse/HBASE-20622
Project: HBase
Issue Type: Task
Reporter: Chia-Ping Tsai
see
[https://lists.apache.org/thread.html/d05bfaa0134502a47f6e1aca56cb0b096d4dd32ddefbbdf28db4952a@%3Cdev.hbase.apache.org%3E]
for more discussion.
The related description is in
[https://hbase.apache.org/book.html#hbase.client.api.surface.]
{code:java}
If a class is not annotated with one of these, it is assumed to be a
InterfaceAudience.Private class.{code}
We should add another rule for nested class - "nested classes take on the IA of
their enclosing class."
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-20575) Fail to config COMPACTION_ENABLED by hbase shell
Chia-Ping Tsai created HBASE-20575:
--
Summary: Fail to config COMPACTION_ENABLED by hbase shell
Key: HBASE-20575
URL: https://issues.apache.org/jira/browse/HBASE-20575
Project: HBase
Issue Type: Bug
Components: shell
Affects Versions: 1.3.2
Reporter: Chia-Ping Tsai
Fix For: 1.2.7, 1.3.3
HBASE-19340 backported the missing option from 1.4+ to 1.3 and 1.2. However, we
made a mistaken to COMPACTION_ENABLED.
{code:java}
htd.setCompactionEnabled(JBoolean.valueOf(arg.delete[COMPACTION_ENABLED])) if
arg[COMPACTION_ENABLED]{code}
arg.delete[COMPACTION_ENABLED] -> arg.delete(COMPACTION_ENABLED)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-20485) Copy constructor of Scan doesn't copy the readType
Chia-Ping Tsai created HBASE-20485: -- Summary: Copy constructor of Scan doesn't copy the readType Key: HBASE-20485 URL: https://issues.apache.org/jira/browse/HBASE-20485 Project: HBase Issue Type: Bug Reporter: Chia-Ping Tsai The field was introduced by HBASE-17045. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20484) Remove the unnecessary autoboxing in FilterListBase
Chia-Ping Tsai created HBASE-20484:
--
Summary: Remove the unnecessary autoboxing in FilterListBase
Key: HBASE-20484
URL: https://issues.apache.org/jira/browse/HBASE-20484
Project: HBase
Issue Type: Bug
Reporter: Chia-Ping Tsai
{code:java}
protected static boolean checkAndGetReversed(List rowFilters, boolean
defaultValue) {
if (rowFilters.isEmpty()) {
return defaultValue;
}
Boolean retValue = rowFilters.get(0).isReversed(); // this one
boolean allEqual =
rowFilters.stream().map(Filter::isReversed).allMatch(retValue::equals);
if (!allEqual) {
throw new IllegalArgumentException("Filters in the list must have the same
reversed flag");
}
return retValue;
}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-20301) Remove the meaningless plus sign from table.jsp
Chia-Ping Tsai created HBASE-20301: -- Summary: Remove the meaningless plus sign from table.jsp Key: HBASE-20301 URL: https://issues.apache.org/jira/browse/HBASE-20301 Project: HBase Issue Type: Bug Components: UI Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai Fix For: 1.5.0, 1.4.4 see the screenshot -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20300) Minor refactor for RpcExecutor
Chia-Ping Tsai created HBASE-20300: -- Summary: Minor refactor for RpcExecutor Key: HBASE-20300 URL: https://issues.apache.org/jira/browse/HBASE-20300 Project: HBase Issue Type: Task Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai Plan to do the following changes. # make Handler be static class # move the threadlocal variables of MonitoredRPCHandler from RpcServer to FifoRpcScheduler since only FifoRpcScheduler use it # create MonitoredRPCHandler in Handler constuction instead of saving the MonitoredRPCHandler in threadlocal variables. In FPBQ mode, the web UI can display all Handlers info even if the rpc Handlers are not used yet. # Threshhold -> Threshold # make RpcExecutor extend ConfigurationObserver # don't create task filter repeatly # add a ut to check whether each Handler has created own MonitoredTask even if no ops -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-19552) update hbase to use new thirdparty libs
[ https://issues.apache.org/jira/browse/HBASE-19552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved HBASE-19552. Resolution: Fixed > update hbase to use new thirdparty libs > --- > > Key: HBASE-19552 > URL: https://issues.apache.org/jira/browse/HBASE-19552 > Project: HBase > Issue Type: Task > Components: dependencies, thirdparty >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Major > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19552.ADDENDUM.patch, > HBASE-19552.branch-2.addendum.patch.patch, HBASE-19552.patch, > HBASE-19552.v2.patch, HBASE-19552.v3.patch, HBASE-19552.v4.BASE.patch, > HBASE-19552.v4.FULL.patch > > > When we release hbase-thirdparty next release, we will need to update our > code for the packaging change. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20246) Remove the spark module
Chia-Ping Tsai created HBASE-20246: -- Summary: Remove the spark module Key: HBASE-20246 URL: https://issues.apache.org/jira/browse/HBASE-20246 Project: HBase Issue Type: Task Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai Fix For: 2.1.0, 2.0.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (HBASE-19552) update hbase to use new thirdparty libs
[ https://issues.apache.org/jira/browse/HBASE-19552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai reopened HBASE-19552: reopen since the spark module was added back to branch-2 > update hbase to use new thirdparty libs > --- > > Key: HBASE-19552 > URL: https://issues.apache.org/jira/browse/HBASE-19552 > Project: HBase > Issue Type: Task > Components: dependencies, thirdparty >Reporter: Mike Drob >Assignee: Mike Drob >Priority: Major > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19552.ADDENDUM.patch, HBASE-19552.patch, > HBASE-19552.v2.patch, HBASE-19552.v3.patch, HBASE-19552.v4.BASE.patch, > HBASE-19552.v4.FULL.patch > > > When we release hbase-thirdparty next release, we will need to update our > code for the packaging change. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-20155) update branch-2 version to 2.1.0-SNAPSHOT
[ https://issues.apache.org/jira/browse/HBASE-20155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved HBASE-20155. Resolution: Fixed > update branch-2 version to 2.1.0-SNAPSHOT > - > > Key: HBASE-20155 > URL: https://issues.apache.org/jira/browse/HBASE-20155 > Project: HBase > Issue Type: Sub-task > Components: build, community >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 2.1.0 > > Attachments: HBASE-20155-branch-2.v0.patch, > HBASE-20155.branch-2.addendum.patch > > > now that we have a branch-2.0, branch-2 should be set to the next minor > release version number -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (HBASE-20155) update branch-2 version to 2.1.0-SNAPSHOT
[ https://issues.apache.org/jira/browse/HBASE-20155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai reopened HBASE-20155: reopen since the spark module isn't updated > update branch-2 version to 2.1.0-SNAPSHOT > - > > Key: HBASE-20155 > URL: https://issues.apache.org/jira/browse/HBASE-20155 > Project: HBase > Issue Type: Sub-task > Components: build, community >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 2.1.0 > > Attachments: HBASE-20155-branch-2.v0.patch > > > now that we have a branch-2.0, branch-2 should be set to the next minor > release version number -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (HBASE-20119) Introduce a pojo class to carry coprocessor information in order to make TableDescriptorBuilder accept multiple cp at once
[
https://issues.apache.org/jira/browse/HBASE-20119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai reopened HBASE-20119:
reopen to revert the changes in Public classes
> Introduce a pojo class to carry coprocessor information in order to make
> TableDescriptorBuilder accept multiple cp at once
> --
>
> Key: HBASE-20119
> URL: https://issues.apache.org/jira/browse/HBASE-20119
> Project: HBase
> Issue Type: Task
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Minor
> Fix For: 2.0.0, 3.0.0, 2.1.0
>
> Attachments: HBASE-20119.branch-2.v0.patch,
> HBASE-20119.v0.patch.patch, HBASE-20119.v1.patch.patch, HBASE-20119.v2.patch,
> HBASE-20119.v3.patch
>
>
> The way to add cp to TableDescriptorBuilder is shown below.
> {code:java}
> public TableDescriptorBuilder addCoprocessor(String className) throws
> IOException {
> return addCoprocessor(className, null, Coprocessor.PRIORITY_USER, null);
> }
> public TableDescriptorBuilder addCoprocessor(String className, Path
> jarFilePath,
> int priority, final Map kvs) throws IOException {
> desc.addCoprocessor(className, jarFilePath, priority, kvs);
> return this;
> }
> public TableDescriptorBuilder addCoprocessorWithSpec(final String specStr)
> throws IOException {
> desc.addCoprocessorWithSpec(specStr);
> return this;
> }{code}
> When loading our config to create table with multiple cps, we have to write
> the ugly for-loop.
> {code:java}
> val builder = TableDescriptorBuilder.newBuilde(tableName)
> .setAAA()
> .setBBB()
> cps.map(toHBaseCp).foreach(builder.addCoprocessor)
> cfs.map(toHBaseCf).foreach(builder.addColumnFamily)
> admin.createTable(builder.build())
> {code}
> If we introduce a pojo to carry the cp data and add the method accepting
> multiple cps and cfs, it is easier to exercise the fluent interface of
> TableDescriptorBuilder.
> {code:java}
> admin.createTable(TableDescriptorBuilder.newBuilde(tableName)
> .addCoprocessor(cps.map(toHBaseCp).asJavaCollection)
> .addColumnFamily(cf.map(toHBaseCf).asJavaCollection)
> .setAAA()
> .setBBB()
> .build){code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-19713) Enable TestInterfaceAudienceAnnotations
[ https://issues.apache.org/jira/browse/HBASE-19713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved HBASE-19713. Resolution: Won't Fix The PR enabling filter only Public class have been merged to warbucks-maven-plugin. Close this Jira as "won't fix". The follow-up is HBASE-20121 > Enable TestInterfaceAudienceAnnotations > --- > > Key: HBASE-19713 > URL: https://issues.apache.org/jira/browse/HBASE-19713 > Project: HBase > Issue Type: Bug >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-19713.branch-2.v0.patch, > HBASE-19713.branch-2.v1.patch, HBASE-19713.branch-2.v2.patch, > HBASE-19713.v0.patch > > > Make sure TestInterfaceAudienceAnnotations pass before 2.0 release. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20212) Make all Public classes have InterfaceAudience category
Chia-Ping Tsai created HBASE-20212: -- Summary: Make all Public classes have InterfaceAudience category Key: HBASE-20212 URL: https://issues.apache.org/jira/browse/HBASE-20212 Project: HBase Issue Type: Task Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai Fix For: 2.0.0, 3.0.0, 2.1.0 The tasks will be resolved are shown below. # add warbucks-maven-plugin to root pom # make sure all sub modules ref the warbucks-maven-plugin # remove old checker (TestInterfaceAudienceAnnotations) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20171) Remove o.a.h.h.ProcedureState
Chia-Ping Tsai created HBASE-20171: -- Summary: Remove o.a.h.h.ProcedureState Key: HBASE-20171 URL: https://issues.apache.org/jira/browse/HBASE-20171 Project: HBase Issue Type: Task Environment: It was introduced by HBASE-15609, and HBASE-18106 make it be a orphan Reporter: Chia-Ping Tsai Fix For: 3.0.0, 2.1.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20132) Change the "KV" to "Cell" for web UI
Chia-Ping Tsai created HBASE-20132: -- Summary: Change the "KV" to "Cell" for web UI Key: HBASE-20132 URL: https://issues.apache.org/jira/browse/HBASE-20132 Project: HBase Issue Type: Task Reporter: Chia-Ping Tsai Fix For: 2.0.0 grep the source code. The related words which should be revised are shown below. # Num. Compacting KVs # Num. Compacted KVs # Remaining KVs -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20130) Document the ports used by RS are changed to 16201/16301 when the RS is bound to localhost
Chia-Ping Tsai created HBASE-20130: -- Summary: Document the ports used by RS are changed to 16201/16301 when the RS is bound to localhost Key: HBASE-20130 URL: https://issues.apache.org/jira/browse/HBASE-20130 Project: HBase Issue Type: Bug Reporter: Chia-Ping Tsai -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20119) Introduce a pojo class to carry coprocessor information in order to make TableDescriptorBuilder accept multiple cp at once
Chia-Ping Tsai created HBASE-20119:
--
Summary: Introduce a pojo class to carry coprocessor information
in order to make TableDescriptorBuilder accept multiple cp at once
Key: HBASE-20119
URL: https://issues.apache.org/jira/browse/HBASE-20119
Project: HBase
Issue Type: Task
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai
Fix For: 2.0.0
The way to add cp to TableDescriptorBuilder is shown below.
{code:java}
public TableDescriptorBuilder addCoprocessor(String className) throws
IOException {
return addCoprocessor(className, null, Coprocessor.PRIORITY_USER, null);
}
public TableDescriptorBuilder addCoprocessor(String className, Path jarFilePath,
int priority, final Map kvs) throws IOException {
desc.addCoprocessor(className, jarFilePath, priority, kvs);
return this;
}
public TableDescriptorBuilder addCoprocessorWithSpec(final String specStr)
throws IOException {
desc.addCoprocessorWithSpec(specStr);
return this;
}{code}
When loading our config to create table with multiple cps, we have to write the
ugly for-loop.
{code:java}
val builder = TableDescriptorBuilder.newBuilde(tableName)
.setAAA()
.setBBB()
cfs.map(cf => ColumnFamilyDescriptorBuilder.of(cf))
.foreach(builder.addColumnFamily(_))
cps.foreach(builder.addCoprocessor(_))
admin.createTable(builder.build())
{code}
If we introduce a pojo to carry the cp data and add the method accepting
multiple cps and cfs, it is easier to exercise the fluent interface of
TableDescriptorBuilder
{code:java}
admin.createTable(TableDescriptorBuilder.newBuilde(tableName)
.addCoprocessor(cps.map(toHBaseCp).asJavaCollection)
.addColumnFamily(cf.map(toHBaseCf).asJavaCollection)
.setAAA()
.setBBB()
.build){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Reopened] (HBASE-20093) Replace ServerLoad by ServerMetrics for ServerManager
[ https://issues.apache.org/jira/browse/HBASE-20093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai reopened HBASE-20093: reopen to submit addendum > Replace ServerLoad by ServerMetrics for ServerManager > - > > Key: HBASE-20093 > URL: https://issues.apache.org/jira/browse/HBASE-20093 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Major > Fix For: 2.0.0 > > Attachments: HBASE-20093.v0.patch, HBASE-20093.v1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20098) Fix TestSplitTransactionOnCluster#testMasterRestartWhenSplittingIsPartial for branch-1.2
Chia-Ping Tsai created HBASE-20098: -- Summary: Fix TestSplitTransactionOnCluster#testMasterRestartWhenSplittingIsPartial for branch-1.2 Key: HBASE-20098 URL: https://issues.apache.org/jira/browse/HBASE-20098 Project: HBase Issue Type: Task Components: test Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai Fix For: 1.2.7 The new master started by TestSplitTransactionOnCluster#testMasterRestartWhenSplittingIsPartial will enable the CatalogJanitor. Hence, the parent region in SPLIT state will be removed by CatalogJanitor since all reference files of the parent region are removed by previous compaction. branch-1.3+ aren't in this trouble since HBASE-13082 and HBASE-14970 make the compacted file be archived by a background thread. Before the thread cleans up the compacted files, the parent region in SPLIT state won't be removed as there are file is referenced by its daughter region. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20097) Remove TableDescriptors#getAll since it just clone the returned object from TableDescriptors#getAllDescriptors
Chia-Ping Tsai created HBASE-20097:
--
Summary: Remove TableDescriptors#getAll since it just clone the
returned object from TableDescriptors#getAllDescriptors
Key: HBASE-20097
URL: https://issues.apache.org/jira/browse/HBASE-20097
Project: HBase
Issue Type: Task
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai
Fix For: 2.0.0
{code:java}
@Override
public Map getAll() throws IOException {
Map htds = new TreeMap<>();
Map allDescriptors = getAllDescriptors();
for (Map.Entry entry : allDescriptors
.entrySet()) {
htds.put(entry.getKey(), entry.getValue());
}
return htds;
}{code}
The returned map from #getAllDescriptors isn't a inner object so doing the
clone is meaningless.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-20093) Replace ServerLoad by ServerMetrics for ServerManager
Chia-Ping Tsai created HBASE-20093: -- Summary: Replace ServerLoad by ServerMetrics for ServerManager Key: HBASE-20093 URL: https://issues.apache.org/jira/browse/HBASE-20093 Project: HBase Issue Type: Task Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai Fix For: 2.0.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20092) Fix TestRegionMetrics
Chia-Ping Tsai created HBASE-20092:
--
Summary: Fix TestRegionMetrics
Key: HBASE-20092
URL: https://issues.apache.org/jira/browse/HBASE-20092
Project: HBase
Issue Type: Task
Components: test
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai
Fix For: 2.0.0
{code:java}
java.lang.AssertionError: expected:<12> but was:<13>
at
org.apache.hadoop.hbase.TestRegionMetrics.testRegionMetrics(TestRegionMetrics.java:111){code}
[http://104.198.223.121:8080/job/HBASE-Flaky-Tests/34589/testReport/junit/org.apache.hadoop.hbase/TestRegionMetrics/testRegionMetrics/]
http://104.198.223.121:8080/job/HBASE-Flaky-Tests/34591/testReport/junit/org.apache.hadoop.hbase/TestRegionMetrics/testRegionMetrics/
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-20084) Refactor the RSRpcServices#doBatchOp
Chia-Ping Tsai created HBASE-20084: -- Summary: Refactor the RSRpcServices#doBatchOp Key: HBASE-20084 URL: https://issues.apache.org/jira/browse/HBASE-20084 Project: HBase Issue Type: Task Components: regionserver Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai Fix For: 2.0.0 follow the discussion in https://issues.apache.org/jira/browse/HBASE-19876?focusedCommentId=16359618&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16359618 RSRpcServices#doBatchOp will throw IOE to log the error in the region-level response if any mutation fails in atomic mode. However, the exception in method signature force normal (non-atomic) batch to handle the exception even though no IOE won't be thrown in non-atomic mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20019) Document the ColumnValueFilter
Chia-Ping Tsai created HBASE-20019: -- Summary: Document the ColumnValueFilter Key: HBASE-20019 URL: https://issues.apache.org/jira/browse/HBASE-20019 Project: HBase Issue Type: Sub-task Reporter: Chia-Ping Tsai -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20017) BufferedMutatorImpl submit the same mutation repeatedly
Chia-Ping Tsai created HBASE-20017: -- Summary: BufferedMutatorImpl submit the same mutation repeatedly Key: HBASE-20017 URL: https://issues.apache.org/jira/browse/HBASE-20017 Project: HBase Issue Type: Bug Components: Client Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai Fix For: 2.0.0, 1.5.0, 1.4.3 BMI pass a iter of inner buffer to AccessProcess to take the undealt mutations, AsyncProcess call iter#next to get the mutation and then call iter#remove to delete the mutation from inner buffer. Hence, There's a good chance that the mutation is processed repeatedly in case there are a bunch of threads which are running the flush. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20016) TestCatalogJanitorInMemoryStates#testInMemoryForReplicaParentCleanup is flaky
Chia-Ping Tsai created HBASE-20016: -- Summary: TestCatalogJanitorInMemoryStates#testInMemoryForReplicaParentCleanup is flaky Key: HBASE-20016 URL: https://issues.apache.org/jira/browse/HBASE-20016 Project: HBase Issue Type: Bug Components: test Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai Fix For: 1.5.0, 1.4.3 It is a time-based test. RegionStates#isRegionOnline will return false if the target region is in transition. The list of region assignment may not updated yet. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-11622) completebulkload/loadIncrementalHFiles cannot specify table with namespace
[
https://issues.apache.org/jira/browse/HBASE-11622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved HBASE-11622.
Resolution: Duplicate
> completebulkload/loadIncrementalHFiles cannot specify table with namespace
> --
>
> Key: HBASE-11622
> URL: https://issues.apache.org/jira/browse/HBASE-11622
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.0
>Reporter: Jianshi Huang
>Priority: Major
>
> I'm using completebulkload to load 500GB of data to a table (presplitted).
> However, it reports the following errors:
> Looks like completebulkload didn't recognize the namespace part
> (namespace:table).
> Caused by: java.net.URISyntaxException: Relative path in absolute URI:
> grapple:vertices,37.bottom
> at java.net.URI.checkPath(URI.java:1804)
> at java.net.URI.(URI.java:752)
> at org.apache.hadoop.fs.Path.initialize(Path.java:203)
> By looking at the source code of LoadIncrementalHFiles.java, it seems the
> temporary path created for splitting will contain ':',
> The error part should be this:
> String uniqueName = getUniqueName(table.getName());
> HColumnDescriptor familyDesc =
> table.getTableDescriptor().getFamily(item.family);
> Path botOut = new Path(tmpDir, uniqueName + ".bottom");
> Path topOut = new Path(tmpDir, uniqueName + ".top");
> splitStoreFile(getConf(), hfilePath, familyDesc, splitKey,
> botOut, topOut);
> uniqueName will be "namespce:table" so "new Path(...)" will fail.
> A bug?
> P.S.
> Comment from Matteo Bertozzi:
> we don't need the name to be related to the table name.
> We can replace the getUniqueName() using something like this
> String getUniqueName(TableName tableName) {
> String name = UUID.randomUUID().toString().replaceAll("-", "") +
> "," + regionCount.incrementAndGet();
> return name;
> }
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Reopened] (HBASE-11622) completebulkload/loadIncrementalHFiles cannot specify table with namespace
[
https://issues.apache.org/jira/browse/HBASE-11622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai reopened HBASE-11622:
> completebulkload/loadIncrementalHFiles cannot specify table with namespace
> --
>
> Key: HBASE-11622
> URL: https://issues.apache.org/jira/browse/HBASE-11622
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.0
>Reporter: Jianshi Huang
>Priority: Major
>
> I'm using completebulkload to load 500GB of data to a table (presplitted).
> However, it reports the following errors:
> Looks like completebulkload didn't recognize the namespace part
> (namespace:table).
> Caused by: java.net.URISyntaxException: Relative path in absolute URI:
> grapple:vertices,37.bottom
> at java.net.URI.checkPath(URI.java:1804)
> at java.net.URI.(URI.java:752)
> at org.apache.hadoop.fs.Path.initialize(Path.java:203)
> By looking at the source code of LoadIncrementalHFiles.java, it seems the
> temporary path created for splitting will contain ':',
> The error part should be this:
> String uniqueName = getUniqueName(table.getName());
> HColumnDescriptor familyDesc =
> table.getTableDescriptor().getFamily(item.family);
> Path botOut = new Path(tmpDir, uniqueName + ".bottom");
> Path topOut = new Path(tmpDir, uniqueName + ".top");
> splitStoreFile(getConf(), hfilePath, familyDesc, splitKey,
> botOut, topOut);
> uniqueName will be "namespce:table" so "new Path(...)" will fail.
> A bug?
> P.S.
> Comment from Matteo Bertozzi:
> we don't need the name to be related to the table name.
> We can replace the getUniqueName() using something like this
> String getUniqueName(TableName tableName) {
> String name = UUID.randomUUID().toString().replaceAll("-", "") +
> "," + regionCount.incrementAndGet();
> return name;
> }
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-19912) The flag "writeToWAL" of Region#checkAndRowMutate is useless
Chia-Ping Tsai created HBASE-19912: -- Summary: The flag "writeToWAL" of Region#checkAndRowMutate is useless Key: HBASE-19912 URL: https://issues.apache.org/jira/browse/HBASE-19912 Project: HBase Issue Type: Improvement Reporter: Chia-Ping Tsai Fix For: 2.0.0-beta-2 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (HBASE-19897) RowMutations should follow the fluent pattern
[
https://issues.apache.org/jira/browse/HBASE-19897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai reopened HBASE-19897:
We can apply the mutation to the adder as BufferedMutator does
> RowMutations should follow the fluent pattern
> -
>
> Key: HBASE-19897
> URL: https://issues.apache.org/jira/browse/HBASE-19897
> Project: HBase
> Issue Type: New Feature
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Minor
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-19897.v0.patch
>
>
> Other row ops, including {{Put}}, {{Delete}}, {{Get}}, {{Scan}}, do have the
> fluent interface. Also, Changing the return type from {{Void}} to
> {{RowMutations}} won't break the API BC (unless someone has interest in
> {{Void}} object...)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-19900) The failed op is added to RetriesExhaustedWithDetailsException repeatedly
Chia-Ping Tsai created HBASE-19900: -- Summary: The failed op is added to RetriesExhaustedWithDetailsException repeatedly Key: HBASE-19900 URL: https://issues.apache.org/jira/browse/HBASE-19900 Project: HBase Issue Type: Bug Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai AsyncRequestFutureImpl#receiveMultiAction process the action-lever error first, and then add the region-level exception to each action. Hence, user may get the various exceptions for the same action (row op) from the RetriesExhaustedWithDetailsException. In fact, if both of action-level exception and region-lever exception exist, they always have the same context. I'm not sure whether that is what RetriesExhaustedWithDetailsException want. As i see it, we shouldn't have the duplicate ops in RetriesExhaustedWithDetailsException since that may confuse users if they catch the RetriesExhaustedWithDetailsException to check the invalid operations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-19897) RowMutations should follow the fluent pattern
Chia-Ping Tsai created HBASE-19897:
--
Summary: RowMutations should follow the fluent pattern
Key: HBASE-19897
URL: https://issues.apache.org/jira/browse/HBASE-19897
Project: HBase
Issue Type: New Feature
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai
Fix For: 2.0.0-beta-2
Other row ops, including {{Put}}, {{Delete}}, {{Get}}, {{Scan}}, do have the
fluent interface. Also, Changing the return type from {{Void}} to
{{RowMutations}} won't break the API BC.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-19877) hbase-common and hbase-zookeeper don't add the log4j.properties to the resource path for testing
Chia-Ping Tsai created HBASE-19877: -- Summary: hbase-common and hbase-zookeeper don't add the log4j.properties to the resource path for testing Key: HBASE-19877 URL: https://issues.apache.org/jira/browse/HBASE-19877 Project: HBase Issue Type: Bug Components: test Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai Fix For: 2.0.0-beta-2 The lack of log4j.properties will disable log4j appenders. I can't imagine how to live without the log. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-19876) The exception happening in converting pb mutation to hbase.mutation messes up the CellScanner
Chia-Ping Tsai created HBASE-19876:
--
Summary: The exception happening in converting pb mutation to
hbase.mutation messes up the CellScanner
Key: HBASE-19876
URL: https://issues.apache.org/jira/browse/HBASE-19876
Project: HBase
Issue Type: Bug
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai
Fix For: 1.3.2, 1.5.0, 1.2.7, 2.0.0-beta-2, 1.4.2
{code:java}
2018-01-27 22:51:43,794 INFO [hconnection-0x3291b443-shared-pool11-t6]
client.AsyncRequestFutureImpl(778): id=5, table=testQuotaStatusFromMaster3,
attempt=6/16 failed=20ops, last
exception=org.apache.hadoop.hbase.client.WrongRowIOException:
org.apache.hadoop.hbase.client.WrongRowIOException: The row in xxx doesn't
match the original one aaa
at org.apache.hadoop.hbase.client.Mutation.add(Mutation.java:776)
at org.apache.hadoop.hbase.client.Put.add(Put.java:282)
at
org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toPut(ProtobufUtil.java:642)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:952)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:896)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2591)
at
org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41560)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:404)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304){code}
I noticed this bug when testing the table space quota.
When rs are converting pb mutation to hbase.mutation, the quota exception or
cell exception may be thrown.
{code}
Unable to find source-code formatter for language:
rsrpcservices#dobatchop.java. Available languages are: actionscript, ada,
applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html,
java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r,
rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml for
(ClientProtos.Action action: mutations) {
MutationProto m = action.getMutation();
Mutation mutation;
if (m.getMutateType() == MutationType.PUT) {
mutation = ProtobufUtil.toPut(m, cells);
batchContainsPuts = true;
} else {
mutation = ProtobufUtil.toDelete(m, cells);
batchContainsDelete = true;
}
mutationActionMap.put(mutation, action);
mArray[i++] = mutation;
checkCellSizeLimit(region, mutation);
// Check if a space quota disallows this mutation
spaceQuotaEnforcement.getPolicyEnforcement(region).check(mutation);
quota.addMutation(mutation);
}
{code}
rs has caught the exception but it doesn't have the cellscanner skip the failed
cells.
{code:java}
} catch (IOException ie) {
if (atomic) {
throw ie;
}
for (Action mutation : mutations) {
builder.addResultOrException(getResultOrException(ie,
mutation.getIndex()));
}
}
{code}
The bug results in the WrongRowIOException to remaining mutations since they
refer to invalid cells.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-19870) Fix the NEP in ReadOnlyZKClient#run
Chia-Ping Tsai created HBASE-19870:
--
Summary: Fix the NEP in ReadOnlyZKClient#run
Key: HBASE-19870
URL: https://issues.apache.org/jira/browse/HBASE-19870
Project: HBase
Issue Type: Sub-task
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai
I notice a NPE from my jenkins.
{code}
2018-01-26 17:26:41,078 DEBUG [M:0;8546d406e429:40557-EventThread]
zookeeper.ZKWatcher(443): replicationLogCleaner-0x161337ddc090004,
quorum=localhost:56060, baseZNode=/hbase Received ZooKeeper Event, type=None,
state=Disconnected, path=null
java.lang.NullPointerException
at
org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:322)
at java.lang.Thread.run(Thread.java:748)
{code}
If any zk task invokes the #onComplete late, the count of current requests will
not zero and then the null from task queue will destroy the work thread in
ReadOnlyZKClient.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-19844) Shell should support to flush by regionserver
Chia-Ping Tsai created HBASE-19844: -- Summary: Shell should support to flush by regionserver Key: HBASE-19844 URL: https://issues.apache.org/jira/browse/HBASE-19844 Project: HBase Issue Type: New Feature Components: shell Reporter: Chia-Ping Tsai Fix For: 2.0.0 HBASE-4224 add a method to admin that can do the flush by regionserver. As with other Admin methods, we should enable shell to use the flush method. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-19779) The chunk encountering the OOM will store in ChunkCreator forever
Chia-Ping Tsai created HBASE-19779:
--
Summary: The chunk encountering the OOM will store in ChunkCreator
forever
Key: HBASE-19779
URL: https://issues.apache.org/jira/browse/HBASE-19779
Project: HBase
Issue Type: Sub-task
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai
Priority: Minor
Fix For: 2.0.0-beta-2
If Chunk#init fail on OOM, the MSLABimpl won't store the id of chunk. We have
no chance to remove the chunk from {{ChunkCreator}} since MSLABimpl have missed
the id.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19778) Add the jira link to the Flaky Test
Chia-Ping Tsai created HBASE-19778: -- Summary: Add the jira link to the Flaky Test Key: HBASE-19778 URL: https://issues.apache.org/jira/browse/HBASE-19778 Project: HBase Issue Type: Brainstorming Reporter: Chia-Ping Tsai Priority: Minor Adding the jira link to do the text search can make life easier. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19746) Add default impl to Cell#getType
Chia-Ping Tsai created HBASE-19746:
--
Summary: Add default impl to Cell#getType
Key: HBASE-19746
URL: https://issues.apache.org/jira/browse/HBASE-19746
Project: HBase
Issue Type: Sub-task
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai
Priority: Critical
Fix For: 2.0.0
Noticed this issue when migrating the app to branch-2.
{{Cell}} is IA.Public so it should obey our compatibility rules. Not sure
whether any related discussion had be in HBASE-19112. It worthwhile, however,
to raise this issue again.
FYI [~anoopsamjohn] [~ram_krish] [~stack]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19720) Rename WALKey#getTabnename to WALKey#getTableName
Chia-Ping Tsai created HBASE-19720: -- Summary: Rename WALKey#getTabnename to WALKey#getTableName Key: HBASE-19720 URL: https://issues.apache.org/jira/browse/HBASE-19720 Project: HBase Issue Type: Task Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai Fix For: 2.0.0 WALKey is denoted as LP so its naming should obey the common rule in our codebase. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19713) Enable TestInterfaceAudienceAnnotations
Chia-Ping Tsai created HBASE-19713: -- Summary: Enable TestInterfaceAudienceAnnotations Key: HBASE-19713 URL: https://issues.apache.org/jira/browse/HBASE-19713 Project: HBase Issue Type: Task Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai Priority: Critical Fix For: 2.0.0 Make sure TestInterfaceAudienceAnnotations pass before 2.0 release. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-19383) [1.2] java.lang.AssertionError: expected:<2> but was:<1> at org.apache.hadoop.hbase.TestChoreService.testTriggerNowFailsWhenNotScheduled(TestChoreService.java:707)
[ https://issues.apache.org/jira/browse/HBASE-19383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved HBASE-19383. Resolution: Fixed Hadoop Flags: Reviewed 1.4 had been released. Pushed this to branch-1.4. > [1.2] java.lang.AssertionError: expected:<2> but was:<1> at > org.apache.hadoop.hbase.TestChoreService.testTriggerNowFailsWhenNotScheduled(TestChoreService.java:707) > > > Key: HBASE-19383 > URL: https://issues.apache.org/jira/browse/HBASE-19383 > Project: HBase > Issue Type: Bug > Components: test >Reporter: stack >Assignee: stack > Fix For: 1.3.2, 1.4.1, 1.2.7, 2.0.0-beta-2 > > Attachments: 19383.txt > > > A test that is based on timers that asserts hard numbers about how many times > something is called. I'm just going to remove it. It killed my 1.2 nightly > test run. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-18289) TestReplicaWithCluster#testReplicaGetWithPrimaryAndMetaDown is a flaky test
[ https://issues.apache.org/jira/browse/HBASE-18289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved HBASE-18289. Resolution: Duplicate HBASE-19392 > TestReplicaWithCluster#testReplicaGetWithPrimaryAndMetaDown is a flaky test > --- > > Key: HBASE-18289 > URL: https://issues.apache.org/jira/browse/HBASE-18289 > Project: HBase > Issue Type: Test > Components: test >Reporter: Mike Drob > Attachments: > org.apache.hadoop.hbase.client.TestReplicaWithCluster-output.txt > > > Started looking into this today, spend about half the day, and couldn't > finish it off, but am filing this JIRA so that I can record progress > somewhere and maybe somebody else with more contextual knowledge can chime in. > I'll attach a truncated log file from one of the flaky job runs that focuses > on only this test. > We enable the regions are down simulation at 16:31:30 in the file, and we can > see that reads on the primary fail and then succeed on the replica for a > while. There's a lot of stack traces starting at that point, so I have > trouble keeping track of when exactly the replica disappears. Scans of the > meta replica look like they work the whole time, it's the user table that > fails. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-16936) TestRateLimiter#testOverconsumptionFixedIntervalRefillStrategy is flaky
[
https://issues.apache.org/jira/browse/HBASE-16936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved HBASE-16936.
Resolution: Duplicate
HBASE-19490
> TestRateLimiter#testOverconsumptionFixedIntervalRefillStrategy is flaky
> ---
>
> Key: HBASE-16936
> URL: https://issues.apache.org/jira/browse/HBASE-16936
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Dima Spivak
>Assignee: huaxiang sun
>
> Seeing this once every month or two in-house. Looks like it's a timing-based
> test, which makes it prone to flakiness, but I've noticed that whenever it
> fails, it fails with the same {{AssertionError}} (including values), so it'd
> be worth digging into. In our case:
> {noformat}
> expected:<1000> but was:<999>
> Stack Trace:
> java.lang.AssertionError: expected:<1000> but was:<999>
> at
> org.apache.hadoop.hbase.quotas.TestRateLimiter.testOverconsumptionFixedIntervalRefillStrategy(TestRateLimiter.java:119)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (HBASE-7722) Fix TestRegionServerCoprocessorExceptionWithAbort flakiness in trunk
[ https://issues.apache.org/jira/browse/HBASE-7722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved HBASE-7722. --- Resolution: Duplicate HBASE-10292 > Fix TestRegionServerCoprocessorExceptionWithAbort flakiness in trunk > > > Key: HBASE-7722 > URL: https://issues.apache.org/jira/browse/HBASE-7722 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 0.95.2 >Reporter: Himanshu Vashishtha >Assignee: Himanshu Vashishtha > > In trunk, the test fails as the table.put() statement in the test passes even > with BuggyRegionCoprocessor failed the transaction: > "The put should have failed, as the coprocessor is buggy". -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19712) Fix TestSnapshotQuotaObserverChore#testSnapshotSize
Chia-Ping Tsai created HBASE-19712: -- Summary: Fix TestSnapshotQuotaObserverChore#testSnapshotSize Key: HBASE-19712 URL: https://issues.apache.org/jira/browse/HBASE-19712 Project: HBase Issue Type: Bug Components: test Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai Fix For: 2.0.0-beta-2 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-15580) Tag coprocessor limitedprivate scope to StoreFile.Reader
[
https://issues.apache.org/jira/browse/HBASE-15580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved HBASE-15580.
Resolution: Fixed
Hadoop Flags: Reviewed
> Tag coprocessor limitedprivate scope to StoreFile.Reader
>
>
> Key: HBASE-15580
> URL: https://issues.apache.org/jira/browse/HBASE-15580
> Project: HBase
> Issue Type: Improvement
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Rajeshbabu Chintaguntla
> Fix For: 1.4.1, 1.5.0
>
> Attachments: HBASE-15580.patch, HBASE-15580_branch-1.0.patch
>
>
> For phoenix local indexing we need to have custom storefile reader
> constructor(IndexHalfStoreFileReader) to distinguish from other storefile
> readers. So wanted to mark StoreFile.Reader scope as
> InterfaceAudience.LimitedPrivate("Coprocessor")
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19685) Fix TestFSErrorsExposed#testFullSystemBubblesFSErrors
Chia-Ping Tsai created HBASE-19685:
--
Summary: Fix TestFSErrorsExposed#testFullSystemBubblesFSErrors
Key: HBASE-19685
URL: https://issues.apache.org/jira/browse/HBASE-19685
Project: HBase
Issue Type: Bug
Components: test
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai
Fix For: 2.0.0-beta-2
{code}
java.lang.AssertionError
at
org.apache.hadoop.hbase.regionserver.TestFSErrorsExposed.testFullSystemBubblesFSErrors(TestFSErrorsExposed.java:221)
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19680) BufferedMutatorImpl#mutate should wait the result from AP in order to throw the failed mutations
Chia-Ping Tsai created HBASE-19680: -- Summary: BufferedMutatorImpl#mutate should wait the result from AP in order to throw the failed mutations Key: HBASE-19680 URL: https://issues.apache.org/jira/browse/HBASE-19680 Project: HBase Issue Type: Improvement Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai Fix For: 2.0.0-beta-2 Currently, BMI#mutate doesn't wait the result from AP so the errors are stored in AP. The only way which can return the errors to user is, calling the flush to catch the exception. That is non-intuitive. I feel BMI#mutate should wait the result. That is to say, user can parse the exception thrown by BM#mutate to get the failed mutations. Also, we can remove the global error from AP. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Reopened] (HBASE-19486) Periodically ensure records are not buffered too long by BufferedMutator
[ https://issues.apache.org/jira/browse/HBASE-19486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai reopened HBASE-19486: > Periodically ensure records are not buffered too long by BufferedMutator > - > > Key: HBASE-19486 > URL: https://issues.apache.org/jira/browse/HBASE-19486 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Niels Basjes >Assignee: Niels Basjes > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19486-20171212-2117.patch, > HBASE-19486-20171218-1229.patch, HBASE-19486-20171218-1300.patch, > HBASE-19486-20171219-0933.patch, HBASE-19486-20171219-1026.patch, > HBASE-19486-20171219-1122-trigger-qa-run.patch, > HBASE-19486-20171220-1612-trigger-qa-run.patch, > HBASE-19486-20171220-2228-trigger-qa-run.patch, > HBASE-19486-20171223-1438-trigger-qa-run.patch, > HBASE-19486-20171223-1728-trigger-qa-run.patch, > HBASE-19486-20171223--trigger-qa-run.patch, > HBASE-19486-20171224-1101-trigger-qa-run.patch, > HBASE-19486-20171224-1602.patch, HBASE-19486-branch-1.v0.patch, > HBASE-19486-branch-1.v1.patch, HBASE-19486.v0.patch > > > I'm working on several projects where we are doing stream / event type > processing instead of batch type processing. We mostly use Apache Flink and > Apache Beam for these projects. > When we ingest a continuous stream of events and feed that into HBase via a > BufferedMutator this all works fine. The buffer fills up at a predictable > rate and we can make sure it flushes several times per second into HBase by > tuning the buffer size. > We also have situations where the event rate is unpredictable. Some times > because the source is in reality a batch job that puts records into Kafka, > sometimes because it is the "predictable in production" application in our > testing environment (where only the dev triggers a handful of events). > For these kinds of use cases we need a way to 'force' the BufferedMutator to > automatically flush any records in the buffer even if the buffer is not full. > I'll put up a pull request with a proposed implementation for review against > the master (i.e. 3.0.0). > When approved I would like to backport this to the 1.x and 2.x versions of > the client in the same (as close as possible) way. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19671) Fix TestMultiParallel#testActiveThreadsCount
Chia-Ping Tsai created HBASE-19671:
--
Summary: Fix TestMultiParallel#testActiveThreadsCount
Key: HBASE-19671
URL: https://issues.apache.org/jira/browse/HBASE-19671
Project: HBase
Issue Type: Bug
Components: test
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai
Priority: Minor
Fix For: 2.0.0
{code}
java.lang.AssertionError: expected:<4> but was:<5>
at
org.apache.hadoop.hbase.client.TestMultiParallel.testActiveThreadsCount(TestMultiParallel.java:168)
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (HBASE-19425) Align the methods in Put/Delete/Increment/Append
[ https://issues.apache.org/jira/browse/HBASE-19425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved HBASE-19425. Resolution: Fixed All tasks are done. > Align the methods in Put/Delete/Increment/Append > > > Key: HBASE-19425 > URL: https://issues.apache.org/jira/browse/HBASE-19425 > Project: HBase > Issue Type: Umbrella >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai > Fix For: 2.0.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19667) gir rid of MasterEnvironment#supportGroupCPs
Chia-Ping Tsai created HBASE-19667:
--
Summary: gir rid of MasterEnvironment#supportGroupCPs
Key: HBASE-19667
URL: https://issues.apache.org/jira/browse/HBASE-19667
Project: HBase
Issue Type: Sub-task
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai
copy the discussion from HBASE-19500.
{quote}
Appy:
Can we remove {{if(((MasterEnvironment)getEnvironment()).supportGroupCPs) }} in
so many places since CP in 2.0 are already broken left and right (and we'll
have to solve legacy issue more holistically) What say Anoop Sam John?
Chia:
The cp user must do something to migrate their cp code from 1.x to 2.0 anyway,
hence the check is unnecessary in 2.0 I'd say. Seems it is unrelated to this
jira. Iron out it in another jira?
Appy:
Chia-Ping Tsai you can take up the check removal?
{quote}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (HBASE-14562) Fix and reenable zombie TestReplicationShell
[ https://issues.apache.org/jira/browse/HBASE-14562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved HBASE-14562. Resolution: Duplicate hbase-15965 > Fix and reenable zombie TestReplicationShell > > > Key: HBASE-14562 > URL: https://issues.apache.org/jira/browse/HBASE-14562 > Project: HBase > Issue Type: Bug >Reporter: stack > > Was disabled over in HBASE-14561 because it hangs with some regularity. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-8053) Reenable TestHBaseFsck#testQuarantineMissingFamdir
[ https://issues.apache.org/jira/browse/HBASE-8053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved HBASE-8053. --- Resolution: Won't Fix The test is gone. HBASE-19272 > Reenable TestHBaseFsck#testQuarantineMissingFamdir > -- > > Key: HBASE-8053 > URL: https://issues.apache.org/jira/browse/HBASE-8053 > Project: HBase > Issue Type: Task >Reporter: stack > > HBASE-8052 disabled it. Reenable after figure why it fails sporadically. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-9010) Reenable TestMultiTableInputFormat
[ https://issues.apache.org/jira/browse/HBASE-9010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved HBASE-9010. --- Resolution: Duplicate HBASE-10774 > Reenable TestMultiTableInputFormat > -- > > Key: HBASE-9010 > URL: https://issues.apache.org/jira/browse/HBASE-9010 > Project: HBase > Issue Type: Task > Components: test >Reporter: stack > > See HBASE-9009 where we disable it because it fails in mysterious ways. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-6127) TestAtomicOperation#testMultiRowMutationMultiThreads occasionally fails
[ https://issues.apache.org/jira/browse/HBASE-6127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved HBASE-6127. --- Resolution: Duplicate Looped the test 20 times for branch-1 and master. no failure. Seems it was resolved by HBASE-14794. > TestAtomicOperation#testMultiRowMutationMultiThreads occasionally fails > --- > > Key: HBASE-6127 > URL: https://issues.apache.org/jira/browse/HBASE-6127 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Ted Yu > > TestAtomicOperation occasionally fails. > Here is one instance: > https://builds.apache.org/job/HBase-TRUNK/2944/testReport/junit/org.apache.hadoop.hbase.regionserver/TestAtomicOperation/testMultiRowMutationMultiThreads/ -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-7141) Cleanup Increment and Append issues
[ https://issues.apache.org/jira/browse/HBASE-7141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved HBASE-7141. --- Resolution: Duplicate bq. Append and Increment should take a TS for their update phase HBASE-18546 bq. Append should access a timerange for the read phase HBASE-19427 bq. Increment should no longer implement Writable (in trunk) HBASE-7215 bq. Increment and Append makes changes visible through the memstore before the WAL is sync'ed HBASE-4583 > Cleanup Increment and Append issues > --- > > Key: HBASE-7141 > URL: https://issues.apache.org/jira/browse/HBASE-7141 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl > > * Append and Increment should take a TS for their update phase > * Append should access a timerange for the read phase > * Increment should no longer implement Writable (in trunk) > * Increment and Append makes changes visible through the memstore before the > WAL is sync'ed > This depends on HBASE-4583 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-5851) TestProcessBasedCluster sometimes fails; currently disabled -- needs to be fixed and reenabled
[
https://issues.apache.org/jira/browse/HBASE-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved HBASE-5851.
---
Resolution: Won't Fix
see HBASE-14571
> TestProcessBasedCluster sometimes fails; currently disabled -- needs to be
> fixed and reenabled
> --
>
> Key: HBASE-5851
> URL: https://issues.apache.org/jira/browse/HBASE-5851
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: disable.txt, hbase-5851.patch, hbase-5851_v2.patch,
> metahang.txt, zkfail.txt
>
>
> TestProcessBasedCluster failed in
> https://builds.apache.org/job/HBase-TRUNK-security/178
> Looks like cluster failed to start:
> {code}
> 2012-04-21 14:22:32,666 INFO [Thread-1]
> util.ProcessBasedLocalHBaseCluster(176): Waiting for HBase to startup.
> Retries left: 2
> java.io.IOException: Giving up trying to location region in meta: thread is
> interrupted.
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1173)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:956)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:917)
> at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:252)
> at org.apache.hadoop.hbase.client.HTable.(HTable.java:192)
> at
> org.apache.hadoop.hbase.util.ProcessBasedLocalHBaseCluster.startHBase(ProcessBasedLocalHBaseCluster.java:174)
> at
> org.apache.hadoop.hbase.util.TestProcessBasedCluster.testProcessBasedCluster(TestProcessBasedCluster.java:56)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
> java.lang.InterruptedException: sleep interrupted at
> java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hbase.util.Threads.sleep(Threads.java:134)
> at
> org.apache.hadoop.hbase.util.ProcessBasedLocalHBaseCluster.startHBase(ProcessBasedLocalHBaseCluster.java:178)
> at
> org.apache.hadoop.hbase.util.TestProcessBasedCluster.testProcessBasedCluster(TestProcessBasedCluster.java:56)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19644) add the checkstyle rule to rejects the illegal imports
Chia-Ping Tsai created HBASE-19644: -- Summary: add the checkstyle rule to rejects the illegal imports Key: HBASE-19644 URL: https://issues.apache.org/jira/browse/HBASE-19644 Project: HBase Issue Type: Task Affects Versions: 2.0.0 Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai The following imports should be disabled in our code. # com.google.common.* (we have the shaded version) # io.netty.* (ditto) # org.apache.curator.shaded.* # org.apache.htrace.shaded.* -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Reopened] (HBASE-19496) Reusing the ByteBuffer in rpc layer corrupt the ServerLoad and RegionLoad
[
https://issues.apache.org/jira/browse/HBASE-19496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai reopened HBASE-19496:
TestClassLoading is broken...
> Reusing the ByteBuffer in rpc layer corrupt the ServerLoad and RegionLoad
> -
>
> Key: HBASE-19496
> URL: https://issues.apache.org/jira/browse/HBASE-19496
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Blocker
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19496.v0.patch, HBASE-19496.v1.patch,
> HBASE-19496.v2.patch, HBASE-19496.wip.patch
>
>
> {{ServerLoad}} and {{RegionLoad}} store the pb object internally but the
> bytebuffer of pb object may be reused in rpc layer. Hence, the {{ServerLoad}}
> and {{RegionLoad}} which saved by {{HMaster}} will be corrupted if the
> bytebuffer backed is modified.
> This issue doesn't happen on branch-1.
> # netty server was introduced in 2.0 (see HBASE-17263)
> # reusing bytebuffer to read RPC requests was introduced in 2.0 (see
> HBASE-15788)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19629) RawCell#getTags should return the Iterator in order to avoid iterate through whole tag array at once
Chia-Ping Tsai created HBASE-19629: -- Summary: RawCell#getTags should return the Iterator in order to avoid iterate through whole tag array at once Key: HBASE-19629 URL: https://issues.apache.org/jira/browse/HBASE-19629 Project: HBase Issue Type: Sub-task Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19628) ByteBufferCell should extend ExtendedCell
Chia-Ping Tsai created HBASE-19628: -- Summary: ByteBufferCell should extend ExtendedCell Key: HBASE-19628 URL: https://issues.apache.org/jira/browse/HBASE-19628 Project: HBase Issue Type: Sub-task Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai All ByteBufferCell impl are assumed be of ExtendedCell, hence we should rename it to ByteBufferExtendedCell and make it be subclass of ExtendedCell. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19627) Stabilize the method signature for Cell and its impl
Chia-Ping Tsai created HBASE-19627: -- Summary: Stabilize the method signature for Cell and its impl Key: HBASE-19627 URL: https://issues.apache.org/jira/browse/HBASE-19627 Project: HBase Issue Type: Umbrella Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai Priority: Critical Fix For: 2.0.0 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19626) Rename Cell.DataType to Cell.Type
Chia-Ping Tsai created HBASE-19626: -- Summary: Rename Cell.DataType to Cell.Type Key: HBASE-19626 URL: https://issues.apache.org/jira/browse/HBASE-19626 Project: HBase Issue Type: Bug Reporter: Chia-Ping Tsai Priority: Minor Fix For: 2.0.0 Align the name with KeyValue.Type. Also, it is the return type of Cell.getType() so the qualifier "Data" is inappropriate. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19624) TestIOFencing hangs
Chia-Ping Tsai created HBASE-19624:
--
Summary: TestIOFencing hangs
Key: HBASE-19624
URL: https://issues.apache.org/jira/browse/HBASE-19624
Project: HBase
Issue Type: Bug
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai
Fix For: 2.0.0
RS calls CompactSplit#join to cease all compactSplit threads.
{code:title=CompactSplit.java}
private void waitFor(ThreadPoolExecutor t, String name) {
boolean done = false;
while (!done) {
try {
done = t.awaitTermination(60, TimeUnit.SECONDS);
LOG.info("Waiting for " + name + " to finish...");
if (!done) {
t.shutdownNow();
}
} catch (InterruptedException ie) {
LOG.warn("Interrupted waiting for " + name + " to finish...");
}
}
}
{code}
In the meantime, the async wal may wait for the sync signal. However, the
single won't happen as the wal sync is failed.
{code}
synchronized long get(long timeoutNs) throws InterruptedException,
ExecutionException, TimeoutIOException {
final long done = System.nanoTime() + timeoutNs;
while (!isDone()) {
wait(1000);
if (System.nanoTime() >= done) {
throw new TimeoutIOException(
"Failed to get sync result after " +
TimeUnit.NANOSECONDS.toMillis(timeoutNs)
+ " ms for txid=" + this.txid + ", WAL system stuck?");
}
}
if (this.throwable != null) {
throw new ExecutionException(this.throwable);
}
return this.doneTxid;
}
{code}
When we shutdown the mini cluster, JVMClusterUtil#shutdown sends the interrupt
single to all rs threads. And then catching the InterruptedException cause
compactionsplit to skip the #shutdownNow, hence the compactionsplit threads
were up until timeout (default is 5 min).
{code}
for (int i = 0; i < 100; ++i) {
boolean atLeastOneLiveServer = false;
for (RegionServerThread t : regionservers) {
if (t.isAlive()) {
atLeastOneLiveServer = true;
try {
LOG.warn("RegionServerThreads remaining, give one more chance
before interrupting");
t.join(1000);
} catch (InterruptedException e) {
wasInterrupted = true;
}
}
}
if (!atLeastOneLiveServer) break;
for (RegionServerThread t : regionservers) {
if (t.isAlive()) {
LOG.warn("RegionServerThreads taking too long to stop,
interrupting");
t.interrupt();
}
}
}
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (HBASE-19125) TestReplicator is flaky
[ https://issues.apache.org/jira/browse/HBASE-19125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved HBASE-19125. Resolution: Duplicate > TestReplicator is flaky > --- > > Key: HBASE-19125 > URL: https://issues.apache.org/jira/browse/HBASE-19125 > Project: HBase > Issue Type: Bug >Reporter: Andrew Purtell >Assignee: Andrew Purtell >Priority: Minor > Fix For: 2.0.0, 3.0.0, 1.4.1, 1.5.0 > > > TestReplicator fails now and again. I had a look at the test. This is > something I contributed a while back but looking at it again it needs a > different approach. I'm going to disable it for now until this issue is > resolved. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-19523) TestLogRolling is flakey
[ https://issues.apache.org/jira/browse/HBASE-19523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved HBASE-19523. Resolution: Duplicate see HBASE-19559 > TestLogRolling is flakey > > > Key: HBASE-19523 > URL: https://issues.apache.org/jira/browse/HBASE-19523 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Duo Zhang > > https://builds.apache.org/job/PreCommit-HBASE-Build/10475/testReport/ -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19607) throw the exception if memstore is incompetent to do the deep clone for cell
Chia-Ping Tsai created HBASE-19607:
--
Summary: throw the exception if memstore is incompetent to do the
deep clone for cell
Key: HBASE-19607
URL: https://issues.apache.org/jira/browse/HBASE-19607
Project: HBase
Issue Type: Bug
Reporter: Chia-Ping Tsai
Priority: Trivial
We must to clone the cell since the cell is backed by the reusable byte array.
Also, we assume all cells passed to AbstractMemStore is of ExtendedCell. Not
only is the type check unnecessary, but it also confuses the readers.
{code:title=AbstractMemStore.java}
private static Cell deepCopyIfNeeded(Cell cell) {
if (cell instanceof ExtendedCell) {
return ((ExtendedCell) cell).deepClone();
}
return cell;
}
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19596) RegionMetrics/ServerMetrics/ClusterMetrics should apply to all public class
Chia-Ping Tsai created HBASE-19596: -- Summary: RegionMetrics/ServerMetrics/ClusterMetrics should apply to all public class Key: HBASE-19596 URL: https://issues.apache.org/jira/browse/HBASE-19596 Project: HBase Issue Type: Task Components: Client Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai Priority: Critical Fix For: 2.0.0-beta-2 ServerLoad/RegionLoad/ClusterLoad are deprecated now. This issue will deprecate all related methods in our Public classes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19595) explicitly discourage public user from creating their Public pojo
Chia-Ping Tsai created HBASE-19595: -- Summary: explicitly discourage public user from creating their Public pojo Key: HBASE-19595 URL: https://issues.apache.org/jira/browse/HBASE-19595 Project: HBase Issue Type: Task Reporter: Chia-Ping Tsai Priority: Blocker HBASE-19496 make both of ReplicationLoadSink and ReplicationLoadSource be IA.Public since they are exposed by other Public class (ServerMetrics). Currently, their constructor are annotated with IA.Private to discourage user from creating their ReplicationLoadSink and ReplicationLoadSource. However, that is too implicit as users have to trace the source code to "see" the warnings. I believe that many pojo in hbase have same issue. We have some kinds of coding style to warn user not to create their Public pojo. # IA.Private constructor. for example, ReplicationLoadSink and ReplicationLoadSource # pure interface. for example, TableDescriptor, ColumnFamilyDescriptor, ClusterMetrics # final class and package private constructor. for example, CacheEvictionStats All of them have pros and cons. #1 is a simple solution but it is too implicit for user. #2 will cause the BC issue (see HBASE-19535). #3 is good solution to prevent user from using the constructor or extending the pojo. However, it may limit us also. (mockito or extend the pojo internally) Any suggestions? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19575) add copy constructor to Mutation
Chia-Ping Tsai created HBASE-19575: -- Summary: add copy constructor to Mutation Key: HBASE-19575 URL: https://issues.apache.org/jira/browse/HBASE-19575 Project: HBase Issue Type: Sub-task Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai Fix For: 2.0.0-beta-1 Put, Delete, Increment, and Append have similar code in copy constructor. Let us move the copy constructor to the upper class - mutation - so as to remove the duplicate code. BTW, the copy constructor of Put miss the clone of priority. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-19548) Backport the missed doc fix from master to branch-2
[ https://issues.apache.org/jira/browse/HBASE-19548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved HBASE-19548. Resolution: Fixed Assignee: Chien Hsiang Tang (was: Chia-Ping Tsai) Fix Version/s: 2.0.0-beta-1 [~Chien Hsiang Tang] compared the commit history of src folder between branch-2 and master. And then all missed commits have been pushed to branch-2. HBASE-16722 HBASE-17839 HBASE-17853 HBASE-18258 HBASE-18259 HBASE-18275 HBASE-18284 HBASE-18311 HBASE-18329 HBASE-18335 HBASE-18384 HBASE-18404 HBASE-18430 HBASE-18504 HBASE-18548 HBASE-18563 HBASE-18592 HBASE-18635 HBASE-18637 HBASE-18677 HBASE-18701 HBASE-18718 HBASE-18849 HBASE-18852 HBASE-19174 HBASE-19416 > Backport the missed doc fix from master to branch-2 > --- > > Key: HBASE-19548 > URL: https://issues.apache.org/jira/browse/HBASE-19548 > Project: HBase > Issue Type: Task >Reporter: Chia-Ping Tsai >Assignee: Chien Hsiang Tang > Fix For: 2.0.0-beta-1 > > > It seems there are many doc fixes aren't in branch-2. File this jira to trace > all missed commits. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19550) Wrap the Put in cp layer to make sure all passed cells are of ExtendedCell
Chia-Ping Tsai created HBASE-19550: -- Summary: Wrap the Put in cp layer to make sure all passed cells are of ExtendedCell Key: HBASE-19550 URL: https://issues.apache.org/jira/browse/HBASE-19550 Project: HBase Issue Type: Task Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai Fix For: 2.0.0-beta-2 We assume all cells in server are of ExtendedCell. However, cp user can add their cell impl via Put#add(Cell) in observer. That will cause UnsupportedOperationException when rs try to update the cell's timestamp and seq Id. We should do something for cp user...For example, wrap the passed cells to be a extendcell type. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19548) Backport the missed doc fix from master to branch-2
Chia-Ping Tsai created HBASE-19548: -- Summary: Backport the missed doc fix from master to branch-2 Key: HBASE-19548 URL: https://issues.apache.org/jira/browse/HBASE-19548 Project: HBase Issue Type: Task Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai It seems there are many doc fixes aren't in branch-2. File this jira to trace all missed commits. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19542) fix TestSafemodeBringsDownMaster
Chia-Ping Tsai created HBASE-19542: -- Summary: fix TestSafemodeBringsDownMaster Key: HBASE-19542 URL: https://issues.apache.org/jira/browse/HBASE-19542 Project: HBase Issue Type: Bug Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai Fix For: 2.0.0-beta-1 We need to check the stability of underlay file system when closing async wal. Otherwise, our hbase can't shutdown gracefully. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19535) Introduce a new annotation to say there is no BC guarantee in the inheritance
Chia-Ping Tsai created HBASE-19535: -- Summary: Introduce a new annotation to say there is no BC guarantee in the inheritance Key: HBASE-19535 URL: https://issues.apache.org/jira/browse/HBASE-19535 Project: HBase Issue Type: Task Reporter: Chia-Ping Tsai Priority: Critical Fix For: 2.0.0 We have added many Public "read-only" interface in 2.0 to save user from the dangerous operations. For example, TableDescriptor, ColumnFamilyDescriptor, Region, Store, etc. However, this change make us be hard to enhance these interface (ie. add the new APIs) because the BC will be broken for the user having the subclass of these interfaces. In contrast with Cell, Filter, and coprocessor, these new interfaces are NOT designed to be a extendable class for user but we have no explicit caveat. As I see it, it is necessary to introduce an new annotation to explicitly say "you CAN extend this Public class with the BC guarantee". On the other hand, user should not extend the Public classes which don't have the such annotation. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Reopened] (HBASE-18309) Support multi threads in CleanerChore
[ https://issues.apache.org/jira/browse/HBASE-18309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai reopened HBASE-18309: > Support multi threads in CleanerChore > - > > Key: HBASE-18309 > URL: https://issues.apache.org/jira/browse/HBASE-18309 > Project: HBase > Issue Type: Improvement >Reporter: binlijin >Assignee: Reid Chan > Fix For: 3.0.0, 2.0.0-beta-1 > > Attachments: HBASE-18309.master.001.patch, > HBASE-18309.master.002.patch, HBASE-18309.master.004.patch, > HBASE-18309.master.005.patch, HBASE-18309.master.006.patch, > HBASE-18309.master.007.patch, HBASE-18309.master.008.patch, > HBASE-18309.master.009.patch, HBASE-18309.master.010.patch, > HBASE-18309.master.011.patch, HBASE-18309.master.012.patch, > space_consumption_in_archive.png > > > There is only one thread in LogCleaner to clean oldWALs and in our big > cluster we find this is not enough. The number of files under oldWALs reach > the max-directory-items limit of HDFS and cause region server crash, so we > use multi threads for LogCleaner and the crash not happened any more. > What's more, currently there's only one thread iterating the archive > directory, and we could use multiple threads cleaning sub directories in > parallel to speed it up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HBASE-19412) Fix TestBlockEvictionFromClient
[
https://issues.apache.org/jira/browse/HBASE-19412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved HBASE-19412.
Resolution: Duplicate
HBASE-19511 has resolved this issue.
> Fix TestBlockEvictionFromClient
> ---
>
> Key: HBASE-19412
> URL: https://issues.apache.org/jira/browse/HBASE-19412
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 2.0.0
>
>
> {code}
> 測試結果 (14 項失敗 / -7)
>
> org.apache.hadoop.hbase.client.TestBlockEvictionFromClient.testScanWithCompaction
>
> org.apache.hadoop.hbase.client.TestBlockEvictionFromClient.testParallelGetsAndScanWithWrappedRegionScanner
>
> org.apache.hadoop.hbase.client.TestBlockEvictionFromClient.testBlockEvictionWithParallelScans
>
> org.apache.hadoop.hbase.client.TestBlockEvictionFromClient.testGetWithCellsInDifferentFiles
>
> org.apache.hadoop.hbase.client.TestBlockEvictionFromClient.testGetWithMultipleColumnFamilies
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19531) Remove needless volatile declaration
Chia-Ping Tsai created HBASE-19531:
--
Summary: Remove needless volatile declaration
Key: HBASE-19531
URL: https://issues.apache.org/jira/browse/HBASE-19531
Project: HBase
Issue Type: Improvement
Reporter: Chia-Ping Tsai
Priority: Trivial
Fix For: 2.0.0
{code:title=ZKNamespaceManager.java}
public class ZKNamespaceManager extends ZKListener {
private static final Log LOG = LogFactory.getLog(ZKNamespaceManager.class);
private final String nsZNode;
private volatile NavigableMap cache; //here
{code}
{code:title=HBaseAdmin.java}
public class HBaseAdmin implements Admin {
private static final Log LOG = LogFactory.getLog(HBaseAdmin.class);
private ClusterConnection connection;
private volatile Configuration conf; //here
{code}
{code:title=BufferedMutatorImpl.java}
public class BufferedMutatorImpl implements BufferedMutator {
private static final Log LOG = LogFactory.getLog(BufferedMutatorImpl.class);
private final ExceptionListener listener;
private final TableName tableName;
private final Configuration conf;
private final ConcurrentLinkedQueue writeAsyncBuffer = new
ConcurrentLinkedQueue<>();
private final AtomicLong currentWriteBufferSize = new AtomicLong(0);
/**
* Count the size of {@link BufferedMutatorImpl#writeAsyncBuffer}.
* The {@link ConcurrentLinkedQueue#size()} is NOT a constant-time operation.
*/
private final AtomicInteger undealtMutationCount = new AtomicInteger(0);
private volatile long writeBufferSize; //here
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Reopened] (HBASE-19484) The value array written by ExtendedCell#write is out of bounds
[
https://issues.apache.org/jira/browse/HBASE-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai reopened HBASE-19484:
getValueArray() can return null so we should add the length check before
writing the value array
> The value array written by ExtendedCell#write is out of bounds
> --
>
> Key: HBASE-19484
> URL: https://issues.apache.org/jira/browse/HBASE-19484
> Project: HBase
> Issue Type: Bug
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Blocker
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19484.v0.patch
>
>
> I move the impl of IndividualBytesFieldCell#write to ExtendedCell so as to
> make it be a default method (see HBASE-19430), but I didn't notice that the
> value array doesn't be handled correctly.
> {code:title=ExtendedCell}
> default int write(OutputStream out, boolean withTags) throws IOException {
> // Key length and then value length
> ByteBufferUtils.putInt(out, KeyValueUtil.keyLength(this));
> ByteBufferUtils.putInt(out, getValueLength());
> // Key
> PrivateCellUtil.writeFlatKey(this, out);
> // Value
> out.write(getValueArray()); // <-- here
> // Tags length and tags byte array
> if (withTags && getTagsLength() > 0) {
> // Tags length
> out.write((byte)(0xff & (getTagsLength() >> 8)));
> out.write((byte)(0xff & getTagsLength()));
> // Tags byte array
> out.write(getTagsArray(), getTagsOffset(), getTagsLength());
> }
> return getSerializedSize(withTags);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19504) Add TimeRange support into checkAndMutate
Chia-Ping Tsai created HBASE-19504: -- Summary: Add TimeRange support into checkAndMutate Key: HBASE-19504 URL: https://issues.apache.org/jira/browse/HBASE-19504 Project: HBase Issue Type: New Feature Reporter: Chia-Ping Tsai Fix For: 2.0.0 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19502) Make sure we have closed all StoreFileScanner if we fail to open any StoreFileScanners
Chia-Ping Tsai created HBASE-19502:
--
Summary: Make sure we have closed all StoreFileScanner if we fail
to open any StoreFileScanners
Key: HBASE-19502
URL: https://issues.apache.org/jira/browse/HBASE-19502
Project: HBase
Issue Type: Sub-task
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai
{code:title=StoreFileScanner.java}
public static List
getScannersForStoreFiles(Collection files,
boolean cacheBlocks, boolean usePread, boolean isCompaction, boolean
canUseDrop,
ScanQueryMatcher matcher, long readPt, boolean isPrimaryReplica) throws
IOException {
List scanners = new
ArrayList(files.size());
List sorted_files = new ArrayList<>(files);
Collections.sort(sorted_files, StoreFile.Comparators.SEQ_ID);
for (int i = 0; i < sorted_files.size(); i++) {
StoreFile.Reader r = sorted_files.get(i).createReader(canUseDrop);
r.setReplicaStoreFile(isPrimaryReplica);
StoreFileScanner scanner = r.getStoreFileScanner(cacheBlocks, usePread,
isCompaction, readPt,
i, matcher != null ? !matcher.hasNullColumnInQuery() : false);
scanners.add(scanner);
}
return scanners;
}
{code}
The missed decrement of ref count will obstruct the cleanup of compacted files.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19496) Reusing the ByteBuffer in rpc layer corrupt the ServerLoad and RegionLoad
Chia-Ping Tsai created HBASE-19496:
--
Summary: Reusing the ByteBuffer in rpc layer corrupt the
ServerLoad and RegionLoad
Key: HBASE-19496
URL: https://issues.apache.org/jira/browse/HBASE-19496
Project: HBase
Issue Type: Bug
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai
Priority: Blocker
Fix For: 2.0.0-beta-1
{{ServerLoad}} and {{RegionLoad}} store the pb object internally but the
bytebuffer of pb object may be reused in rpc layer. Hence, the {{ServerLoad}}
and {{RegionLoad}} which saved by {{HMaster}} will be corrupted if the
bytebuffer backed is modified.
This issue doesn't happen on branch-1.
# netty server was introduced in 2.0 (see HBASE-17263)
# reusing bytebuffer to read RPC requests was introduced in 2.0 (see
HBASE-15788)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-19484) The value array written by ExtendedCell#write is out of bounds
Chia-Ping Tsai created HBASE-19484:
--
Summary: The value array written by ExtendedCell#write is out of
bounds
Key: HBASE-19484
URL: https://issues.apache.org/jira/browse/HBASE-19484
Project: HBase
Issue Type: Bug
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai
Priority: Blocker
Fix For: 2.0.0-beta-1
I move the impl of IndividualBytesFieldCell#write to ExtendedCell so as to make
it be a default method (see HBASE-19430), but I didn't notice that the value
array doesn't be handled correctly.
{code:title=ExtendedCell}
default int write(OutputStream out, boolean withTags) throws IOException {
// Key length and then value length
ByteBufferUtils.putInt(out, KeyValueUtil.keyLength(this));
ByteBufferUtils.putInt(out, getValueLength());
// Key
PrivateCellUtil.writeFlatKey(this, out);
// Value
out.write(getValueArray()); // <-- here
// Tags length and tags byte array
if (withTags && getTagsLength() > 0) {
// Tags length
out.write((byte)(0xff & (getTagsLength() >> 8)));
out.write((byte)(0xff & getTagsLength()));
// Tags byte array
out.write(getTagsArray(), getTagsOffset(), getTagsLength());
}
return getSerializedSize(withTags);
}
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (HBASE-9159) TestSplitTransaction.testFailAfterPONR fails
[
https://issues.apache.org/jira/browse/HBASE-9159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved HBASE-9159.
---
Resolution: Duplicate
see HBASE-8300 and HBASE-13585
> TestSplitTransaction.testFailAfterPONR fails
>
>
> Key: HBASE-9159
> URL: https://issues.apache.org/jira/browse/HBASE-9159
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: stack
>
> Failed here
> http://54.241.6.143/job/HBase-0.95/org.apache.hbase$hbase-server/776/testReport/junit/org.apache.hadoop.hbase.regionserver/TestSplitTransaction/testFailAfterPONR/
> Failure is that mockito did not throw the expected exception:
> {code}
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hbase.regionserver.TestSplitTransaction.testFailAfterPONR(TestSplitTransaction.java:133)
> {code}
> I've not seen this before that I remember. Keeping open a while. Adding ns
> has destabilized our clean test runs. Working on restabilization.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (HBASE-9125) TestJoinedScanners fails
[
https://issues.apache.org/jira/browse/HBASE-9125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved HBASE-9125.
---
Resolution: Duplicate
see HBASE-7606
> TestJoinedScanners fails
>
>
> Key: HBASE-9125
> URL: https://issues.apache.org/jira/browse/HBASE-9125
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>
> http://54.241.6.143/job/HBase-TRUNK/org.apache.hbase$hbase-server/445/testReport/org.apache.hadoop.hbase.regionserver/TestJoinedScanners/testJoinedScanners/
> {code}
> org.apache.hadoop.hbase.DoNotRetryIOException: Failed after retry of
> OutOfOrderScannerNextException: was there a rpc timeout?
> at
> org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:345)
> at
> org.apache.hadoop.hbase.regionserver.TestJoinedScanners.runScanner(TestJoinedScanners.java:167)
> at
> org.apache.hadoop.hbase.regionserver.TestJoinedScanners.testJoinedScanners(TestJoinedScanners.java:140)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runners.Suite.runChild(Suite.java:127)
> at org.junit.runners.Suite.runChild(Suite.java:26)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
> at java.lang.Thread.run(Thread.java:662)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (HBASE-9072) TestSnapshotFromMaster.testSnapshotHFileArchiving fails
[
https://issues.apache.org/jira/browse/HBASE-9072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved HBASE-9072.
---
Resolution: Duplicate
see HBASE-16293
> TestSnapshotFromMaster.testSnapshotHFileArchiving fails
> ---
>
> Key: HBASE-9072
> URL: https://issues.apache.org/jira/browse/HBASE-9072
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: stack
> Attachments: HBASE-9072-fix0.patch
>
>
> http://54.241.6.143/job/HBase-TRUNK-Hadoop-2/org.apache.hbase$hbase-server/428/testReport/junit/org.apache.hadoop.hbase.master.cleaner/TestSnapshotFromMaster/testSnapshotHFileArchiving/
> {code}
> Failed 4600 actions: SocketTimeoutException: 4600 times,
> Stacktrace
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:794)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:756)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.loadTable(HBaseTestingUtility.java:1312)
> at
> org.apache.hadoop.hbase.master.cleaner.TestSnapshotFromMaster.testSnapshotHFileArchiving(TestSnapshotFromMaster.java:297)
> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed
> 4600 actions: SocketTimeoutException: 4600 times,
> at
> org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:157)
> at
> org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$500(AsyncProcess.java:145)
> at
> org.apache.hadoop.hbase.client.AsyncProcess.getErrors(AsyncProcess.java:700)
> at
> org.apache.hadoop.hbase.client.HTable.backgroundFlushCommits(HTable.java:827)
> at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:794)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:756)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.loadTable(HBaseTestingUtility.java:1312)
> at
> org.apache.hadoop.hbase.master.cleaner.TestSnapshotFromMaster.testSnapshotHFileArchiving(TestSnapshotFromMaster.java:297)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runners.Suite.runChild(Suite.java:127)
> at org.junit.runners.Suite.runChild(Suite.java:26)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
> at java.lang.Thread.run(Thread.java:662)
> {code}
> [~mbertozzi] Opinion?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (HBASE-9024) TestLogRolling fails/goes zombie
[
https://issues.apache.org/jira/browse/HBASE-9024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved HBASE-9024.
---
Resolution: Duplicate
see HBASE-10142
> TestLogRolling fails/goes zombie
>
>
> Key: HBASE-9024
> URL: https://issues.apache.org/jira/browse/HBASE-9024
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: stack
>Priority: Critical
>
> TestLogRolling.testLogRollOnPipelineRestart failed on hadoop1 here:
> https://builds.apache.org/job/hbase-0.95/352/consoleText It went zombie.
> In the double thread dump on the end:
> {code}
> "pool-1-thread-1" prio=10 tid=0x73f9dc00 nid=0x3a34 in Object.wait()
> [0x7517d000]
>java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0xcf624ad0> (a java.util.concurrent.atomic.AtomicLong)
> at
> org.apache.hadoop.hbase.client.AsyncProcess.waitForNextTaskDone(AsyncProcess.java:634)
> - locked <0xcf624ad0> (a java.util.concurrent.atomic.AtomicLong)
> at
> org.apache.hadoop.hbase.client.AsyncProcess.waitForMaximumCurrentTasks(AsyncProcess.java:659)
> at
> org.apache.hadoop.hbase.client.AsyncProcess.waitUntilDone(AsyncProcess.java:670)
> at
> org.apache.hadoop.hbase.client.HTable.backgroundFlushCommits(HTable.java:813)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1170)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:753)
> at
> org.apache.hadoop.hbase.regionserver.wal.TestLogRolling.doPut(TestLogRolling.java:640)
> at
> org.apache.hadoop.hbase.regionserver.wal.TestLogRolling.writeData(TestLogRolling.java:248)
> at
> org.apache.hadoop.hbase.regionserver.wal.TestLogRolling.testLogRollOnPipelineRestart(TestLogRolling.java:515)
> {code}
> ... we are stuck here.
> The math looks like it could go wonky. But looking in the output for the
> test, it seems that when this test ran we got this:
> {code}
> 2013-07-23 01:23:29,560 INFO [pool-1-thread-1]
> hbase.HBaseTestingUtility(922): Minicluster is down
> 2013-07-23 01:23:29,574 INFO [pool-1-thread-1] hbase.ResourceChecker(171):
> after: regionserver.wal.TestLogRolling#testLogRollOnPipelineRestart Thread=39
> (was 31) - Thread LEAK? -, OpenFileDescriptor=312 (was 272) -
> OpenFileDescriptor LEAK? -, MaxFileDescriptor=4 (was 4),
> SystemLoadAverage=351 (was 368), ProcessCount=144 (was 142) - ProcessCount
> LEAK? -, AvailableMemoryMB=906 (was 1995), ConnectionCount=0 (was 0)
> {code}
> This test has a history of failures. See HBASE-5995 where it was fixed and
> reenabled once. Thought was that it was a hadoop2 issue but this cited
> failure is on hadoop1.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (HBASE-9022) TestHLogSplit.testIOEOnOutputThread fails
[ https://issues.apache.org/jira/browse/HBASE-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved HBASE-9022. --- Resolution: Won't Fix TestHLogSplit had been removed. see HBASE-10378 and HBASE-12522 > TestHLogSplit.testIOEOnOutputThread fails > - > > Key: HBASE-9022 > URL: https://issues.apache.org/jira/browse/HBASE-9022 > Project: HBase > Issue Type: Bug >Reporter: stack > Attachments: 9022.addthreaddumping-part2.txt, > 9022.addthreaddumping.txt, 9022.fix.txt, 9022.txt, 9022.txt, 9022.txt, > 9022.yet-more-logging.txt, 9022.yet-more-logging2.txt > > > https://builds.apache.org/job/PreCommit-HBASE-Build/6428//testReport/org.apache.hadoop.hbase.regionserver.wal/TestHLogSplit/testIOEOnOutputThread/ -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19462) Deprecate all addImmutable in Put
Chia-Ping Tsai created HBASE-19462:
--
Summary: Deprecate all addImmutable in Put
Key: HBASE-19462
URL: https://issues.apache.org/jira/browse/HBASE-19462
Project: HBase
Issue Type: Sub-task
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai
Users are able to use {{CellBuilder}} to build the cell without array copy for
Put/Delete/Increment/Append, and we always do the copy if user pass the
{{ByteBuffer}}. Hence, the {{addImmutable}} is unnecessary.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Reopened] (HBASE-18112) Write RequestTooBigException back to client for NettyRpcServer
[ https://issues.apache.org/jira/browse/HBASE-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai reopened HBASE-18112: > Write RequestTooBigException back to client for NettyRpcServer > -- > > Key: HBASE-18112 > URL: https://issues.apache.org/jira/browse/HBASE-18112 > Project: HBase > Issue Type: Sub-task > Components: IPC/RPC >Reporter: Duo Zhang >Assignee: Toshihiro Suzuki > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-18112-v2.patch, HBASE-18112-v2.patch, > HBASE-18112-v2.patch, HBASE-18112-v3.patch, HBASE-18112-v3.patch, > HBASE-18112-v4.patch, HBASE-18112-v4.patch, HBASE-18112-v5.patch, > HBASE-18112.patch > > > For now we just close the connection so NettyRpcServer can not pass TestIPC. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19437) Wrap the null result from postAppend/postIncrement
Chia-Ping Tsai created HBASE-19437: -- Summary: Wrap the null result from postAppend/postIncrement Key: HBASE-19437 URL: https://issues.apache.org/jira/browse/HBASE-19437 Project: HBase Issue Type: Bug Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai The null from postAppend/postIncrement sparks inconsistent behavior. The Table#append and #increment can handle the null result but the batch operation throws IllegalStateException. We should wrap the null from postAppend/postIncrement to empty result since cp user should throw IOException rather than return null if they intend to say something is broken. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19431) The tag array written by IndividualBytesFieldCell#write is out of bounds
Chia-Ping Tsai created HBASE-19431:
--
Summary: The tag array written by IndividualBytesFieldCell#write
is out of bounds
Key: HBASE-19431
URL: https://issues.apache.org/jira/browse/HBASE-19431
Project: HBase
Issue Type: Bug
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai
Priority: Critical
Fix For: 2.0.0-beta-1
{code:title=IndividualBytesFieldCell.java}
@Override
public int write(OutputStream out, boolean withTags) throws IOException {
// Key length and then value length
ByteBufferUtils.putInt(out, KeyValueUtil.keyLength(this));
ByteBufferUtils.putInt(out, getValueLength());
// Key
PrivateCellUtil.writeFlatKey(this, out);
// Value
out.write(getValueArray());
// Tags length and tags byte array
if (withTags && getTagsLength() > 0) {
// Tags length
out.write((byte)(0xff & (getTagsLength() >> 8)));
out.write((byte)(0xff & getTagsLength()));
// Tags byte array
out.write(tags); // here
}
return getSerializedSize(withTags);
}
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
