[jira] [Resolved] (HBASE-26062) SIGSEGV in AsyncFSWAL consume

[Michael Stack (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-26062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Stack resolved HBASE-26062.
---
Resolution: Duplicate

Thanks [~anoop.hbase]  There is ASYNC_WAL on this cluster afterall (when I 
wrote the above, thought there was none). Resolving as duplicate of what we see 
over on HBASE-24984

> SIGSEGV in AsyncFSWAL consume
> -
>
> Key: HBASE-26062
> URL: https://issues.apache.org/jira/browse/HBASE-26062
> Project: HBase
>  Issue Type: Bug
>Reporter: Michael Stack
>Priority: Major
>
> Seems related to the parent issue. Its happened a few times on one of our 
> clusters here. Below are two examples. Need more detail but perhaps the call 
> has timed out, the buffer has thus been freed, but the late consume on the 
> other side of the ringbuffer doesn't know that and goes ahead (Just 
> speculation).
>  
> {code:java}
> #  SIGSEGV (0xb) at pc=0x7f8b3ef5b77c, pid=37631, tid=0x7f61560ed700
> RAX=0xdf6e is an unknown valueRBX=0x7f8a38d7b6f8 is an 
> oopjava.nio.DirectByteBuffer - klass: 
> 'java/nio/DirectByteBuffer'RCX=0x7f60e2767898 is pointing into 
> metadataRDX=0x0de7 is an unknown valueRSP=0x7f61560ec6f0 is 
> pointing into the stack for thread: 0x7f8b3017b800RBP=[error occurred 
> during error reporting (printing register info), id 0xb]
> Stack: [0x7f6155fed000,0x7f61560ee000],  sp=0x7f61560ec6f0,  free 
> space=1021kNative frames: (J=compiled Java code, j=interpreted, Vv=VM code, 
> C=native code)J 23901 C2 
> java.util.stream.MatchOps$1MatchSink.accept(Ljava/lang/Object;)V (44 bytes) @ 
> 0x7f8b3ef5b77c [0x7f8b3ef5b640+0x13c]J 16165 C2 
> java.util.ArrayList$ArrayListSpliterator.tryAdvance(Ljava/util/function/Consumer;)Z
>  (79 bytes) @ 0x7f8b3d67b344 [0x7f8b3d67b2c0+0x84]J 16160 C2 
> java.util.stream.MatchOps$MatchOp.evaluateSequential(Ljava/util/stream/PipelineHelper;Ljava/util/Spliterator;)Ljava/lang/Object;
>  (7 bytes) @ 0x7f8b3d67bc9c [0x7f8b3d67b900+0x39c]J 17729 C2 
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALActionListener.visitLogEntryBeforeWrite(Lorg/apache/hadoop/hbase/wal/WALKey;Lorg/apache/hadoop/hbase/wal/WALEdit;)V
>  (10 bytes) @ 0x7f8b3fc39010 [0x7f8b3fc388a0+0x770]J 29991 C2 
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.appendAndSync()V (261 
> bytes) @ 0x7f8b3fd03d90 [0x7f8b3fd039e0+0x3b0]J 20773 C2 
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.consume()V (474 bytes) @ 
> 0x7f8b40283728 [0x7f8b40283480+0x2a8]J 15191 C2 
> org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL$$Lambda$76.run()V (8 
> bytes) @ 0x7f8b3ed69ecc [0x7f8b3ed69ea0+0x2c]J 17383% C2 
> java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V
>  (225 bytes) @ 0x7f8b3d9423f8 [0x7f8b3d942260+0x198]j  
> java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5j  
> java.lang.Thread.run()V+11v  ~StubRoutines::call_stubV  [libjvm.so+0x66b9ba]  
> JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*, 
> Thread*)+0xe1aV  [libjvm.so+0x669073]  JavaCalls::call_virtual(JavaValue*, 
> KlassHandle, Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x263V  
> [libjvm.so+0x669647]  JavaCalls::call_virtual(JavaValue*, Handle, 
> KlassHandle, Symbol*, Symbol*, Thread*)+0x57V  [libjvm.so+0x6aaa4c]  
> thread_entry(JavaThread*, Thread*)+0x6cV  [libjvm.so+0xa224cb]  
> JavaThread::thread_main_inner()+0xdbV  [libjvm.so+0xa22816]  
> JavaThread::run()+0x316V  [libjvm.so+0x8c4202]  java_start(Thread*)+0x102C  
> [libpthread.so.0+0x76ba]  start_thread+0xca {code}
>  
> This one is from a month previous and has a deeper stack... we're trying to 
> read a Cell...
>  
> {code:java}
> Stack: [0x7fa1d5fb8000,0x7fa1d60b9000],  sp=0x7fa1d60b7660,  free 
> space=1021kNative frames: (J=compiled Java code, j=interpreted, Vv=VM code, 
> C=native code)J 30665 C2 
> org.apache.hadoop.hbase.PrivateCellUtil.matchingFamily(Lorg/apache/hadoop/hbase/Cell;[BII)Z
>  (59 bytes) @ 0x7fcc2d29eeb2 [0x7fcc2d29e7c0+0x6f2]J 25816 C2 
> org.apache.hadoop.hbase.CellUtil.matchingFamily(Lorg/apache/hadoop/hbase/Cell;[B)Z
>  (28 bytes) @ 0x7fcc2a0430f8 [0x7fcc2a0430e0+0x18]J 17236 C2 
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALActionListener$$Lambda$254.test(Ljava/lang/Object;)Z
>  (8 bytes) @ 0x7fcc2b40bc68 [0x7fcc2b40bc20+0x48]J 13735 C2 
> java.util.ArrayList$ArrayListSpliterator.tryAdvance(Ljava/util/function/Consumer;)Z
>  (79 bytes) @ 0x7fcc2b7d936c [0x7fcc2b7d92c0+0xac]J 17162 C2 
> java.util.stream.MatchOps$MatchOp.evaluateSequential(Ljava/util/stream/PipelineHelper;Ljava/util/Spliterator;)Ljava/lang/Object;
>  (7 bytes) @ 0x7fcc29bc05e8 [0x7fcc29

[jira] [Created] (HBASE-26111) Release 3.0.0-alpha-2

[Duo Zhang (Jira)]

Duo Zhang created HBASE-26111:
-

 Summary: Release 3.0.0-alpha-2
 Key: HBASE-26111
 URL: https://issues.apache.org/jira/browse/HBASE-26111
 Project: HBase
  Issue Type: Umbrella
Reporter: Duo Zhang






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HBASE-26052) Release 3.0.0-alpha-1

[Duo Zhang (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-26052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang resolved HBASE-26052.
---
Resolution: Fixed

> Release 3.0.0-alpha-1
> -
>
> Key: HBASE-26052
> URL: https://issues.apache.org/jira/browse/HBASE-26052
> Project: HBase
>  Issue Type: Umbrella
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-26110) Add download links for 1.7.1

[Bharath Vissapragada (Jira)]

Bharath Vissapragada created HBASE-26110:


 Summary: Add download links for 1.7.1
 Key: HBASE-26110
 URL: https://issues.apache.org/jira/browse/HBASE-26110
 Project: HBase
  Issue Type: Sub-task
  Components: website
Affects Versions: 1.7.1
Reporter: Bharath Vissapragada
Assignee: Bharath Vissapragada
 Fix For: 1.7.1






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [NOTICE] HBaseTestingUtility is now deprecated, starting from 3.0.0

[Geoffrey Jacoby]

It would of course be possible to solve any particular gap with a special
LP method or interface. I see three problems with that:

1. These special-purpose methods would increase in number over time, and
make the new interface less clean and more cluttered. I really like how
clean it is right now.
2. This requires us to know in advance what special-purpose methods are
needed. There are probably other developers out there with a very different
set of testing needs than mine that I'd never think of. This is
particularly difficult in an Apache-licensed project, where APIs are built
in public but often consumed in private organizations.
3. Some developers who need special-purpose LP methods won't be paying
attention to this thread, or only realize a gap later when building a new
feature, and will have to file a JIRA and either wait for the next HBase
release or just use the old minicluster anyway. That's not a good developer
experience.

Providing a normal mode (the new interface) and an "expert" mode (the
pointer to the inner HBaseTestingUtil for server-side code testing) keeps
the interface clean with just a single additional accessor, and allows
developers to adopt the new interface with confidence, knowing that if
there's a gap they can always work around it easily without a JIRA. Marking
the expert mode LP(COPROC, REPLICATION) makes it clear that it _is_ an
expert mode, just for server-side testing.

Or if you prefer, there could be a ServerTestingHBaseCluster interface that
extends TestingHBaseCluster and adds the extra HBTU accessor, with an
LP(COPROC, REPLICATION).

Could you please explain more about what you don't like about exposing HBTU
even for limited cases? There seems to be an assumption that downstream
developers will follow directions for an IA.Private on HBTU by not using
it, but not follow directions for IA.Private annotations on classes
reachable from HBTU?

Thanks,

Geoffrey



On Wed, Jul 21, 2021 at 1:21 AM 张铎(Duo Zhang)  wrote:

> Inline
>
> Geoffrey Jacoby  于2021年7月21日周三 上午4:43写道：
>
> > Thanks for creating the JIRA for adding the Configuration object.
> >
> > To make the discussion more concrete, I did an initial pass through many
> of
> > the minicluster tests at my dayjob, and here are the gaps I've found so
> > far. These don't count many cases where convenience methods on the HBTU
> are
> > used, but where the tests could be easily modified to use existing client
> > APIs instead.
> >
> > * An enormous number of both internal and Phoenix tests execute MapReduce
> > jobs (for example, Phoenix builds its secondary indexes via MapReduce).
> It
> > doesn't appear possible to run MapReduce jobs through the new testing
> > framework. (I filed HBASE-26102 for this.)
> >
> Usually, you do not need to start a mini MR client for running MR job, it
> will execute locally. Most HBase tests in hbase-mapreduce module do not
> start an actual mini MR cluster.
> And the current MiniMRCluster has already been deprecated in hadoop...
>
> >
> > * A test which uses two miniclusters which share the same mini-ZK quorum
> > under different znodes
> >
> What is the goal for these tests? In HBase, we do this when testing
> replication, but I think it is also OK to use different zk clusters? Unless
> there are problems when starting two zk clusters, then we should try to fix
> this problem.
>
> >
> > * An end-to-end test that verifies that a batch job that appends Cell
> Tags
> > to Delete markers it creates via a coproc in fact writes the Tags. This
> > requires getting a reference to the Region from the minicluster and
> > creating a RegionScanner because the HBase client APIs expressly prevent
> > reading Cell Tags. Supporting this would go against the
> > TestingHBaseCluster's design philosophy, so the old minicluster is still
> > necessary for it.
> >
> This is indeed a problem. We should find a way to let users verify the
> tags, as we do not leak it to client side. We could have something with
> IA.LP to expose the Region interface, maybe.
>
> >
> > So far, I think that with HBASE-26098 and HBASE-26102 the overwhelming
> > majority of internal tests would be fine with the new streamlined
> > TestingHBaseCluster. But not all of them, and there are valid reasons a
> > downstream developer might sometimes need the lower-level control the
> HBTU
> > gives.
> >
> > That's why I still believe that the premise that only HBase and Phoenix
> > need to directly test server-side components is incorrect, because
> > _downstream developers can create server-side components_ like coprocs
> and
> > replication endpoints.
> >
> > My suggested solution:
> > 1. Expose the inner HBaseTestingUtil in TestingHBaseClusterImpl on the
> > TestingHBaseCluster interface as getTestingUtil()
> > 2. Change the InterfaceAudience for HBaseTestingUtil and its related
> > classes to IA.LimitedPrivate(COPROC, REPLICATION) to indicate that these
> > are intended for limited use cases testing server-side integration logic.
> > As Phoen

[jira] [Created] (HBASE-26109) Wrap up 1.7.1 release

[Bharath Vissapragada (Jira)]

Bharath Vissapragada created HBASE-26109:


 Summary: Wrap up 1.7.1 release
 Key: HBASE-26109
 URL: https://issues.apache.org/jira/browse/HBASE-26109
 Project: HBase
  Issue Type: Task
  Components: documentation
Reporter: Bharath Vissapragada
Assignee: Bharath Vissapragada
 Fix For: 1.7.1


- Set branch-1 version to 1.7.2-SNAPSHOT
- Update SVN release bits, add download links to 1.7.1
- Remove 1.7.0 from the mirrors
- Commit HBASE-26071

in that order.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [VOTE] The Release Candidate for HBase 1.7.1 (RC0) is available for download

[Bharath Vissapragada]

Thanks Andrew, Reid and Viraj for voting. With 3 binding +1s and no -1s,
the vote has passed. I'll update the bits, documentation and send out a
notice.

On Mon, Jul 19, 2021 at 10:30 AM Viraj Jasani  wrote:

> +1
>
> * Signature: ok
> * Checksum : ok
> * Rat check (1.8.0_251): ok
>  - mvn clean apache-rat:check
> * Built from source (1.8.0_251): ok
>  - mvn clean install  -DskipTests
> * Unit tests pass (1.8.0_251): ok
>  - mvn package -P runSmallTests  -Dsurefire.rerunFailingTestsCount=3
> * Nightly builds look clean
>
>
> On 2021/07/16 16:59:52, Bharath Vissapragada  wrote:
> > Hi everyone,
> >
> > The first release candidate (RC0) for HBase 1.7.1 is available for
> download:
> > https://dist.apache.org/repos/dist/dev/hbase/1.7.1RC0/
> >
> > Release tag:
> > https://github.com/apache/hbase/tree/1.7.1RC0
> >
> > Maven artifacts are available in a staging repository at:
> > https://repository.apache.org/content/repositories/orgapachehbase-1458/
> >
> > A detailed source and binary compatibility report for this release is at:
> >
> https://dist.apache.org/repos/dist/dev/hbase/1.7.1RC0/api_compare_1.7.0_to_1.7.1-RC0.html
> >
> > Release notes can be found at:
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12350241&styleName=html&projectId=12310753&Create=Create&atl_token=A5KQ-2QAV-T4JA-FDED_2640918bc7c9809c3a16f01bd099bb435d5f87ff_lin
> >
> > Current flaky test report is at:
> >
> https://ci-hadoop.apache.org/job/HBase/job/HBase-Find-Flaky-Tests/job/branch-1/Flaky_20Test_20Report/
> >
> > All the artifacts were signed using my key DC190FA051EE6466 at
> > https://dist.apache.org/repos/dist/release/hbase/KEYS
> >
> > Note: Trigger for this release is HBASE-26021
> >  that addresses an
> > incompatible table serialization added to 1.7.0. Once the vote on an
> 1.7.1
> > RC passes, 1.7.0 will be withdrawn in favor of 1.7.1+. Fixes in this jira
> > (revert of incompatible serialization) are reflected in the compat-check
> > report API changes linked above.
> >
> > Please try out this candidate and vote on it. The vote remains for at
> least
> > 72 working hours. *Since we are approaching a weekend here in the US (and
> > is already a weekend on the other side of the globe), I'll check back mid
> > of next week.*
> >
> > [ ] +1 Release this package as Apache HBase 1.7.1
> > [ ] -1 Do not release this package because ...
> >
> > Thanks,
> > Bharath.
> >
>

[jira] [Created] (HBASE-26108) add option to disable scanMetrics in TableSnapshotInputFormat

[Huaxiang Sun (Jira)]

Huaxiang Sun created HBASE-26108:


 Summary: add option to disable scanMetrics in 
TableSnapshotInputFormat
 Key: HBASE-26108
 URL: https://issues.apache.org/jira/browse/HBASE-26108
 Project: HBase
  Issue Type: Improvement
Affects Versions: 2.3.5
Reporter: Huaxiang Sun
Assignee: Huaxiang Sun


When running spark job with TableSnapshotInputFormat, we found that scan is 
very slower. We found that scanMetrics is hardcoded as enabled, spark's 

newAPIHadoopRDD uses DummyReporter in hadoop, which causes the following 
exception and 80% cpu time is spent on this exception handling. 

Need to provide an option to disable scanMetrics.
java.base@11.0.5/java.lang.Throwable.fillInStackTrace(Native Method)
java.base@11.0.5/java.lang.Throwable.fillInStackTrace(Throwable.java:787) => 
holding Monitor(java.util.MissingResourceException@258206255})
java.base@11.0.5/java.lang.Throwable.(Throwable.java:292)
java.base@11.0.5/java.lang.Exception.(Exception.java:84)
java.base@11.0.5/java.lang.RuntimeException.(RuntimeException.java:80)
java.base@11.0.5/java.util.MissingResourceException.(MissingResourceException.java:85)
java.base@11.0.5/java.util.ResourceBundle.throwMissingResourceException(ResourceBundle.java:2055)
java.base@11.0.5/java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1689)
java.base@11.0.5/java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1593)
java.base@11.0.5/java.util.ResourceBundle.getBundle(ResourceBundle.java:1284)
app//org.apache.hadoop.mapreduce.util.ResourceBundles.getBundle(ResourceBundles.java:37)
app//org.apache.hadoop.mapreduce.util.ResourceBundles.getValue(ResourceBundles.java:56)
 => holding Monitor(java.lang.Class@545605549})
app//org.apache.hadoop.mapreduce.util.ResourceBundles.getCounterGroupName(ResourceBundles.java:77)
app//org.apache.hadoop.mapreduce.counters.CounterGroupFactory.newGroup(CounterGroupFactory.java:94)
app//org.apache.hadoop.mapreduce.counters.AbstractCounters.getGroup(AbstractCounters.java:227)
app//org.apache.hadoop.mapreduce.counters.AbstractCounters.findCounter(AbstractCounters.java:154)
app//org.apache.hadoop.mapreduce.task.TaskAttemptContextImpl$DummyReporter.getCounter(TaskAttemptContextImpl.java:110)
app//org.apache.hadoop.mapreduce.task.TaskAttemptContextImpl.getCounter(TaskAttemptContextImpl.java:76)
org.apache.hadoop.hbase.mapreduce.TableRecordReaderImpl.updateCounters(TableRecordReaderImpl.java:311)
org.apache.hadoop.hbase.mapreduce.TableSnapshotInputFormat$TableSnapshotRegionRecordReader.nextKeyValue(TableSnapshotInputFormat.java:167)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-26107) MOB compaction with missing files catches incorrect exception

[Peter Somogyi (Jira)]

Peter Somogyi created HBASE-26107:
-

 Summary: MOB compaction with missing files catches incorrect 
exception
 Key: HBASE-26107
 URL: https://issues.apache.org/jira/browse/HBASE-26107
 Project: HBase
  Issue Type: Bug
  Components: mob
Affects Versions: 3.0.0-alpha-1
Reporter: Peter Somogyi
Assignee: Peter Somogyi


The MOB compaction catches FileNotFoundException when 
{{hbase.unsafe.mob.discard.miss}} is true to handle missing MOB cells. The FNFE 
is wrapped in DoNotRetryIOException so the compaction fails for the given 
region.

{noformat}
2021-07-21 13:51:05,880 WARN 
org.apache.hadoop.hbase.mob.DefaultMobStoreCompactor: 
hbase.unsafe.mob.discard.miss=true. This is unsafe setting recommended only 
when first upgrading to a version with the distributed mob compaction feature 
on a cluster that has experienced MOB data corruption.
2021-07-21 13:51:05,880 WARN 
org.apache.hadoop.hbase.mob.DefaultMobStoreCompactor: 
hbase.unsafe.mob.discard.miss=true. This is unsafe setting recommended only 
when first upgrading to a version with the distributed mob compaction feature 
on a cluster that has experienced MOB data corruption.
2021-07-21 13:51:05,880 INFO 
org.apache.hadoop.hbase.mob.DefaultMobStoreCompactor: Compact MOB=true 
optimized configured=false optimized enabled=false maximum MOB file 
size=1073741824 major=true store=[table=IntegrationTestIngestWithMOB 
family=test_cf region=3a2ee81f9244c39ba61d694e616c1a89]
2021-07-21 13:51:05,880 INFO 
org.apache.hadoop.hbase.mob.DefaultMobStoreCompactor: Compact MOB=true 
optimized configured=false optimized enabled=false maximum MOB file 
size=1073741824 major=true store=[table=IntegrationTestIngestWithMOB 
family=test_cf region=7a96f55bb9ae04500a06cbaef02da6a3]
2021-07-21 13:51:05,888 INFO 
org.apache.hadoop.hbase.regionserver.RSRpcServices: Compacting 
IntegrationTestIngestWithMOB,,1626787996628.c71cad04514b17ee86a407490bd27424.
2021-07-21 13:51:05,891 INFO 
org.apache.hadoop.hbase.regionserver.RSRpcServices: Compacting 
IntegrationTestIngestWithMOB,,1626787996628.8fd002bda07755decda67b7084d1e0f6.
2021-07-21 13:51:05,895 ERROR org.apache.hadoop.hbase.regionserver.HMobStore: 
The mob file 
1bbd886460827015e5d605ed44252251202107200e5065290b424e38992f5556d9943b6a_7a96f55bb9ae04500a06cbaef02da6a3
 could not be found in the locations 
[hdfs://example.com:8020/hbase/mobdir/data/default/IntegrationTestIngestWithMOB/e9b5d936e7f55a4f1c3246a8d5ce5
3c2/test_cf, 
hdfs://example.com:8020/hbase/archive/data/default/IntegrationTestIngestWithMOB/e9b5d936e7f55a4f1c3246a8d5ce53c2/test_cf]
 or it is corrupt
2021-07-21 13:51:05,895 INFO 
org.apache.hadoop.hbase.regionserver.throttle.PressureAwareThroughputController:
 7a96f55bb9ae04500a06cbaef02da6a3#test_cf#compaction#1 average throughput is 
0.07 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active 
operations remaining, total limit is 10.00 MB/second
2021-07-21 13:51:05,908 INFO 
org.apache.hadoop.hbase.regionserver.RSRpcServices: Compacting 
IntegrationTestIngestWithMOB,,1626787996628.53186ca5008e3a964eee5f96ee3f1b26.
2021-07-21 13:51:05,997 ERROR 
org.apache.hadoop.hbase.regionserver.CompactSplit: Compaction failed 
Request=regionName=IntegrationTestIngestWithMOB,,1626787996628.7a96f55bb9ae04500a06cbaef02da6a3.,
 storeName=test_cf, fileCount=1, fileSize=110.6 M (110.6 M), priority=1, 
time=1626875465819
java.io.IOException: Mob compaction failed for region: 
7a96f55bb9ae04500a06cbaef02da6a3
at 
org.apache.hadoop.hbase.mob.DefaultMobStoreCompactor.performCompaction(DefaultMobStoreCompactor.java:575)
at 
org.apache.hadoop.hbase.regionserver.compactions.Compactor.compact(Compactor.java:327)
at 
org.apache.hadoop.hbase.mob.DefaultMobStoreCompactor.compact(DefaultMobStoreCompactor.java:227)
at 
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:126)
at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1407)
at 
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:2183)
at 
org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.doCompaction(CompactSplit.java:633)
at 
org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.run(CompactSplit.java:675)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.hadoop.hbase.DoNotRetryIOException: 
java.io.FileNotFoundException: File does not exist: 
hdfs://example.com:8020/hbase/archive/data/default/IntegrationTestIngestWithMOB/e9b5d936e7f55a4f1c3246a8d5ce53c2/test_cf/1bbd886460827015e5d605ed44252251202107200e5065290b424e38992f5556d9943b6a_7a9

[jira] [Created] (HBASE-26106) AbstractFSWALProvider#getArchivedLogPath doesn't look for wal file in all oldWALs directory.

[Rushabh Shah (Jira)]

Rushabh Shah created HBASE-26106:


 Summary: AbstractFSWALProvider#getArchivedLogPath doesn't look for 
wal file in all oldWALs directory.
 Key: HBASE-26106
 URL: https://issues.apache.org/jira/browse/HBASE-26106
 Project: HBase
  Issue Type: Bug
  Components: wal
Affects Versions: 2.4.4, 3.0.0-alpha-1, 2.5.0
Reporter: Rushabh Shah
Assignee: Rushabh Shah


Below is the code for AbstractFSWALProvider#getArchivedLogPath

{code:java}
 public static Path getArchivedLogPath(Path path, Configuration conf) throws 
IOException {
Path rootDir = CommonFSUtils.getWALRootDir(conf);
Path oldLogDir = new Path(rootDir, HConstants.HREGION_OLDLOGDIR_NAME);
if (conf.getBoolean(SEPARATE_OLDLOGDIR, DEFAULT_SEPARATE_OLDLOGDIR)) {
  ServerName serverName = getServerNameFromWALDirectoryName(path);
  if (serverName == null) {
LOG.error("Couldn't locate log: " + path);
return path;
  }
  oldLogDir = new Path(oldLogDir, serverName.getServerName());
}
Path archivedLogLocation = new Path(oldLogDir, path.getName());
final FileSystem fs = CommonFSUtils.getWALFileSystem(conf);

if (fs.exists(archivedLogLocation)) {
  LOG.info("Log " + path + " was moved to " + archivedLogLocation);
  return archivedLogLocation;
} else {
  LOG.error("Couldn't locate log: " + path);
  return path;
}
  }
{code}

This method is called from the following places.
[AbstractFSWALProvider#openReader|https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/AbstractFSWALProvider.java#L524]

[ReplicationSource#getFileSize|https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java#L399]

[WALInputFormat.WALRecordReader#nextKeyValue|https://github.com/apache/hbase/blob/master/hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/WALInputFormat.java#L220]

All of the above calls are trying to find the log in archive path after they 
couldn't locate the wal in walsDir and they are not used for moving a log file 
to archive directory.
But we will look for archive path within serverName directory only if conf key 
is true.
Cc [~zhangduo] 





--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HBASE-25807) Move method reportProcedureDone from RegionServerStatus.proto to Master.proto

[Duo Zhang (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-25807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang resolved HBASE-25807.
---
Fix Version/s: (was: 3.0.0-alpha-1)
   Resolution: Fixed

> Move method reportProcedureDone from RegionServerStatus.proto to Master.proto
> -
>
> Key: HBASE-25807
> URL: https://issues.apache.org/jira/browse/HBASE-25807
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Sun Xin
>Assignee: Sun Xin
>Priority: Major
>
> We next need use the procedure mechanism to implement enable/disable/refresh 
> peer, and  ReplicationServer also needs reportProcedureDone to master, so I 
> hope to move method reportProcedureDone to Master.proto from 
> RegionServerStatus.proto.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HBASE-22184) [security] Support get|set LogLevel in HTTPS mode

[Duo Zhang (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-22184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang resolved HBASE-22184.
---
Resolution: Fixed

> [security] Support get|set LogLevel in HTTPS mode
> -
>
> Key: HBASE-22184
> URL: https://issues.apache.org/jira/browse/HBASE-22184
> Project: HBase
>  Issue Type: New Feature
>  Components: logging, security, website
>Reporter: Reid Chan
>Assignee: Wei-Chiu Chuang
>Priority: Major
>  Labels: security
> Fix For: 2.3.0, 1.4.10, 3.0.0-alpha-1
>
> Attachments: HBASE-22184.branch-1.001.patch, 
> HBASE-22184.branch-1.002.patch, HBASE-22184.master.001.patch, 
> HBASE-22184.master.002.patch, HBASE-22184.master.003.patch
>
>
> As title read.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Reopened] (HBASE-22184) [security] Support get|set LogLevel in HTTPS mode

[Duo Zhang (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-22184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang reopened HBASE-22184:
---

> [security] Support get|set LogLevel in HTTPS mode
> -
>
> Key: HBASE-22184
> URL: https://issues.apache.org/jira/browse/HBASE-22184
> Project: HBase
>  Issue Type: New Feature
>  Components: logging, security, website
>Reporter: Reid Chan
>Assignee: Wei-Chiu Chuang
>Priority: Major
>  Labels: security
> Fix For: 3.0.0-alpha-1, 1.4.10, 2.3.0
>
> Attachments: HBASE-22184.branch-1.001.patch, 
> HBASE-22184.branch-1.002.patch, HBASE-22184.master.001.patch, 
> HBASE-22184.master.002.patch, HBASE-22184.master.003.patch
>
>
> As title read.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Reopened] (HBASE-25807) Move method reportProcedureDone from RegionServerStatus.proto to Master.proto

[Duo Zhang (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-25807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang reopened HBASE-25807:
---

> Move method reportProcedureDone from RegionServerStatus.proto to Master.proto
> -
>
> Key: HBASE-25807
> URL: https://issues.apache.org/jira/browse/HBASE-25807
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Sun Xin
>Assignee: Sun Xin
>Priority: Major
> Fix For: 3.0.0-alpha-1
>
>
> We next need use the procedure mechanism to implement enable/disable/refresh 
> peer, and  ReplicationServer also needs reportProcedureDone to master, so I 
> hope to move method reportProcedureDone to Master.proto from 
> RegionServerStatus.proto.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HBASE-26025) Add a flag to mark if the IOError can be solved by retry in thrift IOError

[Duo Zhang (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-26025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang resolved HBASE-26025.
---
Resolution: Fixed

> Add a flag to mark if the IOError can be solved by retry in thrift IOError
> --
>
> Key: HBASE-26025
> URL: https://issues.apache.org/jira/browse/HBASE-26025
> Project: HBase
>  Issue Type: Improvement
>  Components: Thrift
>Reporter: Yutong Xiao
>Assignee: Yutong Xiao
>Priority: Major
> Fix For: 2.5.0, 2.3.6, 1.7.1, 2.4.5, 3.0.0-alpha-1
>
>
> Currently, if an HBaseIOException occurs, the thrift client can only get the 
> error message. This is inconvenient for the client constructing a retry 
> mechanism to handle the exception. So I added a canRetry mark in IOError to 
> make the client side exception handling smarter.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Reopened] (HBASE-20876) Improve docs style in HConstants

[Duo Zhang (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang reopened HBASE-20876:
---

> Improve docs style in HConstants
> 
>
> Key: HBASE-20876
> URL: https://issues.apache.org/jira/browse/HBASE-20876
> Project: HBase
>  Issue Type: Improvement
>Reporter: Reid Chan
>Assignee: Wei-Chiu Chuang
>Priority: Minor
>  Labels: beginner, beginners, newbie
> Fix For: 3.0.0-alpha-1
>
> Attachments: HBASE-20876.master.001.patch
>
>
> In {{HConstants}}, there's a docs snippet:
> {code}
>  /** Don't use it! This'll get you the wrong path in a secure cluster.
>   * Use FileSystem.getHomeDirectory() or
>   * "/user/" + UserGroupInformation.getCurrentUser().getShortUserName()  */
> {code}
> It's ugly style.
> Let's improve this docs with following
> {code}
> /**
>  * Description
>  */
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Reopened] (HBASE-26025) Add a flag to mark if the IOError can be solved by retry in thrift IOError

[Duo Zhang (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-26025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang reopened HBASE-26025:
---

> Add a flag to mark if the IOError can be solved by retry in thrift IOError
> --
>
> Key: HBASE-26025
> URL: https://issues.apache.org/jira/browse/HBASE-26025
> Project: HBase
>  Issue Type: Improvement
>  Components: Thrift
>Reporter: Yutong Xiao
>Assignee: Yutong Xiao
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.3.6, 1.7.1, 2.4.5
>
>
> Currently, if an HBaseIOException occurs, the thrift client can only get the 
> error message. This is inconvenient for the client constructing a retry 
> mechanism to handle the exception. So I added a canRetry mark in IOError to 
> make the client side exception handling smarter.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HBASE-20876) Improve docs style in HConstants

[Duo Zhang (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-20876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang resolved HBASE-20876.
---
Resolution: Fixed

> Improve docs style in HConstants
> 
>
> Key: HBASE-20876
> URL: https://issues.apache.org/jira/browse/HBASE-20876
> Project: HBase
>  Issue Type: Improvement
>Reporter: Reid Chan
>Assignee: Wei-Chiu Chuang
>Priority: Minor
>  Labels: beginner, beginners, newbie
> Fix For: 3.0.0-alpha-1
>
> Attachments: HBASE-20876.master.001.patch
>
>
> In {{HConstants}}, there's a docs snippet:
> {code}
>  /** Don't use it! This'll get you the wrong path in a secure cluster.
>   * Use FileSystem.getHomeDirectory() or
>   * "/user/" + UserGroupInformation.getCurrentUser().getShortUserName()  */
> {code}
> It's ugly style.
> Let's improve this docs with following
> {code}
> /**
>  * Description
>  */
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HBASE-21404) Master/RS navbar active state does not work

[Duo Zhang (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-21404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang resolved HBASE-21404.
---
Resolution: Fixed

> Master/RS navbar active state does not work
> ---
>
> Key: HBASE-21404
> URL: https://issues.apache.org/jira/browse/HBASE-21404
> Project: HBase
>  Issue Type: Bug
>  Components: UI
>Affects Versions: 3.0.0-alpha-1, 2.0.0
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Minor
> Fix For: 3.0.0-alpha-1
>
> Attachments: HBASE-21404.master.001.patch, 
> HBASE-21404.master.001.patch, HBASE-21404.master.002.patch, 
> master_after_patch.png, master_before.png, rs_after_patch.png, rs_before.png
>
>
> In master/rs web UI, the current active tab is not updated when user switches 
> to any tab other than "Home" tab.
> For example: even though say if we are on "tabledetailed.jsp", the navbar 
> does not update the active state of that tab. See master_before.png



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Reopened] (HBASE-21404) Master/RS navbar active state does not work

[Duo Zhang (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-21404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang reopened HBASE-21404:
---

> Master/RS navbar active state does not work
> ---
>
> Key: HBASE-21404
> URL: https://issues.apache.org/jira/browse/HBASE-21404
> Project: HBase
>  Issue Type: Bug
>  Components: UI
>Affects Versions: 3.0.0-alpha-1, 2.0.0
>Reporter: Nihal Jain
>Assignee: Nihal Jain
>Priority: Minor
> Fix For: 3.0.0-alpha-1
>
> Attachments: HBASE-21404.master.001.patch, 
> HBASE-21404.master.001.patch, HBASE-21404.master.002.patch, 
> master_after_patch.png, master_before.png, rs_after_patch.png, rs_before.png
>
>
> In master/rs web UI, the current active tab is not updated when user switches 
> to any tab other than "Home" tab.
> For example: even though say if we are on "tabledetailed.jsp", the navbar 
> does not update the active state of that tab. See master_before.png



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [ANNOUNCE] Apache HBase 3.0.0-alpha-1 is now available for download

[Duo Zhang]

The first email has been rejected by the announce mailing list because
of wrong MIME type text/html...
Send with plain text mode of gmail again...

Duo Zhang  于2021年7月21日周三 下午10:07写道：

>
> The HBase team is happy to announce the immediate availability of HBase
> 3.0.0-alpha-1.
>
> Apache HBase™ is an open-source, distributed, versioned, non-relational
> database. Apache HBase gives you low latency random access to billions of rows
> with millions of columns atop non-specialized hardware. To learn more about
> HBase, see https://hbase.apache.org/.
>
> HBase 3.0.0-alpha-1 is the first alpha release in the HBase 3.x line, which is
> the first release of our next major release line. It includes roughly 3000+
> resolved issues since 2.0.0.
>
> Notice that this is not a production ready release. It is used to let our
> users try and test the new major release, to get feedback before the final GA
> release is out. So please do NOT use it in production. Just try it and report
> back everything you find unusual.
>
> Notable new features include:
>   Synchronous Replication
>   OpenTelemetry Tracing
>   Distributed MOB Compaction
>   Backup and Restore
>
> For other important changes:
>   We do not support hadoop 2.x any more
>   Move RSGroup balancer to core
>   Reimplement sync client on async client
>   CPEPs on shaded proto
>   Move the logging framework from log4j to log4j2
>
> The full list of issues and release notes can be found here:
>
> CHANGELOG: https://downloads.apache.org/hbase/3.0.0-alpha-1/CHANGES.md
> RELEASENOTES: https://downloads.apache.org/hbase/3.0.0-alpha-1/RELEASENOTES.md
>
> or via our issue tracker:
>
>   https://issues.apache.org/jira/projects/HBASE/versions/12332342
>
> To download please follow the links and instructions on our website:
>
>   https://hbase.apache.org/downloads.html
>
> Questions, comments, and problems are always welcome at:
>
>   dev@hbase.apache.org
>   u...@hbase.apache.org
>   user...@hbase.apache.org
>
> Thanks to all who contributed and made this release possible.
>
> Cheers,
> The HBase Dev Team

[ANNOUNCE] Apache HBase 3.0.0-alpha-1 is now available for download

[Duo Zhang]

The HBase team is happy to announce the immediate availability of HBase
3.0.0-alpha-1.

Apache HBase™ is an open-source, distributed, versioned, non-relational
database. Apache HBase gives you low latency random access to billions of rows
with millions of columns atop non-specialized hardware. To learn more about
HBase, see https://hbase.apache.org/.

HBase 3.0.0-alpha-1 is the first alpha release in the HBase 3.x line, which is
the first release of our next major release line. It includes roughly 3000+
resolved issues since 2.0.0.

Notice that this is not a production ready release. It is used to let our
users try and test the new major release, to get feedback before the final GA
release is out. So please do NOT use it in production. Just try it and report
back everything you find unusual.

Notable new features include:
  Synchronous Replication
  OpenTelemetry Tracing
  Distributed MOB Compaction
  Backup and Restore

For other important changes:
  We do not support hadoop 2.x any more
  Move RSGroup balancer to core
  Reimplement sync client on async client
  CPEPs on shaded proto
  Move the logging framework from log4j to log4j2

The full list of issues and release notes can be found here:

CHANGELOG: https://downloads.apache.org/hbase/3.0.0-alpha-1/CHANGES.md
RELEASENOTES: https://downloads.apache.org/hbase/3.0.0-alpha-1/RELEASENOTES.md

or via our issue tracker:

  https://issues.apache.org/jira/projects/HBASE/versions/12332342

To download please follow the links and instructions on our website:

  https://hbase.apache.org/downloads.html

Questions, comments, and problems are always welcome at:

  dev@hbase.apache.org
  u...@hbase.apache.org
  user...@hbase.apache.org

Thanks to all who contributed and made this release possible.

Cheers,
The HBase Dev Team

[ANNOUNCE] Apache HBase 3.0.0-alpha-1 is now available for download

[Duo Zhang]

The HBase team is happy to announce the immediate availability of HBase
3.0.0-alpha-1.

Apache HBase™ is an open-source, distributed, versioned, non-relational
database. Apache HBase gives you low latency random access to billions of
rows
with millions of columns atop non-specialized hardware. To learn more about
HBase, see https://hbase.apache.org/.

HBase 3.0.0-alpha-1 is the first alpha release in the HBase 3.x line, which
is
the first release of our next major release line. It includes roughly 3000+
resolved issues since 2.0.0.

Notice that this is not a production ready release. It is used to let our
users try and test the new major release, to get feedback before the final
GA
release is out. So please do NOT use it in production. Just try it and
report
back everything you find unusual.

Notable new features include:
  Synchronous Replication
  OpenTelemetry Tracing
  Distributed MOB Compaction
  Backup and Restore

For other important changes:
  We do not support hadoop 2.x any more
  Move RSGroup balancer to core
  Reimplement sync client on async client
  CPEPs on shaded proto
  Move the logging framework from log4j to log4j2

The full list of issues and release notes can be found here:

CHANGELOG: https://downloads.apache.org/hbase/3.0.0-alpha-1/CHANGES.md
RELEASENOTES:
https://downloads.apache.org/hbase/3.0.0-alpha-1/RELEASENOTES.md

or via our issue tracker:

  https://issues.apache.org/jira/projects/HBASE/versions/12332342

To download please follow the links and instructions on our website:

  https://hbase.apache.org/downloads.html

Questions, comments, and problems are always welcome at:

  dev@hbase.apache.org
  u...@hbase.apache.org
  user...@hbase.apache.org

Thanks to all who contributed and made this release possible.

Cheers,
The HBase Dev Team

[jira] [Resolved] (HBASE-26088) conn.getBufferedMutator(tableName) leaks thread executors and other problems

[Anoop Sam John (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-26088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anoop Sam John resolved HBASE-26088.

Fix Version/s: 2.3.6
 Hadoop Flags: Reviewed
   Resolution: Fixed

Pushed to branch-2.3, branch-2.4 and branch-2.
Thanks for the patch [~shahrs87].
Thanks for the great find [~whitney13]

> conn.getBufferedMutator(tableName) leaks thread executors and other problems
> 
>
> Key: HBASE-26088
> URL: https://issues.apache.org/jira/browse/HBASE-26088
> Project: HBase
>  Issue Type: Bug
>  Components: Client
>Affects Versions: 1.4.13, 2.4.4
>Reporter: Whitney Jackson
>Assignee: Rushabh Shah
>Priority: Critical
> Fix For: 2.5.0, 2.3.6, 2.4.5
>
>
> TL;DR: {{conn.getBufferedMutator(tableName)}} is dangerous in hbase client 
> 2.4.4 and doesn't match documented behavior in 1.4.13.
> To work around the problems until fixed do this:
> {code:java}
> var mySingletonPool = HTable.getDefaultExecutor(hbaseConf);
> var params = new BufferedMutatorParams(tableName);
> params.pool(mySingletonPool);
> var myMutator = conn.getBufferedMutator(params);
> {code}
> And avoid code like this:
> {code:java}
> var myMutator = conn.getBufferedMutator(tableName);
> {code}
> The full story:
> My application started leaking threads after upgrading from hbase client 
> 1.4.13 to 2.4.4. So much so that after less than a minute of runtime more 
> that 30k threads are leaked and all available virtual memory on the box (> 50 
> GB) is consumed. Other processes on the box start crashing with memory 
> allocation errors. Even running {{ls}} at the shell fails with OS resource 
> allocation failures.
> A thread dump after just a few seconds of runtime shows thousands of threads 
> like this:
> {code:java}
> "htable-pool-0" #8841 prio=5 os_prio=0 cpu=0.15ms elapsed=7.49s 
> tid=0x7efb6d2a1000 nid=0x57d2 waiting on condition [0x7ef8a6c38000]
>  java.lang.Thread.State: TIMED_WAITING (parking)
>  at jdk.internal.misc.Unsafe.park(java.base@11.0.6/Native Method)
>  - parking to wait for <0x0007e7cd6188> (a 
> java.util.concurrent.SynchronousQueue$TransferStack)
>  at 
> java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.6/LockSupport.java:234)
>  at 
> java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(java.base@11.0.6/SynchronousQueue.java:462)
>  at 
> java.util.concurrent.SynchronousQueue$TransferStack.transfer(java.base@11.0.6/SynchronousQueue.java:361)
>  at 
> java.util.concurrent.SynchronousQueue.poll(java.base@11.0.6/SynchronousQueue.java:937)
>  at 
> java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.6/ThreadPoolExecutor.java:1053)
>  at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.6/ThreadPoolExecutor.java:1114)
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.6/ThreadPoolExecutor.java:628)
>  at java.lang.Thread.run(java.base@11.0.6/Thread.java:834)
> {code}
>  
> Note: All the threads are labeled {{htable-pool-0}}. That suggests we're 
> leaking thread executors not just threads. The {{htable-pool}} part indicates 
> the problem is to do with {{HTable.getDefaultExecutor(conf)}} and the only 
> part of my code that interacts with that is a call to 
> {{conn.getBufferedMutator(tableName)}}.
>  
> Looking at the hbase client code shows a few problems:
> 1) Neither 1.4.13 nor 2.4.4's behavior matches the documentation for 
> {{conn.getBufferedMutator(tableName)}} which says:
> {quote}This BufferedMutator will use the Connection's ExecutorService.
> {quote}
> That suggests some singleton thread executor is being used which is not the 
> case.
>  
> 2) Under 1.4.13 you get a new {{ThreadPoolExecutor}} for every 
> {{BufferedMutator}}. That's probably not what you want but you likely won't 
> notice. I didn't. It's a code path I hadn't profiled much.
>  
> 3) Under 2.4.4 you get a new {{ThreadPoolExecutor}} for every 
> {{BufferedMutator}} *and* that {{ThreadPoolExecutor}} *is not* cleaned up 
> after the {{Mutator}} is closed. Each completed {{ThreadPoolExecutor}} 
> carries with it one thread which hangs around until a timeout value which 
> defaults to 60 seconds.
> My application creates one {{BufferedMutator}} for every incoming stream and 
> there are lots of streams, some of them are short lived so my code leaks 
> threads fast under 2.4.4.
> Here's the part where a new executor is created for every {{BufferedMutator}} 
> (it's similar for 1.4.13):
> [https://github.com/apache/hbase/blob/branch-2.4/hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java#L420]
>  
> The reason for the leak in 2.4.4 is the should-we/shouldn't-we cleanup logic 
> added here:
> [https://github.com/apache/hbase/blob/branch-2.4/hbase-cli

[jira] [Resolved] (HBASE-26104) HBase "delete" cannot delete data versions smaller than the specified timestamp

[Edward (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-26104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Edward resolved HBASE-26104.

Resolution: Not A Problem

> HBase "delete" cannot delete data versions smaller than the specified 
> timestamp
> ---
>
> Key: HBASE-26104
> URL: https://issues.apache.org/jira/browse/HBASE-26104
> Project: HBase
>  Issue Type: Improvement
>  Components: Deletes
>Affects Versions: 2.1.0
>Reporter: Edward
>Priority: Minor
>
> When I use 'delete' to delete a piece of data, I set a value larger than the 
> timestamp of the data, and then the operation does not take effect. But 
> 'deleteall' is feasible. Is this difference designed on purpose?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)