[jira] [Resolved] (HBASE-23337) Several modules missing in nexus for Apache HBase 2.2.2

[Sean Busbey (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-23337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey resolved HBASE-23337.
-
Fix Version/s: 1.4.13
   2.1.9
   2.2.3
   1.6.0
   2.3.0
   Resolution: Fixed

the missing 2.2.2 artifacts have been published and the needed changes for the 
release scripts + branch poms are in place.

> Several modules missing in nexus for Apache HBase 2.2.2
> ---
>
> Key: HBASE-23337
> URL: https://issues.apache.org/jira/browse/HBASE-23337
> Project: HBase
>  Issue Type: Bug
>  Components: build, community, scripts
>Affects Versions: 2.2.2
>Reporter: Chao
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 3.0.0, 2.3.0, 1.6.0, 2.2.3, 2.1.9, 1.4.13
>
> Attachments: HBASE-23337.0.patch, HBASE-23337.1.patch
>
>
> The latest version of hbase-shaded-client is currently 2.2.1. It has been a 
> while since 2.2.2 release (2019/10/25). See: 
> [https://search.maven.org/search?q=hbase-shaded-client].



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Error Handling and Logging

[Stack]

Thanks for the helpful note David. Appreciated.
S

On Tue, Nov 26, 2019 at 1:44 PM David Mollitor  wrote:

> Hello Team,
>
> I am one of many people responsible for supporting the Hadoop products out
> in the field.  Error handling and logging are crucial to my success.  I've
> been reading over the code and I see many of the same mistakes again and
> again.  I just wanted to bring some of these things to your attention so
> that moving forward, we can make these products better.
>
> The general best-practice is:
>
> public class TestExceptionLogging
> {
>   private static final Logger LOG =
> LoggerFactory.getLogger(TestExceptionLogging.class);
>
>   public static void main(String[] args) {
> try {
>   processData();
> } catch (Exception e) {
>   LOG.error("Application failed", e);
> }
>   }
>
>   public static void processData() throws Exception {
> try {
>   readData();
> } catch (Exception e) {
>   throw new Exception("Failed to process data", e);
> }
>   }
>
>   public static byte[] readData() throws Exception {
> throw new IOException("Failed to read device");
>   }
> }
>
> Produces:
>
> [main] ERROR TestExceptionLogging - Application failed
> java.lang.Exception: Failed to process data
> at TestExceptionLogging.processData(TestExceptionLogging.java:22)
> at TestExceptionLogging.main(TestExceptionLogging.java:12)
> Caused by: java.io.IOException: Failed to read device
> at TestExceptionLogging.readData(TestExceptionLogging.java:27)
> at TestExceptionLogging.processData(TestExceptionLogging.java:20)
> ... 1 more
>
>
>
> Please notice that when an exception is thrown, and caught, it is wrapped
> at each level and each level adds some more context to describe what was
> happening when the error occurred.  It also produces a complete stack trace
> that pinpoints the issue.  For Hive folks, it is rarely the case that a
> method consuming a HMS API call should itself throw a MetaException.  The
> MetaException has no way of wrapping an underlying Exception and helpful
> data is often loss.  A method may chooses to wrap a MetaException, but it
> should not be throwing them around as the default behavior.
>
> Also important to note is that there is exactly one place that is doing the
> logging.  There does not need to be any logging at the lower levels.  A
> catch block should throw or log, not both.  This is an anti-pattern and
> annoying as the end user: having to deal with multiple stack traces at
> different log levels for the same error condition.  The log message should
> be at the highest level only.
>
> https://community.oracle.com/docs/DOC-983543#logAndThrow
>
> Both projects use SLF4J as the logging framework (facade anyway).  Please
> familiarize yourself with how to correctly log an Exception.  There is no
> need to log a thread name, a time stamp, a class name, or a stack trace.
> The logging framework will do that all for you.
>
> http://www.slf4j.org/faq.html#paramException
>
> Again, there is no need to 'stringify' an exception. For example, do not
> use this:
>
>
> https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/StringUtils.java#L86
>
>
> If you do however want to dump a stack trace for debugging (or trace)
> purposes, consider performing the following:
>
> if (LOG.isDebugEnabled()) {
>   LOG.debug("Dump Thread Stack", new Exception("Thread Stack Trace (Not an
> Error)"));
> }
>
> Finally, I've seen it a couple of times in Apache project that enabling
> debug-level logging causes the application to emit logs at other levels,
> for example:
>
> LOG.warn("Some error occurred: {}", e.getMessage());
> if (LOG.isDebugEnabled()) {
>   LOG. warn("Dump Warning Thread Stack", e);
> }
>
> Please refrain from doing this.  The inner log statement should be at DEBUG
> level to match the check.  Otherwise, when I enable DEBUG logging in the
> application, the expectation that I have is that I will have the exact
> logging as the INFO level, but I will also have additional DEBUG details as
> well.  I am going to be using 'grep' to find DEBUG and I will miss this
> additional logging.  I will then be peeved when I say "ah ha! It's a WARN
> log message, I don't need DEBUG enabled to get this logging when the issue
> happens again!"... but then I restart the application and discover I do in
> fact need DEBUG logging enabled.
>
> Finally, finally, do not pass server-generated stack traces to a client
> application.  When a client dumps a stack trace, it is not always trivial
> to decide if the stack trace is in regards to the client or the server.
> This can also be a security issue as it may hint to the client which
> libraries and (based on line numbers) which versions of the libraries are
> being used.  If the server is going to generate an error in response to a
> client API call, the server should log the exception (right before
> returning the response to the client) and it should only pass a 

[jira] [Created] (HBASE-23363) MobCompactionChore takes a long time to complete once job

[Bo Cui (Jira)]

Bo Cui created HBASE-23363:
--

 Summary: MobCompactionChore takes a long time to complete once job
 Key: HBASE-23363
 URL: https://issues.apache.org/jira/browse/HBASE-23363
 Project: HBase
  Issue Type: Bug
  Components: mob
Affects Versions: 2.2.2, 2.1.1
Reporter: Bo Cui
 Attachments: image-2019-12-04-11-01-20-352.png

mob table compcation is done in master
poolSize of hbase choreService is 1
if hbase has 1000 mob table,MobCompactionChore takes a long time to complete 
once job, other chore need to wait

!image-2019-12-04-11-01-20-352.png!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HBASE-23356) When construct StoreScanner throw exceptions it is possible to left some KeyValueScanner not closed.

[Lijin Bin (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-23356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lijin Bin resolved HBASE-23356.
---
Fix Version/s: 2.1.9
   2.2.3
   2.3.0
   3.0.0
   Resolution: Fixed

> When construct StoreScanner throw exceptions it is possible to left some 
> KeyValueScanner not closed.
> 
>
> Key: HBASE-23356
> URL: https://issues.apache.org/jira/browse/HBASE-23356
> Project: HBase
>  Issue Type: Bug
>Reporter: Lijin Bin
>Assignee: Lijin Bin
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
>
> {code}
> try {
>   // Pass columns to try to filter out unnecessary StoreFiles.
>   List scanners = selectScannersFrom(store,
> store.getScanners(cacheBlocks, scanUsePread, false, matcher, 
> scan.getStartRow(),
>   scan.includeStartRow(), scan.getStopRow(), scan.includeStopRow(), 
> this.readPt));
>   // Seek all scanners to the start of the Row (or if the exact matching 
> row
>   // key does not exist, then to the start of the next matching Row).
>   // Always check bloom filter to optimize the top row seek for delete
>   // family marker.
>   seekScanners(scanners, matcher.getStartKey(), explicitColumnQuery && 
> lazySeekEnabledGlobally,
> parallelSeekEnabled);
> 
> ..
> } catch (IOException e) {
>   // remove us from the HStore#changedReaderObservers here or we'll have 
> no chance to
>   // and might cause memory leak
>   store.deleteChangedReaderObserver(this);
>   throw e;
> }
> {code}
> Like the above, if the seekScanners(scanners,...) throws exception, the 
> scanners do not closed, we need to close the scanners.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-23362) WalPrettyPrinter should include the table name

[Bharath Vissapragada (Jira)]

Bharath Vissapragada created HBASE-23362:


 Summary: WalPrettyPrinter should include the table name
 Key: HBASE-23362
 URL: https://issues.apache.org/jira/browse/HBASE-23362
 Project: HBase
  Issue Type: Bug
  Components: tooling
Affects Versions: master
Reporter: Bharath Vissapragada
Assignee: Bharath Vissapragada


I was playing around with a large WAL file to debug something and I noticed a 
couple missing items.

- Pretty printer doesn't print the table name. It is difficult to map the 
region hashes to a table name manually.
- It should include an option to filter the edits by table so that we can only 
see the entries for a given table. A similar option exists for regions.

I hacked it locally, I thought this might save some time for others too if the 
fix goes into the master.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-23361) [UI] Limit two decimals even for total average load

[Karthik Palanisamy (Jira)]

Karthik Palanisamy created HBASE-23361:
--

 Summary: [UI] Limit two decimals even for total average load
 Key: HBASE-23361
 URL: https://issues.apache.org/jira/browse/HBASE-23361
 Project: HBase
  Issue Type: Improvement
  Components: UI
Affects Versions: 3.0.0
Reporter: Karthik Palanisamy
Assignee: Karthik Palanisamy
 Attachments: Screen Shot 2019-12-03 at 3.17.19 PM.png

We somehow missed limiting decimal points in the total average load.

 

!Screen Shot 2019-12-03 at 3.17.19 PM.png!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-23360) [CLI] Fix help command "set_quota"

[Karthik Palanisamy (Jira)]

Karthik Palanisamy created HBASE-23360:
--

 Summary: [CLI] Fix help command "set_quota"
 Key: HBASE-23360
 URL: https://issues.apache.org/jira/browse/HBASE-23360
 Project: HBase
  Issue Type: Improvement
Reporter: Karthik Palanisamy
Assignee: Karthik Palanisamy


To remove a quota by throttle_type,
{code:java}
hbase> help "set_quota"
...
hbase> set_quota TYPE => THROTTLE, THROTTLE_TYPE => WRITE, USER => 'u1', LIMIT 
=> NONE

{code}
but the actual command should be, 
{code:java}
hbase> set_quota TYPE => THROTTLE, THROTTLE_TYPE => WRITE_NUMBER, USER => 'u1', 
LIMIT => NONE{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-23359) RS going down with NPE when splitting a region with compaction disabled in branch-1

[Toshihiro Suzuki (Jira)]

Toshihiro Suzuki created HBASE-23359:


 Summary: RS going down with NPE when splitting a region with 
compaction disabled in branch-1
 Key: HBASE-23359
 URL: https://issues.apache.org/jira/browse/HBASE-23359
 Project: HBase
  Issue Type: Bug
Reporter: Toshihiro Suzuki


Trying to backport HBASE-22096 to brach-1, I faced the issue where a RS goes 
down with NPE when splitting a region with compaction disabled.

The steps to reproduce this issue are as follows:
{code}
compaction_switch false
create "test", "cf"
(0...2000).each{|i| put "test", "row#{i}", "cf:col", "val#{i}"}
split "test"
{code}

Looking at the regionserver log, I saw the following log:
{code}
2019-12-03 22:25:38,611 INFO  [RS:0;10.0.1.11:53504-splits-0] 
regionserver.SplitRequest: Running rollback/cleanup of failed split of 
test,,1575379535506.50e322ec68162025e17cddffdc2fb17e.; null
java.lang.NullPointerException
at 
org.apache.hadoop.hbase.regionserver.HStore.cancelRequestedCompaction(HStore.java:1834)
at 
org.apache.hadoop.hbase.regionserver.CompactSplitThread$Rejection.rejectedExecution(CompactSplitThread.java:656)
at 
java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
at 
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
at 
org.apache.hadoop.hbase.regionserver.CompactSplitThread.requestCompactionInternal(CompactSplitThread.java:401)
at 
org.apache.hadoop.hbase.regionserver.CompactSplitThread.requestSystemCompaction(CompactSplitThread.java:348)
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:2111)
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:2097)
at 
org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.openDaughters(SplitTransactionImpl.java:478)
at 
org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsAfterPONR(SplitTransactionImpl.java:549)
at 
org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:532)
at 
org.apache.hadoop.hbase.regionserver.SplitRequest.doSplitting(SplitRequest.java:82)
at 
org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:153)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-12-03 22:25:38,613 FATAL [RS:0;10.0.1.11:53504-splits-0] 
regionserver.HRegionServer: ABORTING region server 
10.0.1.11,53504,1575379011279: Abort; we got an error after point-of-no-return
{code}




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] EOM branch-1.3

[Sean Busbey]

If it would change anyone's willingness to maintain the branch, then I
encourage them to go ask about the need on user@hbase.

AFAIK in the year since we started talking about shutting down branch-1.3
no committer or PMC has expressed that their interest would change if
someone on user@hbase felt stuck on 1.3.z.

Also worth noting that in the month since the 1.3.6 announcement went out
noone has showed up to say they can't move off of the release line.


On Mon, Dec 2, 2019, 22:53 Andrew Purtell  wrote:

> And if a non dev says they won’t move off 1.3? Will it change any
> committer or PMC minds on actually continuing to do 1.3 releases? If not I
> think we have to call it for lack of interest and bandwidth.
>
> 1.4 is a functional superset of 1.3 and the current stable line anyway.
> Seems little reason not to upgrade save inertia or risk aversion.
>
>
> > On Dec 2, 2019, at 5:43 PM, Sean Busbey  wrote:
> >
> > Anyone who wants branch-1.3 to keep having releases has to be willing
> > to volunteer to maintain it. If the note in the 1.3.6 release wasn't
> > sufficient motivation to get them to show up on dev@hbase to do so, I
> > could put a more explicit mention of it in the EOM message. We'd need
> > to come up with some phrasing that didn't leave the status of the
> > release line ambiguous though.
> >
> > For reference, these are the last two EOM announcements we did:
> >
> > * 2.0.z in Sep 2019: https://s.apache.org/slgsa
> > * 1.2.z in Jun 2019:  https://s.apache.org/g8lnu
> >
> > 2.0 and 1.3 were never a release line with the "stable" marker on it.
> > 1.2 was the stable release line prior to 1.4.
> >
> >> On Mon, Dec 2, 2019 at 1:58 PM Misty Linville  wrote:
> >>
> >> Whether any non-dev users are unable to move off 1.3, I suppose.
> >>
> >>> On Mon, Dec 2, 2019 at 11:04 AM Sean Busbey  wrote:
> >>>
> >>> On what, specifically?
> >>>
>  On Mon, Dec 2, 2019, 11:24 Misty Linville  wrote:
> >>>
>  Should the user list be allowed to weigh in?
> 
>  On Mon, Dec 2, 2019 at 7:33 AM Andrew Purtell <
> andrew.purt...@gmail.com>
>  wrote:
> 
> > I think there is a consensus on moving the stable pointer, based on
> > earlier discussion. What I would suggest is a separate thread to
> >>> propose
> > it, and if nobody objects, do it.
> >
> >> On Dec 2, 2019, at 5:14 AM, 张铎(Duo Zhang) 
>  wrote:
> >>
> >> +1.
> >>
> >> And I think it is time to move the stable pointer to 2.2.x? I know
> >>> that
> >> 2.2.x still has some bugs, especially on the procedure store, but
>  anyway,
> >> we have HBCK2 to fix them.
> >>
> >> And for the current stable release line, 1.4.x, the assignment
> >>> manager
> > also
> >> has bugs, as it is the reason why we introduced AMv2.
> >>
> >> So I do not think bug free is the 'must have' for a stable release
>  line.
> >>
> >> Jan Hentschel  于2019年12月2日周一
>  下午4:57写道：
> >>
> >>> +1
> >>>
> >>> From: Sakthi 
> >>> Reply-To: "dev@hbase.apache.org" 
> >>> Date: Monday, December 2, 2019 at 3:32 AM
> >>> To: "dev@hbase.apache.org" 
> >>> Subject: Re: [DISCUSS] EOM branch-1.3
> >>>
> >>> +1
> >>>
> >>> On Sun, Dec 1, 2019 at 6:28 PM Andrew Purtell <
>  andrew.purt...@gmail.com
> >>> >
> >>> wrote:
> >>>
> >>> +1 for EOL of 1.3.
> >>>
> >>> Onward to 1.6!
> >>>
> >>>
>  On Dec 1, 2019, at 5:38 PM, Sean Busbey   >>> bus...@apache.org>> wrote:
> 
>  Hi folks!
> 
>  It's been about a month since the last 1.3.z release came out.
> >>> We've
>  been talking about EOM for branch-1.3 for about a year. Most
>  recently,
>  we had a growing consensus[1] to EOM after getting the 1.3.6
> >>> release
>  out with the fixes for Jackson in HBASE-22728 out.
> 
>  Looking at the things that have since landed in branch-1.3 and
>  nothing
>  looks critical (these are all Major or Minor)[2]:
> 
>  - HBASE-23149 hbase shouldPerformMajorCompaction logic is not
> >>> correct
>  - HBASE-23185 High cpu usage because getTable()#put() gets config
>  value every time
>  - HBASE-23261 Region stuck in transition while splitting
>  - HBASE-18439 Subclasses of o.a.h.h.chaos.actions.Action all use
> >>> the
>  same logger
>  - HBASE-23207 Log a region open journal
>  - HBASE-23250 Log message about CleanerChore delegate
> >>> initialization
>  should be at INFO
> 
>  Someone on 1.3.6 can get all these same things fixed by upgrading
> >>> to
>  our current stable release.
> 
>  Releases on 1.3.z started in 2017. The branch has only averaged ~2
>  maintenance releases a year; I think reflecting a lack of
> community
>  interest in maintaining the branch. For compari

[jira] [Resolved] (HBASE-23351) hbase-connectors Test fail when stating HBase mini cluster

[Balazs Meszaros (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-23351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Balazs Meszaros resolved HBASE-23351.
-
Fix Version/s: connector-1.0.1
   Resolution: Fixed

> hbase-connectors Test fail when stating HBase mini cluster
> --
>
> Key: HBASE-23351
> URL: https://issues.apache.org/jira/browse/HBASE-23351
> Project: HBase
>  Issue Type: Bug
>Affects Versions: connector-1.0.0
>Reporter: István Adamcsik
>Assignee: István Adamcsik
>Priority: Major
> Fix For: connector-1.0.1
>
>
> Due to a shading issue in hbase-shaded-testing-util, whereby jackson is 
> omitted from the fat jar, the hbase-connectors tests fail to start mini 
> clusters which fail with the below error.
> {code:java}
> *** RUN ABORTED ***  java.lang.NoClassDefFoundError: Could not initialize 
> class org.apache.hadoop.hdfs.web.WebHdfsFileSystem  at 
> org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.initWebHdfs(NameNodeHttpServer.java:78)
>   at 
> org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.start(NameNodeHttpServer.java:166)
>   at 
> org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:842)
>   at 
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:693) 
>  at org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:906) 
>  at org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:885) 
>  at 
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1626)
>   at 
> org.apache.hadoop.hdfs.MiniDFSCluster.createNameNode(MiniDFSCluster.java:1162)
>   at 
> org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:1037)
>   at 
> org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:830{code}
> This issue is fixed in hbase 2.2.2 with HBASE-23007.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-23358) Set version as 2.1.9-SNAPSHOT in branch-2.1

[Duo Zhang (Jira)]

Duo Zhang created HBASE-23358:
-

 Summary: Set version as 2.1.9-SNAPSHOT in branch-2.1
 Key: HBASE-23358
 URL: https://issues.apache.org/jira/browse/HBASE-23358
 Project: HBase
  Issue Type: Sub-task
  Components: build, pom
Reporter: Duo Zhang
Assignee: Duo Zhang
 Fix For: 2.1.9






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HBASE-23302) Put up 2.1.8RC0

[Duo Zhang (Jira)]

 [ 
https://issues.apache.org/jira/browse/HBASE-23302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang resolved HBASE-23302.
---
  Assignee: Duo Zhang
Resolution: Fixed

> Put up 2.1.8RC0
> ---
>
> Key: HBASE-23302
> URL: https://issues.apache.org/jira/browse/HBASE-23302
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-23357) Add 2.1.8 to download page

[Duo Zhang (Jira)]

Duo Zhang created HBASE-23357:
-

 Summary: Add 2.1.8 to download page
 Key: HBASE-23357
 URL: https://issues.apache.org/jira/browse/HBASE-23357
 Project: HBase
  Issue Type: Sub-task
  Components: website
Reporter: Duo Zhang
Assignee: Duo Zhang
 Fix For: 3.0.0






--
This message was sent by Atlassian Jira
(v8.3.4#803005)