[jira] [Resolved] (HBASE-23973) Backport HBASE-23561 to branch-2
[ https://issues.apache.org/jira/browse/HBASE-23973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Minwoo Kang resolved HBASE-23973. - Resolution: Won't Do > Backport HBASE-23561 to branch-2 > > > Key: HBASE-23973 > URL: https://issues.apache.org/jira/browse/HBASE-23973 > Project: HBase > Issue Type: Bug >Reporter: Minwoo Kang >Assignee: Minwoo Kang >Priority: Trivial > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24131) [Flakey Tests] TestExportSnapshot takes too long; up against 13min max
Michael Stack created HBASE-24131: - Summary: [Flakey Tests] TestExportSnapshot takes too long; up against 13min max Key: HBASE-24131 URL: https://issues.apache.org/jira/browse/HBASE-24131 Project: HBase Issue Type: Bug Reporter: Michael Stack TestExportSnapshot fails fairly regularly locally. Looking, its test timeout. Looking at how long it ran, its 13minutes plus. Looking at recent successful branch-2 run, it passed but it took about 7-8minutes. Let me break up the test into to pieces. org.junit.runners.model.TestTimedOutException: test timed out after 780 seconds at org.apache.hadoop.hbase.snapshot.TestExportSnapshot.testExportFileSystemState(TestExportSnapshot.java:227) at org.apache.hadoop.hbase.snapshot.TestExportSnapshot.testExportRetry(TestExportSnapshot.java:267) ... I see this in the log: > TEST TIMED OUT. PRINTING THREAD DUMP. < Test started at: 2020-04-06 17:19:21,739 INFO ... and the timestamp just above the TIMED OUT was 2020-04-06 17:31:01,758 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24130) rat plugin complains about having an unlicensed file.
Minwoo Kang created HBASE-24130: --- Summary: rat plugin complains about having an unlicensed file. Key: HBASE-24130 URL: https://issues.apache.org/jira/browse/HBASE-24130 Project: HBase Issue Type: Bug Reporter: Minwoo Kang Assignee: Minwoo Kang Files with unapproved licenses: dev-support/HOW_TO_YETUS_LOCAL.md -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24077) When encounter RowTooBigException, log the row info.
[ https://issues.apache.org/jira/browse/HBASE-24077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lijin Bin resolved HBASE-24077. --- Resolution: Fixed > When encounter RowTooBigException, log the row info. > > > Key: HBASE-24077 > URL: https://issues.apache.org/jira/browse/HBASE-24077 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.2.4 >Reporter: Lijin Bin >Assignee: Lijin Bin >Priority: Minor > Fix For: 3.0.0, 2.2.5 > > > Current when we encounter a big row and throw RowTooBigException, there is no > information about the row, it hard to debug. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24081) Provide documentation for running Yetus with HBase
[ https://issues.apache.org/jira/browse/HBASE-24081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-24081. -- Fix Version/s: 2.4.0 1.7.0 3.0.0 Resolution: Fixed Applied to master, branch-2, branch-1. > Provide documentation for running Yetus with HBase > -- > > Key: HBASE-24081 > URL: https://issues.apache.org/jira/browse/HBASE-24081 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: 3.0.0, 1.7.0, 2.4.0 > > > A colleague asked how to use Yetus with HBase, so I wrote up a little how-to > doc. Maybe it's useful to someone else? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24129) Reenable TestCompactingToCellFlatMapMemStore
Michael Stack created HBASE-24129: - Summary: Reenable TestCompactingToCellFlatMapMemStore Key: HBASE-24129 URL: https://issues.apache.org/jira/browse/HBASE-24129 Project: HBase Issue Type: Sub-task Reporter: Michael Stack Disabled in the parent. Reenable when flakyness fixed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24128) [F
Michael Stack created HBASE-24128: - Summary: [F Key: HBASE-24128 URL: https://issues.apache.org/jira/browse/HBASE-24128 Project: HBase Issue Type: Bug Reporter: Michael Stack -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (HBASE-23779) Up the default fork count to make builds complete faster; make count relative to CPU count
[ https://issues.apache.org/jira/browse/HBASE-23779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack reopened HBASE-23779: --- Reopening. Was resolved because thought there no elbow room here but have been making progress on this master project off in side issues. Let me bring the work together back here under this issue. High-level, trying to run w/ more parallelism -- forkcount and the mvn -T -- we ran into limits. Experiments trying to up our forkcount to 0.5C (8 cores on apache jenkins) ran into container memory and host/user ulimit upper-bounds. Returning fork count down to 0.25C re-stabilized builds (along w/ test fixes). Minimally would at least like to add -T0.25C to maven before resolving this issue again. Maximally, would like to get build running on jenkins at 0.5C (In local tests build takes twice as long if forkcount+ T arg are halved... say from 0.5C to 0.25C). In local tests, I can run with 1.0C. On a 40CPU linux machine, it is a bit wobbly. It was wobbly on big GCE instances but I should revisit. On a 16CPU mac, it seems fine as on a 4CPU VM. For Jenkins, 0.5C might be doable. Lets try again now we know more. > Up the default fork count to make builds complete faster; make count relative > to CPU count > -- > > Key: HBASE-23779 > URL: https://issues.apache.org/jira/browse/HBASE-23779 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: addendum2.patch, test_yetus_934.0.patch > > > Tests take a long time. Our fork count running all tests are conservative -- > 1 (small) for first part and 5 for second part (medium and large). Rather > than hardcoding we should set the fork count to be relative to machine size. > Suggestion here is 0.75C where C is CPU count. This ups the CPU use on my box. > Looking up at jenkins, it seems like the boxes are 24 cores... at least going > by my random survey. The load reported on a few seems low though this not > representative (looking at machine/uptime). > More parallelism willl probably mean more test failure. Let me take a look > see. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24127) Switch to `RawLocalFileSystem` as default FS for tests
Nick Dimiduk created HBASE-24127:
Summary: Switch to `RawLocalFileSystem` as default FS for tests
Key: HBASE-24127
URL: https://issues.apache.org/jira/browse/HBASE-24127
Project: HBase
Issue Type: Test
Components: test
Reporter: Nick Dimiduk
In the PR discussion on HBASE-24106, [~elserj]
[mentioned|https://github.com/apache/hbase/pull/1422#discussion_r403789851]
that another project has had success with switching its unit test suite to use
{{RawLocalFileSystem}} from {{LocalFileSystem}}. This is discussed in the
context of the availability of the {{hsync}} and {{hflush}} stream
capabilities, which are not available on {{LocalFileSystem}}.
Perhaps we should also consider doing the same for our default {{standalone}}
user experience. I don't know if doing so should be a separate task from this
one, or if completing one is effectively performing the implementation for the
other.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-24106) Update getting started documentation after HBASE-24086
[
https://issues.apache.org/jira/browse/HBASE-24106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk resolved HBASE-24106.
--
Fix Version/s: 3.0.0
Resolution: Fixed
> Update getting started documentation after HBASE-24086
> --
>
> Key: HBASE-24106
> URL: https://issues.apache.org/jira/browse/HBASE-24106
> Project: HBase
> Issue Type: Sub-task
> Components: documentation
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 3.0.0
>
>
> HBASE-24086 allows HBase to degrade gracefully to running on a
> {{LocalFileSystem}} without further user configuration. Update the docs
> accordingly.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-24126) Up the container nproc uplimit from 10000 to 12500
Michael Stack created HBASE-24126: - Summary: Up the container nproc uplimit from 1 to 12500 Key: HBASE-24126 URL: https://issues.apache.org/jira/browse/HBASE-24126 Project: HBase Issue Type: Sub-task Components: build Reporter: Michael Stack Let me up the container nproc count from 1. If many tests running in parallel, container could breach the 1 limit: i.e. if a bunch of heavy-duty long tests run at same time (0.5C on a 16CPU machine means 8 possible concurrents) and we spin up 1-2000 threads in each test, then could hit the 1 limit. Move up the limit some expecially after INFRA just upped our limit to 3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24118) [Flakey Tests] TestCloseRegionWhileRSCrash
[
https://issues.apache.org/jira/browse/HBASE-24118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack resolved HBASE-24118.
---
Resolution: Fixed
Reverted then reapplied as an @Ignore on the test:
{code}
- @Test
+ @org.junit.Ignore @Test // Until root-cause of flakeyness, HBASE-24117, is
addressed.
public void testRetryBackoff() throws IOException, InterruptedException {
HRegionServer srcRs = UTIL.getRSForFirstRegionInTable(TABLE_NAME);
RegionInfo region = srcRs.getRegions(TABLE_NAME).get(0).getRegionInfo();
@@ -190,11 +192,10 @@ public class TestCloseRegionWhileRSCrash {
// here we start a new master
UTIL.getMiniHBaseCluster().startMaster();
t.join();
-// Make sure that the region is online, it may not be on the original
target server, as we will
-// set forceNewPlan to true if there is a server crash.
-// DISABLED THIS CHECK. See HBASE-24117.
-// try (Table table = UTIL.getConnection().getTable(TABLE_NAME)) {
-// table.put(new Put(Bytes.toBytes(1)).addColumn(CF,
Bytes.toBytes("cq"), Bytes.toBytes(1)));
-// }
+// Make sure that the region is online, it may not on the original target
server, as we will set
+// forceNewPlan to true if there is a server crash
+try (Table table = UTIL.getConnection().getTable(TABLE_NAME)) {
+ table.put(new Put(Bytes.toBytes(1)).addColumn(CF, Bytes.toBytes("cq"),
Bytes.toBytes(1)));
+}
{code}
Re-resolving.
> [Flakey Tests] TestCloseRegionWhileRSCrash
> --
>
> Key: HBASE-24118
> URL: https://issues.apache.org/jira/browse/HBASE-24118
> Project: HBase
> Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-HBASE-24118-Flakey-Tests-TestCloseRegionWhileRSCrash.patch
>
>
> TestCloseRegionWhileRSCrash is flakey because of HBASE-24117 -- because
> moved Region may not online in the end. Remove the last bit of the test...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Reopened] (HBASE-24118) [Flakey Tests] TestCloseRegionWhileRSCrash
[ https://issues.apache.org/jira/browse/HBASE-24118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack reopened HBASE-24118: --- Reopening to revert and replace the revert w/ an @Ignore for this test. > [Flakey Tests] TestCloseRegionWhileRSCrash > -- > > Key: HBASE-24118 > URL: https://issues.apache.org/jira/browse/HBASE-24118 > Project: HBase > Issue Type: Bug >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: > 0001-HBASE-24118-Flakey-Tests-TestCloseRegionWhileRSCrash.patch > > > TestCloseRegionWhileRSCrash is flakey because of HBASE-24117 -- because > moved Region may not online in the end. Remove the last bit of the test... -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24125) hbase-filesystem to use hbase-thirdparty 3.2.0
Wei-Chiu Chuang created HBASE-24125: --- Summary: hbase-filesystem to use hbase-thirdparty 3.2.0 Key: HBASE-24125 URL: https://issues.apache.org/jira/browse/HBASE-24125 Project: HBase Issue Type: Task Affects Versions: 1.0.0-alpha1 Reporter: Wei-Chiu Chuang Assignee: Wei-Chiu Chuang Fix For: 1.0.0-alpha2 hbase-filesystem is currently on hbase-thirdparty 2.2.1. Update it to 3.2.0 so we can use the latest guava. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24124) hbase-filesystem to use guava from hbase-thirdparty
Wei-Chiu Chuang created HBASE-24124:
---
Summary: hbase-filesystem to use guava from hbase-thirdparty
Key: HBASE-24124
URL: https://issues.apache.org/jira/browse/HBASE-24124
Project: HBase
Issue Type: Task
Components: Filesystem Integration
Affects Versions: 1.0.0-alpha2
Reporter: Wei-Chiu Chuang
hbase-filesystem repo is on guava23.0:
{noformat}
$ grep -r "guava" .
./pom.xml:23.0
./hbase-oss/pom.xml: com.google.guava
./hbase-oss/pom.xml: guava
./hbase-oss/pom.xml: ${guava.version}
./hbase-oss/pom.xml:
[jira] [Resolved] (HBASE-24113) Upgrade the maven we use from 3.5.4 to 3.6.3 in nightlies
[ https://issues.apache.org/jira/browse/HBASE-24113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-24113. --- Resolution: Fixed Ok. Leaving as is. Only branch-2.3+ get maven 3.6.3. Old branches differ a bunch so patch doesn't go back easily. > Upgrade the maven we use from 3.5.4 to 3.6.3 in nightlies > - > > Key: HBASE-24113 > URL: https://issues.apache.org/jira/browse/HBASE-24113 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > I want to up parallelism of nightlies and hopefully improve stability. Lets > use latest maven, go from 3.5.4 to 3.6.3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24122) Change machine ulimit-l to ulimit-a so dumps full ulimit rather than just 'max locked memory'
[ https://issues.apache.org/jira/browse/HBASE-24122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-24122. --- Fix Version/s: 2.2.5 2.3.0 3.0.0 Release Note: Our 'Build Artifacts' have a machine directory under which we emit vitals on the host the build was run on. We used to emit the result of 'ulimit -l' as a file named 'ulimit-l'. This has been hijacked to instead emit result of running 'ulimit -a' which includes stat on ulimit -l. Assignee: Michael Stack Resolution: Fixed Pushed this on all branches after confirming it works on branch-2. > Change machine ulimit-l to ulimit-a so dumps full ulimit rather than just > 'max locked memory' > - > > Key: HBASE-24122 > URL: https://issues.apache.org/jira/browse/HBASE-24122 > Project: HBase > Issue Type: Bug > Components: build >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0, 2.3.0, 2.2.5 > > Attachments: > 0001-HBASE-24122-Change-machine-ulimit-l-to-ulimit-a-so-d.patch > > > Dump out full ulimit list under the machine dir job output rather than > one-liner. More utility. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24123) [Flakey Tests] TestThrift?ServerCmdLine
Michael Stack created HBASE-24123:
-
Summary: [Flakey Tests] TestThrift?ServerCmdLine
Key: HBASE-24123
URL: https://issues.apache.org/jira/browse/HBASE-24123
Project: HBase
Issue Type: Bug
Reporter: Michael Stack
TestThrift2ServerCmdLine and TestThriftServerCmdLine are flakey. They fail
frequently in local runs for myriad reasons: protocol failures, wrong answers
on query, connection resets, etc. Each test runs 40 different combinations of
config.
This is after work done to try and stabilize by fixing port clash issues.
Let me list in here the ways in which it fails. I will disable after I get a
good list. Will also try and do some more stabilization too in sub-issues. Lets
see which prevails; stabilization or disabling.
For starters, this morning I got this:
{code}
---
Test set: org.apache.hadoop.hbase.thrift2.TestThrift2ServerCmdLine
---
Tests run: 32, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 126.301 s <<<
FAILURE! - in org.apache.hadoop.hbase.thrift2.TestThrift2ServerCmdLine
org.apache.hadoop.hbase.thrift2.TestThrift2ServerCmdLine.testRunThriftServer[29]
Time elapsed: 0.964 s <<< ERROR!
java.lang.Exception: org.apache.thrift.transport.TTransportException:
java.net.SocketException: Connection reset
at
org.apache.hadoop.hbase.thrift2.TestThrift2ServerCmdLine.talkToThriftServer(TestThrift2ServerCmdLine.java:92)
Caused by: java.net.SocketException: Connection reset
at
org.apache.hadoop.hbase.thrift2.TestThrift2ServerCmdLine.talkToThriftServer(TestThrift2ServerCmdLine.java:92)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-24122) Change machine ulimit-l to ulimit-a so dumps full ulimit rather than just 'max locked memory'
Michael Stack created HBASE-24122: - Summary: Change machine ulimit-l to ulimit-a so dumps full ulimit rather than just 'max locked memory' Key: HBASE-24122 URL: https://issues.apache.org/jira/browse/HBASE-24122 Project: HBase Issue Type: Bug Components: build Reporter: Michael Stack Dump out full ulimit list under the machine dir job output rather than one-liner. More utility. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (HBASE-24113) Upgrade the maven we use from 3.5.4 to 3.6.3 in nightlies
[ https://issues.apache.org/jira/browse/HBASE-24113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack reopened HBASE-24113: --- Let me reopen this. I'll close it after I've pushed this change everywhere so one version of maven doing all our building. > Upgrade the maven we use from 3.5.4 to 3.6.3 in nightlies > - > > Key: HBASE-24113 > URL: https://issues.apache.org/jira/browse/HBASE-24113 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > I want to up parallelism of nightlies and hopefully improve stability. Lets > use latest maven, go from 3.5.4 to 3.6.3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-24119) Polish the protobuf usage in hbase-examples
[ https://issues.apache.org/jira/browse/HBASE-24119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-24119. --- Hadoop Flags: Reviewed Resolution: Fixed Merged to master. Thanks [~janh] and [~vjasani] for reviewing. > Polish the protobuf usage in hbase-examples > --- > > Key: HBASE-24119 > URL: https://issues.apache.org/jira/browse/HBASE-24119 > Project: HBase > Issue Type: Sub-task > Components: Protobufs >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24121) [Authorization] ServiceAuthorizationManager isn't dynamically updatable. And it should be.
Reid Chan created HBASE-24121: - Summary: [Authorization] ServiceAuthorizationManager isn't dynamically updatable. And it should be. Key: HBASE-24121 URL: https://issues.apache.org/jira/browse/HBASE-24121 Project: HBase Issue Type: Bug Reporter: Reid Chan Assignee: Reid Chan Fix For: 3.0.0, 2.3.0, 1.7.0, 1.4.14, 2.2.5 -- This message was sent by Atlassian Jira (v8.3.4#803005)
