[jira] [Commented] (HBASE-20520) Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize
[
https://issues.apache.org/jira/browse/HBASE-20520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478976#comment-16478976
]
Hudson commented on HBASE-20520:
Results for branch master
[build #334 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/334/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/334//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/334//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/334//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize
> --
>
> Key: HBASE-20520
> URL: https://issues.apache.org/jira/browse/HBASE-20520
> Project: HBase
> Issue Type: Sub-task
> Components: conf
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 2.0.1
>
> Attachments: HBASE-20520.branch-2.0.001.patch,
> HBASE-20520.branch-2.0.002.patch, HBASE-20520.branch-2.0.003.patch,
> HBASE-20520.branch-2.0.004.patch, HBASE-20520.branch-2.0.005.patch
>
>
> Good one found by our [~mdrob]. Problem in the parent issue where failed
> attempt doubling default block size but halving the size at which roll -- so
> we end up in roughly same place only we make less small WALs in those cases
> where we are writing furiously and fail to roll before starting a new block.
> From Drob: ".../we drop default log roll to 0.5, but we don't actually
> increase the block size... AbstractProtobufLogWriter.java:164 still uses
> default HDFS block sizing and AbstractFSWAL.java:408 block size is only used
> in a log message, never actually makes it to files...".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20520) Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize
[
https://issues.apache.org/jira/browse/HBASE-20520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478203#comment-16478203
]
Hudson commented on HBASE-20520:
Results for branch branch-2.0
[build #309 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/309/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/309//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/309//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/309//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize
> --
>
> Key: HBASE-20520
> URL: https://issues.apache.org/jira/browse/HBASE-20520
> Project: HBase
> Issue Type: Sub-task
> Components: conf
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 2.0.1
>
> Attachments: HBASE-20520.branch-2.0.001.patch,
> HBASE-20520.branch-2.0.002.patch, HBASE-20520.branch-2.0.003.patch,
> HBASE-20520.branch-2.0.004.patch, HBASE-20520.branch-2.0.005.patch
>
>
> Good one found by our [~mdrob]. Problem in the parent issue where failed
> attempt doubling default block size but halving the size at which roll -- so
> we end up in roughly same place only we make less small WALs in those cases
> where we are writing furiously and fail to roll before starting a new block.
> From Drob: ".../we drop default log roll to 0.5, but we don't actually
> increase the block size... AbstractProtobufLogWriter.java:164 still uses
> default HDFS block sizing and AbstractFSWAL.java:408 block size is only used
> in a log message, never actually makes it to files...".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20520) Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize
[
https://issues.apache.org/jira/browse/HBASE-20520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478197#comment-16478197
]
Hudson commented on HBASE-20520:
Results for branch branch-2
[build #744 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/744/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/744//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/744//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/744//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize
> --
>
> Key: HBASE-20520
> URL: https://issues.apache.org/jira/browse/HBASE-20520
> Project: HBase
> Issue Type: Sub-task
> Components: conf
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 2.0.1
>
> Attachments: HBASE-20520.branch-2.0.001.patch,
> HBASE-20520.branch-2.0.002.patch, HBASE-20520.branch-2.0.003.patch,
> HBASE-20520.branch-2.0.004.patch, HBASE-20520.branch-2.0.005.patch
>
>
> Good one found by our [~mdrob]. Problem in the parent issue where failed
> attempt doubling default block size but halving the size at which roll -- so
> we end up in roughly same place only we make less small WALs in those cases
> where we are writing furiously and fail to roll before starting a new block.
> From Drob: ".../we drop default log roll to 0.5, but we don't actually
> increase the block size... AbstractProtobufLogWriter.java:164 still uses
> default HDFS block sizing and AbstractFSWAL.java:408 block size is only used
> in a log message, never actually makes it to files...".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20520) Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize
[ https://issues.apache.org/jira/browse/HBASE-20520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477593#comment-16477593 ] Mike Drob commented on HBASE-20520: --- [~stack] - any activity here? > Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize > -- > > Key: HBASE-20520 > URL: https://issues.apache.org/jira/browse/HBASE-20520 > Project: HBase > Issue Type: Sub-task > Components: conf >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.1 > > Attachments: HBASE-20520.branch-2.0.001.patch, > HBASE-20520.branch-2.0.002.patch, HBASE-20520.branch-2.0.003.patch, > HBASE-20520.branch-2.0.004.patch, HBASE-20520.branch-2.0.005.patch > > > Good one found by our [~mdrob]. Problem in the parent issue where failed > attempt doubling default block size but halving the size at which roll -- so > we end up in roughly same place only we make less small WALs in those cases > where we are writing furiously and fail to roll before starting a new block. > From Drob: ".../we drop default log roll to 0.5, but we don't actually > increase the block size... AbstractProtobufLogWriter.java:164 still uses > default HDFS block sizing and AbstractFSWAL.java:408 block size is only used > in a log message, never actually makes it to files...". -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20520) Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize
[ https://issues.apache.org/jira/browse/HBASE-20520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471187#comment-16471187 ] Mike Drob commented on HBASE-20520: --- Left a minor nit on RB, otherwise patch looks good. +1 to commit it, sir. > Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize > -- > > Key: HBASE-20520 > URL: https://issues.apache.org/jira/browse/HBASE-20520 > Project: HBase > Issue Type: Sub-task > Components: conf >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.1 > > Attachments: HBASE-20520.branch-2.0.001.patch, > HBASE-20520.branch-2.0.002.patch, HBASE-20520.branch-2.0.003.patch, > HBASE-20520.branch-2.0.004.patch, HBASE-20520.branch-2.0.005.patch > > > Good one found by our [~mdrob]. Problem in the parent issue where failed > attempt doubling default block size but halving the size at which roll -- so > we end up in roughly same place only we make less small WALs in those cases > where we are writing furiously and fail to roll before starting a new block. > From Drob: ".../we drop default log roll to 0.5, but we don't actually > increase the block size... AbstractProtobufLogWriter.java:164 still uses > default HDFS block sizing and AbstractFSWAL.java:408 block size is only used > in a log message, never actually makes it to files...". -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20520) Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize
[
https://issues.apache.org/jira/browse/HBASE-20520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462036#comment-16462036
]
ramkrishna.s.vasudevan commented on HBASE-20520:
bq.Example please ramkrishna.s.vasudevan (HBASE-20503 ? Where Archiving seems
to cut in prematurely?)
{code}
2018-04-30 16:17:46,666 INFO [regionserver/aep-slave1:16020.logRoller]
wal.AbstractFSWAL: Rolled WAL
/hbase1/WALs/aep-slave1,16020,1525076228236/aep-slave1%2C16020%2C1525076228236.1525076265809
with entries=9184, filesize=128.70 MB; new WAL
/hbase1/WALs/aep-slave1,16020,1525076228236/aep-slave1%2C16020%2C1525076228236.1525076266640
2018-04-30 16:17:47,716 INFO [regionserver/aep-slave1:16020.logRoller]
wal.AbstractFSWAL: Rolled WAL
/hbase1/WALs/aep-slave1,16020,1525076228236/aep-slave1%2C16020%2C1525076228236.1525076266640
with entries=9802, filesize=137.23 MB; new WAL
/hbase1/WALs/aep-slave1,16020,1525076228236/aep-slave1%2C16020%2C1525076228236.1525076267674
{code}
Logs like above. Exactly in one second we have another log roll. And both logs
are of size ~130MB in size.
> Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize
> --
>
> Key: HBASE-20520
> URL: https://issues.apache.org/jira/browse/HBASE-20520
> Project: HBase
> Issue Type: Sub-task
> Components: conf
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 2.0.1
>
> Attachments: HBASE-20520.branch-2.0.001.patch,
> HBASE-20520.branch-2.0.002.patch, HBASE-20520.branch-2.0.003.patch,
> HBASE-20520.branch-2.0.004.patch, HBASE-20520.branch-2.0.005.patch
>
>
> Good one found by our [~mdrob]. Problem in the parent issue where failed
> attempt doubling default block size but halving the size at which roll -- so
> we end up in roughly same place only we make less small WALs in those cases
> where we are writing furiously and fail to roll before starting a new block.
> From Drob: ".../we drop default log roll to 0.5, but we don't actually
> increase the block size... AbstractProtobufLogWriter.java:164 still uses
> default HDFS block sizing and AbstractFSWAL.java:408 block size is only used
> in a log message, never actually makes it to files...".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20520) Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize
[ https://issues.apache.org/jira/browse/HBASE-20520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461893#comment-16461893 ] stack commented on HBASE-20520: --- Thanks [~mdrob] I got a little lost in the hierarchy here. We should do a bit of a refactor one day > Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize > -- > > Key: HBASE-20520 > URL: https://issues.apache.org/jira/browse/HBASE-20520 > Project: HBase > Issue Type: Sub-task > Components: conf >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.1 > > Attachments: HBASE-20520.branch-2.0.001.patch, > HBASE-20520.branch-2.0.002.patch, HBASE-20520.branch-2.0.003.patch, > HBASE-20520.branch-2.0.004.patch, HBASE-20520.branch-2.0.005.patch > > > Good one found by our [~mdrob]. Problem in the parent issue where failed > attempt doubling default block size but halving the size at which roll -- so > we end up in roughly same place only we make less small WALs in those cases > where we are writing furiously and fail to roll before starting a new block. > From Drob: ".../we drop default log roll to 0.5, but we don't actually > increase the block size... AbstractProtobufLogWriter.java:164 still uses > default HDFS block sizing and AbstractFSWAL.java:408 block size is only used > in a log message, never actually makes it to files...". -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20520) Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize
[ https://issues.apache.org/jira/browse/HBASE-20520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461842#comment-16461842 ] Mike Drob commented on HBASE-20520: --- change makes sense, v5 LGTM. +1 assuming QA is happy. > Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize > -- > > Key: HBASE-20520 > URL: https://issues.apache.org/jira/browse/HBASE-20520 > Project: HBase > Issue Type: Sub-task > Components: conf >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.1 > > Attachments: HBASE-20520.branch-2.0.001.patch, > HBASE-20520.branch-2.0.002.patch, HBASE-20520.branch-2.0.003.patch, > HBASE-20520.branch-2.0.004.patch, HBASE-20520.branch-2.0.005.patch > > > Good one found by our [~mdrob]. Problem in the parent issue where failed > attempt doubling default block size but halving the size at which roll -- so > we end up in roughly same place only we make less small WALs in those cases > where we are writing furiously and fail to roll before starting a new block. > From Drob: ".../we drop default log roll to 0.5, but we don't actually > increase the block size... AbstractProtobufLogWriter.java:164 still uses > default HDFS block sizing and AbstractFSWAL.java:408 block size is only used > in a log message, never actually makes it to files...". -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20520) Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize
[ https://issues.apache.org/jira/browse/HBASE-20520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461821#comment-16461821 ] stack commented on HBASE-20520: --- .005 Edit. Fix test. > Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize > -- > > Key: HBASE-20520 > URL: https://issues.apache.org/jira/browse/HBASE-20520 > Project: HBase > Issue Type: Sub-task > Components: conf >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.1 > > Attachments: HBASE-20520.branch-2.0.001.patch, > HBASE-20520.branch-2.0.002.patch, HBASE-20520.branch-2.0.003.patch, > HBASE-20520.branch-2.0.004.patch, HBASE-20520.branch-2.0.005.patch > > > Good one found by our [~mdrob]. Problem in the parent issue where failed > attempt doubling default block size but halving the size at which roll -- so > we end up in roughly same place only we make less small WALs in those cases > where we are writing furiously and fail to roll before starting a new block. > From Drob: ".../we drop default log roll to 0.5, but we don't actually > increase the block size... AbstractProtobufLogWriter.java:164 still uses > default HDFS block sizing and AbstractFSWAL.java:408 block size is only used > in a log message, never actually makes it to files...". -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20520) Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize
[
https://issues.apache.org/jira/browse/HBASE-20520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461702#comment-16461702
]
Hadoop QA commented on HBASE-20520:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.0 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
2s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
15s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
31s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
0s{color} | {color:green} branch-2.0 passed {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 4m
12s{color} | {color:blue} branch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
32s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
51s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
33s{color} | {color:green} branch-2.0 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 15m
22s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
1s{color} | {color:red} hbase-server: The patch generated 5 new + 51 unchanged
- 1 fixed = 56 total (was 52) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
59s{color} | {color:red} root: The patch generated 5 new + 51 unchanged - 1
fixed = 56 total (was 52) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 3m
50s{color} | {color:blue} patch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
34s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 7s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
40s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 32m 37s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} |
[jira] [Commented] (HBASE-20520) Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize
[ https://issues.apache.org/jira/browse/HBASE-20520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461604#comment-16461604 ] stack commented on HBASE-20520: --- Add test to verify default takes in both WAL providers. Let me see how this does. Will be back with an edit to see if can clean up some of these stream constuctions... hard to follow what is going on . > Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize > -- > > Key: HBASE-20520 > URL: https://issues.apache.org/jira/browse/HBASE-20520 > Project: HBase > Issue Type: Sub-task > Components: conf >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.1 > > Attachments: HBASE-20520.branch-2.0.001.patch, > HBASE-20520.branch-2.0.002.patch, HBASE-20520.branch-2.0.003.patch, > HBASE-20520.branch-2.0.004.patch > > > Good one found by our [~mdrob]. Problem in the parent issue where failed > attempt doubling default block size but halving the size at which roll -- so > we end up in roughly same place only we make less small WALs in those cases > where we are writing furiously and fail to roll before starting a new block. > From Drob: ".../we drop default log roll to 0.5, but we don't actually > increase the block size... AbstractProtobufLogWriter.java:164 still uses > default HDFS block sizing and AbstractFSWAL.java:408 block size is only used > in a log message, never actually makes it to files...". -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20520) Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize
[
https://issues.apache.org/jira/browse/HBASE-20520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461560#comment-16461560
]
Hadoop QA commented on HBASE-20520:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-2.0 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
47s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
9s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
2s{color} | {color:green} branch-2.0 passed {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 4m
23s{color} | {color:blue} branch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
39s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
59s{color} | {color:green} branch-2.0 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
36s{color} | {color:green} branch-2.0 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 1m
29s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 12m
45s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 12m 45s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 3m
51s{color} | {color:blue} patch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 2m
21s{color} | {color:red} patch has 19 errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m
28s{color} | {color:red} The patch causes 19 errors with Hadoop v2.6.5. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m
54s{color} | {color:red} The patch causes 19 errors with Hadoop v2.7.4. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m
32s{color} | {color:red} The patch causes 19 errors with Hadoop v3.0.0. {color}
|
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
21s{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
35s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 9m 58s{color}
| {color:red} root in the patch failed. {color} |
|
[jira] [Commented] (HBASE-20520) Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize
[ https://issues.apache.org/jira/browse/HBASE-20520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461468#comment-16461468 ] stack commented on HBASE-20520: --- bq. You think the too-early log roll maybe had perf impact on our write loads? There's a hiccup each time we roll where writes are blocked for a short while so yes, this will have had some impact but small in the scheme of things I'd say given the new WAL is made ahead of the roll and dealing with the old is backgrounded but let me remeasure (There is also our blocking when too many WALs but in my perf tests I'd turned that off). bq. In the logs I noticed that WAL A gets renamed to WAL B and in the immediate second WAL B gets renamed to WAL C. Is that because of the issue mentioned here? Example please [~ram_krish] (HBASE-20503 ? Where Archiving seems to cut in prematurely?) > Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize > -- > > Key: HBASE-20520 > URL: https://issues.apache.org/jira/browse/HBASE-20520 > Project: HBase > Issue Type: Sub-task > Components: conf >Reporter: stack >Priority: Major > > Good one found by our [~mdrob]. Problem in the parent issue where failed > attempt doubling default block size but halving the size at which roll -- so > we end up in roughly same place only we make less small WALs in those cases > where we are writing furiously and fail to roll before starting a new block. > From Drob: ".../we drop default log roll to 0.5, but we don't actually > increase the block size... AbstractProtobufLogWriter.java:164 still uses > default HDFS block sizing and AbstractFSWAL.java:408 block size is only used > in a log message, never actually makes it to files...". -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20520) Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize
[ https://issues.apache.org/jira/browse/HBASE-20520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461427#comment-16461427 ] ramkrishna.s.vasudevan commented on HBASE-20520: In the logs I noticed that WAL A gets renamed to WAL B and in the immediate second WAL B gets renamed to WAL C. Is that because of the issue mentioned here? > Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize > -- > > Key: HBASE-20520 > URL: https://issues.apache.org/jira/browse/HBASE-20520 > Project: HBase > Issue Type: Sub-task > Components: conf >Reporter: stack >Priority: Major > > Good one found by our [~mdrob]. Problem in the parent issue where failed > attempt doubling default block size but halving the size at which roll -- so > we end up in roughly same place only we make less small WALs in those cases > where we are writing furiously and fail to roll before starting a new block. > From Drob: ".../we drop default log roll to 0.5, but we don't actually > increase the block size... AbstractProtobufLogWriter.java:164 still uses > default HDFS block sizing and AbstractFSWAL.java:408 block size is only used > in a log message, never actually makes it to files...". -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20520) Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize
[ https://issues.apache.org/jira/browse/HBASE-20520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461416#comment-16461416 ] Mike Drob commented on HBASE-20520: --- You think the too-early log roll maybe had perf impact on our write loads? > Failed effort upping default HDFS blocksize, hbase.regionserver.hlog.blocksize > -- > > Key: HBASE-20520 > URL: https://issues.apache.org/jira/browse/HBASE-20520 > Project: HBase > Issue Type: Sub-task > Components: conf >Reporter: stack >Priority: Major > > Good one found by our [~mdrob]. Problem in the parent issue where failed > attempt doubling default block size but halving the size at which roll -- so > we end up in roughly same place only we make less small WALs in those cases > where we are writing furiously and fail to roll before starting a new block. > From Drob: ".../we drop default log roll to 0.5, but we don't actually > increase the block size... AbstractProtobufLogWriter.java:164 still uses > default HDFS block sizing and AbstractFSWAL.java:408 block size is only used > in a log message, never actually makes it to files...". -- This message was sent by Atlassian JIRA (v7.6.3#76005)
