[jira] [Commented] (HBASE-15600) Add provision for adding mutations to memstore or able to write to same region in batchMutate coprocessor hooks
[
https://issues.apache.org/jira/browse/HBASE-15600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259599#comment-15259599
]
Anoop Sam John commented on HBASE-15600:
Looks like the set of ops doing in doMiniBatchMutate and processRowsWithLocks
(as part of mutateRow) is different ? Ya we can handle all such existing
things in another issue. And make these 2 share code as much as it can..
> Add provision for adding mutations to memstore or able to write to same
> region in batchMutate coprocessor hooks
> ---
>
> Key: HBASE-15600
> URL: https://issues.apache.org/jira/browse/HBASE-15600
> Project: HBase
> Issue Type: Improvement
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Rajeshbabu Chintaguntla
> Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.20
>
> Attachments: HBASE-15600.patch, HBASE-15600_v1.patch,
> HBASE-15600_v2.patch, hbase-15600_v3.patch, hbase-15600_v4.patch
>
>
> As part of PHOENIX-1734 we need to write the index updates to same region
> from coprocessors but writing from batchMutate API is not allowed because of
> mvcc.
> Raised PHOENIX-2742 to discuss any alternative way to write to the same
> region directly or not but not having any proper solution there.
> Currently we have provision to write wal edits from coprocessors. We can set
> wal edits in MiniBatchOperationInProgress.
> {noformat}
> /**
>* Sets the walEdit for the operation(Mutation) at the specified position.
>* @param index
>* @param walEdit
>*/
> public void setWalEdit(int index, WALEdit walEdit) {
> this.walEditsFromCoprocessors[getAbsoluteIndex(index)] = walEdit;
> }
> {noformat}
> Similarly we can allow to write mutations from coprocessors to memstore as
> well. Or else we should provide the batch mutation API allow write in batch
> mutate coprocessors.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15536) Make AsyncFSWAL as our default WAL
[ https://issues.apache.org/jira/browse/HBASE-15536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259597#comment-15259597 ] Anoop Sam John commented on HBASE-15536: So if the config 'hbase.wal.provider' is given as multiwal, it will be old way WAL with multiple WALs. We have added a new config to specify this multi wal thing any way. Just for confirmation asked. > Make AsyncFSWAL as our default WAL > -- > > Key: HBASE-15536 > URL: https://issues.apache.org/jira/browse/HBASE-15536 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-15536-v1.patch, HBASE-15536-v2.patch, > HBASE-15536.patch > > > As it should be predicated on passing basic cluster ITBLL -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15658) RegionServerCallable / RpcRetryingCaller clear meta cache on retries
[ https://issues.apache.org/jira/browse/HBASE-15658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259588#comment-15259588 ] Gary Helmling commented on HBASE-15658: --- The LinkVerifier job failed when committing output due to the number of counters, but it looks like the verification itself succeeded. Going to go ahead and commit this. > RegionServerCallable / RpcRetryingCaller clear meta cache on retries > > > Key: HBASE-15658 > URL: https://issues.apache.org/jira/browse/HBASE-15658 > Project: HBase > Issue Type: Sub-task > Components: Client >Affects Versions: 1.2.1 >Reporter: Gary Helmling >Assignee: Gary Helmling >Priority: Critical > Fix For: 1.3.0 > > Attachments: hbase-15658.001.patch, hbase-15658.002.patch, > hbase-15658.branch-1.3.001.patch > > > When RpcRetryingCaller.callWithRetries() attempts a retry, it calls > RetryingCallable.prepare(tries != 0). For RegionServerCallable (and probably > others), this will wind up calling > RegionLocator.getRegionLocation(reload=true), which will drop the meta cache > for the given region and always go back to meta. > This is kind of silly, since in the case of exceptions, we already call > RetryingCallable.throwable(), which goes to great pains to only refresh the > meta cache when necessary. Since we are already doing this on failure, I > don't really understand why we are doing duplicate work to refresh the meta > cache on prepare() at all. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15607) Remove PB references from Admin for 2.0
[ https://issues.apache.org/jira/browse/HBASE-15607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15607: --- Status: Open (was: Patch Available) > Remove PB references from Admin for 2.0 > --- > > Key: HBASE-15607 > URL: https://issues.apache.org/jira/browse/HBASE-15607 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-15607.patch, HBASE-15607_1.patch, > HBASE-15607_2.patch, HBASE-15607_3.patch > > > This is a sub-task for HBASE-15174. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15607) Remove PB references from Admin for 2.0
[ https://issues.apache.org/jira/browse/HBASE-15607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-15607: --- Attachment: HBASE-15607_3.patch Updated patch. All the failed tests will pass now. > Remove PB references from Admin for 2.0 > --- > > Key: HBASE-15607 > URL: https://issues.apache.org/jira/browse/HBASE-15607 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Fix For: 2.0.0 > > Attachments: HBASE-15607.patch, HBASE-15607_1.patch, > HBASE-15607_2.patch, HBASE-15607_3.patch > > > This is a sub-task for HBASE-15174. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15454) Archive store files older than max age
[ https://issues.apache.org/jira/browse/HBASE-15454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259573#comment-15259573 ] Clara Xiong commented on HBASE-15454: - Please give me some time to review. I am more concerned about triggering mechanism. Please do fully test it while we are reviewing. > Archive store files older than max age > -- > > Key: HBASE-15454 > URL: https://issues.apache.org/jira/browse/HBASE-15454 > Project: HBase > Issue Type: Sub-task > Components: Compaction >Affects Versions: 2.0.0, 1.3.0, 0.98.18, 1.4.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.20 > > Attachments: HBASE-15454-v1.patch, HBASE-15454-v2.patch, > HBASE-15454-v3.patch, HBASE-15454-v4.patch, HBASE-15454-v5.patch, > HBASE-15454-v6.patch, HBASE-15454-v7.patch, HBASE-15454.patch > > > In date tiered compaction, the store files older than max age are never > touched by minor compactions. Here we introduce a 'freeze window' operation, > which does the follow things: > 1. Find all store files that contains cells whose timestamp are in the give > window. > 2. Compaction all these files and output one file for each window that these > files covered. > After the compaction, we will have only one in the give window, and all cells > whose timestamp are in the give window are in the only file. And if you do > not write new cells with an older timestamp in this window, the file will > never be changed. This makes it easier to do erasure coding on the freezed > file to reduce redundence. And also, it makes it possible to check > consistency between master and peer cluster incrementally. > And why use the word 'freeze'? > Because there is already an 'HFileArchiver' class. I want to use a different > word to prevent confusing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15697) Excessive TestHRegion running time on branch-1
[
https://issues.apache.org/jira/browse/HBASE-15697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259571#comment-15259571
]
ramkrishna.s.vasudevan commented on HBASE-15697:
[~apurtell]
Can you try out this patch?
> Excessive TestHRegion running time on branch-1
> --
>
> Key: HBASE-15697
> URL: https://issues.apache.org/jira/browse/HBASE-15697
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Andrew Purtell
>Assignee: ramkrishna.s.vasudevan
> Attachments: HBASE-15697_branch-1.patch
>
>
> On my dev box TestHRegion takes about 90 seconds to complete in master and
> about 60 seconds in 0.98, but about 370 seconds in branch-1. Furthermore
> TestHRegion in branch-1 blew past my open files ulimit. I had to raise it
> from default in order for the unit to complete at all.
> I am going to bisect the recent history of branch-1 in search of a culprit
> and report back.
> {panel:title=master}
> Running org.apache.hadoop.hbase.regionserver.TestHRegion
> Tests run: 102, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 87.299 sec
> - in org.apache.hadoop.hbase.regionserver.TestHRegion
> Running org.apache.hadoop.hbase.regionserver.TestHRegion
> Tests run: 102, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 91.529 sec
> - in org.apache.hadoop.hbase.regionserver.TestHRegion
> Running org.apache.hadoop.hbase.regionserver.TestHRegion
> Tests run: 102, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 89.23 sec -
> in org.apache.hadoop.hbase.regionserver.TestHRegion
> {panel}
> {panel:title=branch-1}
> Running org.apache.hadoop.hbase.regionserver.TestHRegion
> Tests run: 102, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 368.868 sec
> - in org.apache.hadoop.hbase.regionserver.TestHRegion
> Running org.apache.hadoop.hbase.regionserver.TestHRegion
> Tests run: 102, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 366.203 sec
> - in org.apache.hadoop.hbase.regionserver.TestHRegion
> Running org.apache.hadoop.hbase.regionserver.TestHRegion
> Tests run: 102, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 345.806 sec
> - in org.apache.hadoop.hbase.regionserver.TestHRegion
> {panel}
> {panel:title=0.98}
> Running org.apache.hadoop.hbase.regionserver.TestHRegion
> Tests run: 90, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 61.038 sec -
> in org.apache.hadoop.hbase.regionserver.TestHRegion
> Running org.apache.hadoop.hbase.regionserver.TestHRegion
> Tests run: 90, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 56.382 sec -
> in org.apache.hadoop.hbase.regionserver.TestHRegion
> Running org.apache.hadoop.hbase.regionserver.TestHRegion
> Tests run: 90, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 63.509 sec -
> in org.apache.hadoop.hbase.regionserver.TestHRegion
> {panel}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15676) FuzzyRowFilter fails and matches all the rows in the table if the mask consists of all 0s
[
https://issues.apache.org/jira/browse/HBASE-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259563#comment-15259563
]
Hadoop QA commented on HBASE-15676:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 26s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
23s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 6s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
32s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
58s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 19s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 14s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
7s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 12s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 12s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m 12s
{color} | {color:red} hbase-client: patch generated 1 new + 16 unchanged - 1
fixed = 17 total (was 17) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m 32s
{color} | {color:red} hbase-server: patch generated 1 new + 16 unchanged - 1
fixed = 17 total (was 17) {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 6s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 16s
{color} | {color:red} hbase-client generated 1 new + 0 unchanged - 0 fixed = 1
total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 55s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 101m 39s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 157m 54s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-client |
| | Unsigned right shift cast to short/byte in
[jira] [Updated] (HBASE-15454) Archive store files older than max age
[ https://issues.apache.org/jira/browse/HBASE-15454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-15454: -- Attachment: HBASE-15454-v7.patch Addressing the comments on rb. > Archive store files older than max age > -- > > Key: HBASE-15454 > URL: https://issues.apache.org/jira/browse/HBASE-15454 > Project: HBase > Issue Type: Sub-task > Components: Compaction >Affects Versions: 2.0.0, 1.3.0, 0.98.18, 1.4.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.20 > > Attachments: HBASE-15454-v1.patch, HBASE-15454-v2.patch, > HBASE-15454-v3.patch, HBASE-15454-v4.patch, HBASE-15454-v5.patch, > HBASE-15454-v6.patch, HBASE-15454-v7.patch, HBASE-15454.patch > > > In date tiered compaction, the store files older than max age are never > touched by minor compactions. Here we introduce a 'freeze window' operation, > which does the follow things: > 1. Find all store files that contains cells whose timestamp are in the give > window. > 2. Compaction all these files and output one file for each window that these > files covered. > After the compaction, we will have only one in the give window, and all cells > whose timestamp are in the give window are in the only file. And if you do > not write new cells with an older timestamp in this window, the file will > never be changed. This makes it easier to do erasure coding on the freezed > file to reduce redundence. And also, it makes it possible to check > consistency between master and peer cluster incrementally. > And why use the word 'freeze'? > Because there is already an 'HFileArchiver' class. I want to use a different > word to prevent confusing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15454) Archive store files older than max age
[
https://issues.apache.org/jira/browse/HBASE-15454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259558#comment-15259558
]
Duo Zhang commented on HBASE-15454:
---
{quote}
I'm unsure about all the edge cases.
{quote}
That's why I introduce a flag to enable/disable the feature, and it is disabled
by default. I can test it in our production to see if it works well.
Thanks.
> Archive store files older than max age
> --
>
> Key: HBASE-15454
> URL: https://issues.apache.org/jira/browse/HBASE-15454
> Project: HBase
> Issue Type: Sub-task
> Components: Compaction
>Affects Versions: 2.0.0, 1.3.0, 0.98.18, 1.4.0
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.20
>
> Attachments: HBASE-15454-v1.patch, HBASE-15454-v2.patch,
> HBASE-15454-v3.patch, HBASE-15454-v4.patch, HBASE-15454-v5.patch,
> HBASE-15454-v6.patch, HBASE-15454.patch
>
>
> In date tiered compaction, the store files older than max age are never
> touched by minor compactions. Here we introduce a 'freeze window' operation,
> which does the follow things:
> 1. Find all store files that contains cells whose timestamp are in the give
> window.
> 2. Compaction all these files and output one file for each window that these
> files covered.
> After the compaction, we will have only one in the give window, and all cells
> whose timestamp are in the give window are in the only file. And if you do
> not write new cells with an older timestamp in this window, the file will
> never be changed. This makes it easier to do erasure coding on the freezed
> file to reduce redundence. And also, it makes it possible to check
> consistency between master and peer cluster incrementally.
> And why use the word 'freeze'?
> Because there is already an 'HFileArchiver' class. I want to use a different
> word to prevent confusing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15685) Typo in REST documentation
[ https://issues.apache.org/jira/browse/HBASE-15685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259527#comment-15259527 ] Bin Wang commented on HBASE-15685: -- The patch is available and passed the mvn site test, please review when you can. Thanks! > Typo in REST documentation > -- > > Key: HBASE-15685 > URL: https://issues.apache.org/jira/browse/HBASE-15685 > Project: HBase > Issue Type: Bug > Components: documentation >Reporter: Bin Wang >Assignee: Bin Wang >Priority: Minor > Attachments: HBASE-15685.patch > > Original Estimate: 2h > Time Spent: 3h > Remaining Estimate: 0h > > The Chapter - [REST|http://hbase.apache.org/book.html#_table_information] of > HBase Book has a few typo in the provided example links, like > "http://example.com:8000//:?v=" > which misses a forward slash between the port number 8000 and table name. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15714) We are calling checkRow() twice in doMiniBatchMutation()
[
https://issues.apache.org/jira/browse/HBASE-15714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259483#comment-15259483
]
Heng Chen commented on HBASE-15714:
---
To be safe, we could add one param in getRowLock to skip checkRow ?
> We are calling checkRow() twice in doMiniBatchMutation()
>
>
> Key: HBASE-15714
> URL: https://issues.apache.org/jira/browse/HBASE-15714
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
> Fix For: 2.0.0, 1.4.0
>
>
> In {{multi()}} -> {{doMiniBatchMutation()}} code path, we end up calling
> {{checkRow()}} twice, once from {{checkBatchOp()}} and once from
> {{getRowLock()}}.
> See [~anoop.hbase]'s comments at
> https://issues.apache.org/jira/browse/HBASE-15600?focusedCommentId=15257636=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15257636.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15536) Make AsyncFSWAL as our default WAL
[
https://issues.apache.org/jira/browse/HBASE-15536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-15536:
--
Release Note:
Now the default WALProvider is AsyncFSWALProvider, i.e. 'asyncfs'.
If you want to change back to use FSHLog, please add this in hbase-site.xml
{code}
hbase.wal.provider
filesystem
{code}
> Make AsyncFSWAL as our default WAL
> --
>
> Key: HBASE-15536
> URL: https://issues.apache.org/jira/browse/HBASE-15536
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.0.0
>Reporter: Duo Zhang
>Assignee: Duo Zhang
> Fix For: 2.0.0
>
> Attachments: HBASE-15536-v1.patch, HBASE-15536-v2.patch,
> HBASE-15536.patch
>
>
> As it should be predicated on passing basic cluster ITBLL
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15645) hbase.rpc.timeout is not used in operations of HTable
[
https://issues.apache.org/jira/browse/HBASE-15645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phil Yang updated HBASE-15645:
--
Release Note:
Fixes regression where hbase.rpc.timeout configuration was ignored in
branch-1.0+
Adds new methods setOperationTimeout, getOperationTimeout, setRpcTimeout, and
getRpcTimeout to Table. In branch-1.3+ they are public interfaces and in
1.0-1.2 they are labeled as @InterfaceAudience.Private.
Adds hbase.client.operation.timeout to hbase-default.xml with default of 120
was:
Fixes regression where hbase.rpc.timeout configuration was ignored in
branch-1.0+
Adds new methods setOperationTimeout, getOperationTimeout, setRpcTimeout, and
getRpcTimeout to Table
Adds hbase.client.operation.timeout to hbase-default.xml with default of 120
> hbase.rpc.timeout is not used in operations of HTable
> -
>
> Key: HBASE-15645
> URL: https://issues.apache.org/jira/browse/HBASE-15645
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.3.0, 1.2.1, 1.0.3, 1.1.4
>Reporter: Phil Yang
>Assignee: Phil Yang
>Priority: Critical
> Fix For: 2.0.0, 1.3.0, 1.0.4, 1.4.0, 1.1.5, 1.2.2
>
> Attachments: HBASE-15645-branch-1-v1.patch,
> HBASE-15645-branch-1.0-v1.patch, HBASE-15645-branch-1.1-v1.patch,
> HBASE-15645-branch-1.2-v1.patch, HBASE-15645-v1.patch, HBASE-15645-v2.patch,
> HBASE-15645-v3.patch, HBASE-15645-v4.patch, lable.patch
>
>
> While fixing HBASE-15593, I find that we use operationTimeout as the timeout
> of Get operation rpc call (hbase.client.scanner.timeout.period is used in
> scan rpc), not the hbase.rpc.timeout.
> This can be verified by add one line in TestHCM.setUpBeforeClass():
> {code}
> TEST_UTIL.getConfiguration().setLong(HConstants.HBASE_RPC_TIMEOUT_KEY, 3000);
> {code}
> and then run testOperationTimeout(), the test passes but it should have
> failed because we should get rpc timeout first after 3 seconds then client
> should retry and timeout again and again until operationTimeout or max
> retries reached.
> If I port this test to 0.98, it will fail as expected.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15645) hbase.rpc.timeout is not used in operations of HTable
[
https://issues.apache.org/jira/browse/HBASE-15645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phil Yang updated HBASE-15645:
--
Attachment: lable.patch
Upload a small patch based on current committed branches, add four
@InterfaceAudience.Private annotations. [~stack] Does it work for you?
> hbase.rpc.timeout is not used in operations of HTable
> -
>
> Key: HBASE-15645
> URL: https://issues.apache.org/jira/browse/HBASE-15645
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.3.0, 1.2.1, 1.0.3, 1.1.4
>Reporter: Phil Yang
>Assignee: Phil Yang
>Priority: Critical
> Fix For: 2.0.0, 1.3.0, 1.0.4, 1.4.0, 1.1.5, 1.2.2
>
> Attachments: HBASE-15645-branch-1-v1.patch,
> HBASE-15645-branch-1.0-v1.patch, HBASE-15645-branch-1.1-v1.patch,
> HBASE-15645-branch-1.2-v1.patch, HBASE-15645-v1.patch, HBASE-15645-v2.patch,
> HBASE-15645-v3.patch, HBASE-15645-v4.patch, lable.patch
>
>
> While fixing HBASE-15593, I find that we use operationTimeout as the timeout
> of Get operation rpc call (hbase.client.scanner.timeout.period is used in
> scan rpc), not the hbase.rpc.timeout.
> This can be verified by add one line in TestHCM.setUpBeforeClass():
> {code}
> TEST_UTIL.getConfiguration().setLong(HConstants.HBASE_RPC_TIMEOUT_KEY, 3000);
> {code}
> and then run testOperationTimeout(), the test passes but it should have
> failed because we should get rpc timeout first after 3 seconds then client
> should retry and timeout again and again until operationTimeout or max
> retries reached.
> If I port this test to 0.98, it will fail as expected.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15711) Add client side property to allow logging details for batch errors
[
https://issues.apache.org/jira/browse/HBASE-15711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259469#comment-15259469
]
Hadoop QA commented on HBASE-15711:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
52s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 17s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 18s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
5s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
58s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 20s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 27s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 18s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 18s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
5s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 14s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 54s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
7s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 39m 12s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12800807/HBASE-15711.patch |
| JIRA Issue | HBASE-15711 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / e512c40 |
| Default Java | 1.7.0_79 |
| Multi-JDK versions | /home/jenkins/tools/java/jdk1.8.0:1.8.0
/usr/local/jenkins/java/jdk1.7.0_79:1.7.0_79 |
| findbugs | v3.0.0
[jira] [Updated] (HBASE-15711) Add client side property to allow logging details for batch errors
[
https://issues.apache.org/jira/browse/HBASE-15711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yu Li updated HBASE-15711:
--
Status: Patch Available (was: Open)
Submit patch for HadoopQA
> Add client side property to allow logging details for batch errors
> --
>
> Key: HBASE-15711
> URL: https://issues.apache.org/jira/browse/HBASE-15711
> Project: HBase
> Issue Type: Bug
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-15711.patch
>
>
> Recently we observed below error message on client side when put process
> blocked:
> {noformat}
> 2016-04-26 10:27:11,707 ERROR [Sink: Unnamed (1/1)]
> com.alibaba.search.blink.streaming.connector.hbase.HBaseOutputFormat: Doing
> mutation failed
> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 1
> action: IOException: 1 time,
> at
> org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:228)
> at
> org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$1700(AsyncProcess.java:208)
> at
> org.apache.hadoop.hbase.client.AsyncProcess.waitForAllPreviousOpsAndReset(AsyncProcess.java:1694)
> at
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:208)
> at
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:183)
> {noformat}
> And checking RS logs we found nothing noticable. After adding some logging to
> show the detailed exception, it turns out something went wrong in one of our
> coprocessors:
> {noformat}
> 2016-04-26 12:03:13,776 ERROR [Sink: Unnamed (1/1)]
> org.apache.hadoop.hbase.client.AsyncProcess: Exception occurred! Exception
> details: [java.io.IOException: java.io.IOException: notify meta has not load
> success.
> at
> com.taobao.kart.coprocessor.server.CoprocessorNotifyMeta.checkMetaLoadSuccess(CoprocessorNotifyMeta.java:38)
> at
> com.taobao.kart.coprocessor.server.NotifyQualifySetter.updatePut(NotifyQualifySetter.java:47)
> at
> com.taobao.kart.coprocessor.server.NotifyQualifySetter.updatePut(NotifyQualifySetter.java:39)
> at
> com.taobao.kart.coprocessor.server.KartCoprocessor.prePut(KartCoprocessor.java:176)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$30.call(RegionCoprocessorHost.java:902)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionOperation.call(RegionCoprocessorHost.java:1673)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1748)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1705)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.prePut(RegionCoprocessorHost.java:898)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doPreMutationHook(HRegion.java:2890)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2865)
> {noformat}
> So in this JIRA we propose to add a property to allow logging detailed
> exception stacktrace rather than statistics for batch errors, and I believe
> this would be useful for debugging in some cases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15711) Add client side property to allow logging details for batch errors
[
https://issues.apache.org/jira/browse/HBASE-15711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259424#comment-15259424
]
Yu Li commented on HBASE-15711:
---
Thanks for review [~stack]
bq. Needs some doc though else we'll just forget it exists.
Agree. Where should be place the doc? Here in the release note, or
hbase-default.xml, or anywhere else? Thanks.
bq. PIty we could not dynamically set this config? Can we not? Via the UI as we
set log levels now? We could set a few UI configs... like this one?
It's also possible to dynamically set the client-side config such as in
AsyncProcess? I thought it's a server-side feature only... Could you give me
more hints here? Thanks.
> Add client side property to allow logging details for batch errors
> --
>
> Key: HBASE-15711
> URL: https://issues.apache.org/jira/browse/HBASE-15711
> Project: HBase
> Issue Type: Bug
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-15711.patch
>
>
> Recently we observed below error message on client side when put process
> blocked:
> {noformat}
> 2016-04-26 10:27:11,707 ERROR [Sink: Unnamed (1/1)]
> com.alibaba.search.blink.streaming.connector.hbase.HBaseOutputFormat: Doing
> mutation failed
> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 1
> action: IOException: 1 time,
> at
> org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:228)
> at
> org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$1700(AsyncProcess.java:208)
> at
> org.apache.hadoop.hbase.client.AsyncProcess.waitForAllPreviousOpsAndReset(AsyncProcess.java:1694)
> at
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:208)
> at
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:183)
> {noformat}
> And checking RS logs we found nothing noticable. After adding some logging to
> show the detailed exception, it turns out something went wrong in one of our
> coprocessors:
> {noformat}
> 2016-04-26 12:03:13,776 ERROR [Sink: Unnamed (1/1)]
> org.apache.hadoop.hbase.client.AsyncProcess: Exception occurred! Exception
> details: [java.io.IOException: java.io.IOException: notify meta has not load
> success.
> at
> com.taobao.kart.coprocessor.server.CoprocessorNotifyMeta.checkMetaLoadSuccess(CoprocessorNotifyMeta.java:38)
> at
> com.taobao.kart.coprocessor.server.NotifyQualifySetter.updatePut(NotifyQualifySetter.java:47)
> at
> com.taobao.kart.coprocessor.server.NotifyQualifySetter.updatePut(NotifyQualifySetter.java:39)
> at
> com.taobao.kart.coprocessor.server.KartCoprocessor.prePut(KartCoprocessor.java:176)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$30.call(RegionCoprocessorHost.java:902)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionOperation.call(RegionCoprocessorHost.java:1673)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1748)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1705)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.prePut(RegionCoprocessorHost.java:898)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doPreMutationHook(HRegion.java:2890)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2865)
> {noformat}
> So in this JIRA we propose to add a property to allow logging detailed
> exception stacktrace rather than statistics for batch errors, and I believe
> this would be useful for debugging in some cases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15536) Make AsyncFSWAL as our default WAL
[ https://issues.apache.org/jira/browse/HBASE-15536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259421#comment-15259421 ] Duo Zhang commented on HBASE-15536: --- Will retry, we are in an infinite loop here. Do I need to upload the patch to review board for a better review? Thanks. > Make AsyncFSWAL as our default WAL > -- > > Key: HBASE-15536 > URL: https://issues.apache.org/jira/browse/HBASE-15536 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0 > > Attachments: HBASE-15536-v1.patch, HBASE-15536-v2.patch, > HBASE-15536.patch > > > As it should be predicated on passing basic cluster ITBLL -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15676) FuzzyRowFilter fails and matches all the rows in the table if the mask consists of all 0s
[
https://issues.apache.org/jira/browse/HBASE-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-15676:
---
Attachment: hbase-15676-v2.patch
> FuzzyRowFilter fails and matches all the rows in the table if the mask
> consists of all 0s
> -
>
> Key: HBASE-15676
> URL: https://issues.apache.org/jira/browse/HBASE-15676
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 2.0.0, 0.98.13, 1.0.2, 1.2.0, 1.1.1
>Reporter: Rohit Sinha
> Attachments: hbase-15676-v1.patch, hbase-15676-v2.patch
>
>
> While using FuzzyRowFilter we noticed that if the mask array consists of all
> 0s (fixed) the FuzzyRowFilter matches all the rows in the table. We noticed
> this on HBase 1.1, 1.2 and higher.
> After some digging we suspect that this is because of isPreprocessedMask()
> check which is used in preprocessMask() which was added here:
> https://issues.apache.org/jira/browse/HBASE-13761
> If the mask consists of all 0s then the isPreprocessedMask() returns true and
> the preprocessing which responsible for changing 0s to -1 doesn't happen and
> hence all rows are matched in scan.
> This scenario can be tested in TestFuzzyRowFilterEndToEnd#testHBASE14782() If
> we change the
> byte[] fuzzyKey = Bytes.toBytesBinary("\\x00\\x00\\x044");
> byte[] mask = new byte[] {1,0,0,0};
> to
> byte[] fuzzyKey = Bytes.toBytesBinary("\\x9B\\x00\\x044e");
> byte[] mask = new byte[] {0,0,0,0,0};
> We expect one match but this will match all the rows in the table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15676) FuzzyRowFilter fails and matches all the rows in the table if the mask consists of all 0s
[
https://issues.apache.org/jira/browse/HBASE-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259418#comment-15259418
]
Matt Warhaftig commented on HBASE-15676:
{noformat}
"Have you thought about changing the encoding ( 0 -> -1 (0xff), 1 -> 0) so that
we can tell whether the mask has been preprocessed without introducing marker ?"
{noformat}
[~tedyu], thanks for the suggestion, that approach seems to work and is much
more elegant. Attached patch 'hbase-15676-v2.patch' uses new mask encoding of
0 -> -1 & 1 -> 2 in the FuzzyRowFilter constructor and then switches the mask
to needed values of -1 & 0 at filterKeyValue().
> FuzzyRowFilter fails and matches all the rows in the table if the mask
> consists of all 0s
> -
>
> Key: HBASE-15676
> URL: https://issues.apache.org/jira/browse/HBASE-15676
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 2.0.0, 0.98.13, 1.0.2, 1.2.0, 1.1.1
>Reporter: Rohit Sinha
> Attachments: hbase-15676-v1.patch
>
>
> While using FuzzyRowFilter we noticed that if the mask array consists of all
> 0s (fixed) the FuzzyRowFilter matches all the rows in the table. We noticed
> this on HBase 1.1, 1.2 and higher.
> After some digging we suspect that this is because of isPreprocessedMask()
> check which is used in preprocessMask() which was added here:
> https://issues.apache.org/jira/browse/HBASE-13761
> If the mask consists of all 0s then the isPreprocessedMask() returns true and
> the preprocessing which responsible for changing 0s to -1 doesn't happen and
> hence all rows are matched in scan.
> This scenario can be tested in TestFuzzyRowFilterEndToEnd#testHBASE14782() If
> we change the
> byte[] fuzzyKey = Bytes.toBytesBinary("\\x00\\x00\\x044");
> byte[] mask = new byte[] {1,0,0,0};
> to
> byte[] fuzzyKey = Bytes.toBytesBinary("\\x9B\\x00\\x044e");
> byte[] mask = new byte[] {0,0,0,0,0};
> We expect one match but this will match all the rows in the table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15710) Include issue servers information in RetriesExhaustedWithDetailsException message
[
https://issues.apache.org/jira/browse/HBASE-15710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259414#comment-15259414
]
Yu Li commented on HBASE-15710:
---
Thanks for review [~stack]
> Include issue servers information in RetriesExhaustedWithDetailsException
> message
> -
>
> Key: HBASE-15710
> URL: https://issues.apache.org/jira/browse/HBASE-15710
> Project: HBase
> Issue Type: Bug
>Reporter: Yu Li
>Assignee: Yu Li
>Priority: Minor
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15710.patch
>
>
> In current {{RetriesExhaustedWithDetailsException#getDesc}}, we have
> constructed a StringBuilder to add information of issue servers but returned
> the wrong string:
> {code}
> public static String getDesc(List exceptions,
>List actions,
>List hostnamePort) {
> String s = getDesc(classifyExs(exceptions));
> StringBuilder addrs = new StringBuilder(s);
> addrs.append("servers with issues: ");
> Set uniqAddr = new HashSet();
> uniqAddr.addAll(hostnamePort);
> for(String addr : uniqAddr) {
> addrs.append(addr).append(", ");
> }
> return s;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15716) HRegion#RegionScannerImpl scannerReadPoints synchronization costs
[ https://issues.apache.org/jira/browse/HBASE-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259385#comment-15259385 ] stack commented on HBASE-15716: --- bq. The problem is that adding a readpoint and finding the oldest need to be in sync. The mvcc read point is itself an atomic and always ascending. getSmallestReadpoint starts with current mvcc read point gotten from mvcc. A thread will find its smallest read point by iterating the CHM or in the mvcc. I suppose there is the possibility that: # Start to construct scanner # Get read point A but we have not put it into the CHM yet # Another thread calls getSmallestReadpoint and gets current mvcc readpoint A... this is fine # Another thread moves the read point forward to B # Another thread again calls getSmallestReadpoint and gets the advanced mvcc readpoint B # Some cleanup of Cells happens based off readpoint B # NOW we add the readpoint A to the region CHM ... damage is done. > HRegion#RegionScannerImpl scannerReadPoints synchronization costs > - > > Key: HBASE-15716 > URL: https://issues.apache.org/jira/browse/HBASE-15716 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: stack > Attachments: 15716.prune.synchronizations.patch, Screen Shot > 2016-04-26 at 2.05.45 PM.png, Screen Shot 2016-04-26 at 2.06.14 PM.png, > Screen Shot 2016-04-26 at 2.07.06 PM.png, Screen Shot 2016-04-26 at 2.25.26 > PM.png, Screen Shot 2016-04-26 at 6.02.29 PM.png, remove_cslm.patch > > > Here is a [~lhofhansl] special. > When we construct the region scanner, we get our read point and then store it > with the scanner instance in a Region scoped CSLM. This is done under a > synchronize on the CSLM. > This synchronize on a region-scoped Map creating region scanners is the > outstanding point of lock contention according to flight recorder (My work > load is workload c, random reads). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15719) TestMasterCoprocessorExceptionWithAbort#testExceptionFromCoprocessorWhenCreatingTable fails due to premature master abortion
[
https://issues.apache.org/jira/browse/HBASE-15719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259375#comment-15259375
]
Hadoop QA commented on HBASE-15719:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
8s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 40s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 31s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
52s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 31s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 44s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 44s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
36s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
25m 35s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 31s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 95m 26s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 143m 21s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.security.access.TestNamespaceCommands |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12800909/15719.v2.patch |
| JIRA Issue | HBASE-15719 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / e512c40 |
| Default Java | 1.7.0_79 |
| Multi-JDK versions | /home/jenkins/tools/java/jdk1.8.0:1.8.0
/usr/local/jenkins/java/jdk1.7.0_79:1.7.0_79 |
| findbugs | v3.0.0 |
| unit |
[jira] [Commented] (HBASE-15716) HRegion#RegionScannerImpl scannerReadPoints synchronization costs
[ https://issues.apache.org/jira/browse/HBASE-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259371#comment-15259371 ] stack commented on HBASE-15716: --- Smile. Relies on the fact that as we create new scanners, they'll always be with the same or newer read points. CHM which seems to have ok guarantees in face of concurrency. getSmallestReadPoint can be called by more than one thread and "Iterators and Enumerations return elements reflecting the state of the hash table at some point at or since the creation of the iterator/enumeration. ..., iterators are designed to be used by only one thread at a time.".. but should be ok since only one thread will iterate at a time. Was thinking change this to sortedset of read points and just get the oldest... then no iteration. Thanks for looking [~lhofhansl] > HRegion#RegionScannerImpl scannerReadPoints synchronization costs > - > > Key: HBASE-15716 > URL: https://issues.apache.org/jira/browse/HBASE-15716 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: stack > Attachments: 15716.prune.synchronizations.patch, Screen Shot > 2016-04-26 at 2.05.45 PM.png, Screen Shot 2016-04-26 at 2.06.14 PM.png, > Screen Shot 2016-04-26 at 2.07.06 PM.png, Screen Shot 2016-04-26 at 2.25.26 > PM.png, Screen Shot 2016-04-26 at 6.02.29 PM.png, remove_cslm.patch > > > Here is a [~lhofhansl] special. > When we construct the region scanner, we get our read point and then store it > with the scanner instance in a Region scoped CSLM. This is done under a > synchronize on the CSLM. > This synchronize on a region-scoped Map creating region scanners is the > outstanding point of lock contention according to flight recorder (My work > load is workload c, random reads). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15716) HRegion#RegionScannerImpl scannerReadPoints synchronization costs
[ https://issues.apache.org/jira/browse/HBASE-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259354#comment-15259354 ] Lars Hofhansl commented on HBASE-15716: --- How do you know it works? :) > HRegion#RegionScannerImpl scannerReadPoints synchronization costs > - > > Key: HBASE-15716 > URL: https://issues.apache.org/jira/browse/HBASE-15716 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: stack > Attachments: 15716.prune.synchronizations.patch, Screen Shot > 2016-04-26 at 2.05.45 PM.png, Screen Shot 2016-04-26 at 2.06.14 PM.png, > Screen Shot 2016-04-26 at 2.07.06 PM.png, Screen Shot 2016-04-26 at 2.25.26 > PM.png, Screen Shot 2016-04-26 at 6.02.29 PM.png, remove_cslm.patch > > > Here is a [~lhofhansl] special. > When we construct the region scanner, we get our read point and then store it > with the scanner instance in a Region scoped CSLM. This is done under a > synchronize on the CSLM. > This synchronize on a region-scoped Map creating region scanners is the > outstanding point of lock contention according to flight recorder (My work > load is workload c, random reads). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15634) TestDateTieredCompactionPolicy#negativeForMajor is flaky
[
https://issues.apache.org/jira/browse/HBASE-15634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259353#comment-15259353
]
Hudson commented on HBASE-15634:
ABORTED: Integrated in HBase-1.3 #672 (See
[https://builds.apache.org/job/HBase-1.3/672/])
HBASE-15634 TestDateTieredCompactionPolicy#negativeForMajor is flaky (tedyu:
rev 82544f47bbfde64a0c5b1f974e9fd782d2050fdc)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDateTieredCompactionPolicy.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DateTieredCompactionPolicy.java
> TestDateTieredCompactionPolicy#negativeForMajor is flaky
>
>
> Key: HBASE-15634
> URL: https://issues.apache.org/jira/browse/HBASE-15634
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.4.0
>Reporter: Ted Yu
>Assignee: Duo Zhang
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: 15634.v1.patch, 15634.v2.patch, 15634.v3.patch,
> HBASE-15634.patch
>
>
> https://builds.apache.org/job/PreCommit-HBASE-Build/1365/artifact/patchprocess/patch-unit-hbase-server.txt
> :
> {code}
> negativeForMajor(org.apache.hadoop.hbase.regionserver.TestDateTieredCompactionPolicy)
> Time elapsed: 0.48 sec <<< FAILURE!
> java.lang.AssertionError: expected: but was:
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:834)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hbase.regionserver.TestDateTieredCompactionPolicy.compactEquals(TestDateTieredCompactionPolicy.java:94)
> at
> org.apache.hadoop.hbase.regionserver.TestDateTieredCompactionPolicy.negativeForMajor(TestDateTieredCompactionPolicy.java:301)
> {code}
> Similar test failure occurred in master JDK 8 build as well
> (https://builds.apache.org/job/HBase-TRUNK_matrix/839/jdk=latest1.8,label=yahoo-not-h2/testReport/org.apache.hadoop.hbase.regionserver/TestDateTieredCompactionPolicy/negativeForMajor/).
> Since TestDateTieredCompactionPolicy is a small test, its failure prevented
> large tests from running.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15645) hbase.rpc.timeout is not used in operations of HTable
[
https://issues.apache.org/jira/browse/HBASE-15645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259352#comment-15259352
]
Hudson commented on HBASE-15645:
ABORTED: Integrated in HBase-1.3 #672 (See
[https://builds.apache.org/job/HBase-1.3/672/])
HBASE-15645 hbase.rpc.timeout is not used in operations of HTable (stack: rev
e917a74f077e1822ae9dc351c8e4f7e5d7cf1b30)
* hbase-common/src/main/resources/hbase-default.xml
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/client/HTableWrapper.java
* hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/Table.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerFactory.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCaller.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTablePool.java
> hbase.rpc.timeout is not used in operations of HTable
> -
>
> Key: HBASE-15645
> URL: https://issues.apache.org/jira/browse/HBASE-15645
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.3.0, 1.2.1, 1.0.3, 1.1.4
>Reporter: Phil Yang
>Assignee: Phil Yang
>Priority: Critical
> Fix For: 2.0.0, 1.3.0, 1.0.4, 1.4.0, 1.1.5, 1.2.2
>
> Attachments: HBASE-15645-branch-1-v1.patch,
> HBASE-15645-branch-1.0-v1.patch, HBASE-15645-branch-1.1-v1.patch,
> HBASE-15645-branch-1.2-v1.patch, HBASE-15645-v1.patch, HBASE-15645-v2.patch,
> HBASE-15645-v3.patch, HBASE-15645-v4.patch
>
>
> While fixing HBASE-15593, I find that we use operationTimeout as the timeout
> of Get operation rpc call (hbase.client.scanner.timeout.period is used in
> scan rpc), not the hbase.rpc.timeout.
> This can be verified by add one line in TestHCM.setUpBeforeClass():
> {code}
> TEST_UTIL.getConfiguration().setLong(HConstants.HBASE_RPC_TIMEOUT_KEY, 3000);
> {code}
> and then run testOperationTimeout(), the test passes but it should have
> failed because we should get rpc timeout first after 3 seconds then client
> should retry and timeout again and again until operationTimeout or max
> retries reached.
> If I port this test to 0.98, it will fail as expected.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15716) HRegion#RegionScannerImpl scannerReadPoints synchronization costs
[ https://issues.apache.org/jira/browse/HBASE-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259350#comment-15259350 ] Lars Hofhansl commented on HBASE-15716: --- Hmm... As is we need the synchronized I think. Off-hand I cannot think of any way out of this. The problem is that adding a readpoint and finding the oldest need to be in sync. "Add" alters the CSLM, in theory we cannot be entirely sure in what ways. Since it is true that we will *never* add an older scanner later, it *might* work to keep the CSLM and not synchronize - and anyway, we're taking the same risk by not synchronizing on removing already. We could potentially miss the first scanner added, though. Lemme mull this over some more. > HRegion#RegionScannerImpl scannerReadPoints synchronization costs > - > > Key: HBASE-15716 > URL: https://issues.apache.org/jira/browse/HBASE-15716 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: stack > Attachments: 15716.prune.synchronizations.patch, Screen Shot > 2016-04-26 at 2.05.45 PM.png, Screen Shot 2016-04-26 at 2.06.14 PM.png, > Screen Shot 2016-04-26 at 2.07.06 PM.png, Screen Shot 2016-04-26 at 2.25.26 > PM.png, Screen Shot 2016-04-26 at 6.02.29 PM.png, remove_cslm.patch > > > Here is a [~lhofhansl] special. > When we construct the region scanner, we get our read point and then store it > with the scanner instance in a Region scoped CSLM. This is done under a > synchronize on the CSLM. > This synchronize on a region-scoped Map creating region scanners is the > outstanding point of lock contention according to flight recorder (My work > load is workload c, random reads). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15626) RetriesExhaustedWithDetailsException#getDesc won't return the full message
[
https://issues.apache.org/jira/browse/HBASE-15626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259345#comment-15259345
]
Jianwei Cui commented on HBASE-15626:
-
Same problem as [HBASE-15710|https://issues.apache.org/jira/browse/HBASE-15710].
> RetriesExhaustedWithDetailsException#getDesc won't return the full message
> --
>
> Key: HBASE-15626
> URL: https://issues.apache.org/jira/browse/HBASE-15626
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 2.0.0
>Reporter: Jianwei Cui
>Priority: Minor
> Attachments: HBASE-15626-v1.patch
>
>
> The RetriesExhaustedWithDetailsException#getDesc will include server
> addresses as:
> {code}
> public static String getDesc(List exceptions,
>List actions,
>List hostnamePort) {
> String s = getDesc(classifyExs(exceptions));
> StringBuilder addrs = new StringBuilder(s);
> addrs.append("servers with issues: ");
> Set uniqAddr = new HashSet();
> uniqAddr.addAll(hostnamePort);
> for(String addr : uniqAddr) {
> addrs.append(addr).append(", ");
> }
> return s; // ==> should be addrs.toString()
> }
> {code}
> However, the returned value is {{s}}, only includes the exceptions. To
> include the server addresses, the returned value should be
> {{addrs.toString()}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15645) hbase.rpc.timeout is not used in operations of HTable
[
https://issues.apache.org/jira/browse/HBASE-15645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259300#comment-15259300
]
Hudson commented on HBASE-15645:
ABORTED: Integrated in HBase-1.2 #610 (See
[https://builds.apache.org/job/HBase-1.2/610/])
HBASE-15645 hbase.rpc.timeout is not used in operations of HTable (stack: rev
15ee4040aa3f2b7a65209eae3b47e2d6c82891e8)
* hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerFactory.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCaller.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java
* hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTablePool.java
* hbase-common/src/main/resources/hbase-default.xml
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/Table.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/client/HTableWrapper.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/StatsTrackingRpcRetryingCaller.java
> hbase.rpc.timeout is not used in operations of HTable
> -
>
> Key: HBASE-15645
> URL: https://issues.apache.org/jira/browse/HBASE-15645
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.3.0, 1.2.1, 1.0.3, 1.1.4
>Reporter: Phil Yang
>Assignee: Phil Yang
>Priority: Critical
> Fix For: 2.0.0, 1.3.0, 1.0.4, 1.4.0, 1.1.5, 1.2.2
>
> Attachments: HBASE-15645-branch-1-v1.patch,
> HBASE-15645-branch-1.0-v1.patch, HBASE-15645-branch-1.1-v1.patch,
> HBASE-15645-branch-1.2-v1.patch, HBASE-15645-v1.patch, HBASE-15645-v2.patch,
> HBASE-15645-v3.patch, HBASE-15645-v4.patch
>
>
> While fixing HBASE-15593, I find that we use operationTimeout as the timeout
> of Get operation rpc call (hbase.client.scanner.timeout.period is used in
> scan rpc), not the hbase.rpc.timeout.
> This can be verified by add one line in TestHCM.setUpBeforeClass():
> {code}
> TEST_UTIL.getConfiguration().setLong(HConstants.HBASE_RPC_TIMEOUT_KEY, 3000);
> {code}
> and then run testOperationTimeout(), the test passes but it should have
> failed because we should get rpc timeout first after 3 seconds then client
> should retry and timeout again and again until operationTimeout or max
> retries reached.
> If I port this test to 0.98, it will fail as expected.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15716) HRegion#RegionScannerImpl scannerReadPoints synchronization costs
[ https://issues.apache.org/jira/browse/HBASE-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15716: -- Attachment: Screen Shot 2016-04-26 at 6.02.29 PM.png 15716.prune.synchronizations.patch This patch works. Could make it dumber still by using a Set and just keeping the read point Longs. Thanks [~lhofhansl] Also, I think the bit where we do the cleanup of memstore versions is going to be subsumed by the segment stuff [~eshcar] is up to. > HRegion#RegionScannerImpl scannerReadPoints synchronization costs > - > > Key: HBASE-15716 > URL: https://issues.apache.org/jira/browse/HBASE-15716 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: stack > Attachments: 15716.prune.synchronizations.patch, Screen Shot > 2016-04-26 at 2.05.45 PM.png, Screen Shot 2016-04-26 at 2.06.14 PM.png, > Screen Shot 2016-04-26 at 2.07.06 PM.png, Screen Shot 2016-04-26 at 2.25.26 > PM.png, Screen Shot 2016-04-26 at 6.02.29 PM.png, remove_cslm.patch > > > Here is a [~lhofhansl] special. > When we construct the region scanner, we get our read point and then store it > with the scanner instance in a Region scoped CSLM. This is done under a > synchronize on the CSLM. > This synchronize on a region-scoped Map creating region scanners is the > outstanding point of lock contention according to flight recorder (My work > load is workload c, random reads). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15716) HRegion#RegionScannerImpl scannerReadPoints synchronization costs
[ https://issues.apache.org/jira/browse/HBASE-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259271#comment-15259271 ] Lars Hofhansl commented on HBASE-15716: --- Looking now. > HRegion#RegionScannerImpl scannerReadPoints synchronization costs > - > > Key: HBASE-15716 > URL: https://issues.apache.org/jira/browse/HBASE-15716 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: stack > Attachments: Screen Shot 2016-04-26 at 2.05.45 PM.png, Screen Shot > 2016-04-26 at 2.06.14 PM.png, Screen Shot 2016-04-26 at 2.07.06 PM.png, > Screen Shot 2016-04-26 at 2.25.26 PM.png, remove_cslm.patch > > > Here is a [~lhofhansl] special. > When we construct the region scanner, we get our read point and then store it > with the scanner instance in a Region scoped CSLM. This is done under a > synchronize on the CSLM. > This synchronize on a region-scoped Map creating region scanners is the > outstanding point of lock contention according to flight recorder (My work > load is workload c, random reads). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15717) TestClassLoading#testMasterCoprocessorsReported fails due to Master Coprocessors mismatch
[
https://issues.apache.org/jira/browse/HBASE-15717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15717:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks for the review.
> TestClassLoading#testMasterCoprocessorsReported fails due to Master
> Coprocessors mismatch
> -
>
> Key: HBASE-15717
> URL: https://issues.apache.org/jira/browse/HBASE-15717
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Labels: backup
> Attachments: 15717.v1.txt
>
>
> BackupController was not accounted for:
> {code}
> testMasterCoprocessorsReported(org.apache.hadoop.hbase.coprocessor.TestClassLoading)
> Time elapsed: 0.042 sec <<< FAILURE!
> org.junit.ComparisonFailure: expected:<[Ba[]seMasterObserver]> but
> was:<[Ba[ckupController, Ba]seMasterObserver]>
> at org.junit.Assert.assertEquals(Assert.java:115)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hbase.coprocessor.TestClassLoading.testMasterCoprocessorsReported(TestClassLoading.java:538)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15713) Backport "HBASE-15477 Do not save 'next block header' when we cache hfileblocks"
[
https://issues.apache.org/jira/browse/HBASE-15713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259244#comment-15259244
]
Hadoop QA commented on HBASE-15713:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 16 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 12s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
30s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 32s
{color} | {color:green} branch-1 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 57s
{color} | {color:green} branch-1 passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
35s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
50s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 12s
{color} | {color:green} branch-1 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s
{color} | {color:green} branch-1 passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 18s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 18s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 2s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
37s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
11m 43s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
34s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 17s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 49s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 79m 25s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 18s
{color} | {color:green} hbase-external-blockcache in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 115m 36s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12800896/15477.backport.branch-1.v8.patch
|
| JIRA Issue | HBASE-15713 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti
[jira] [Commented] (HBASE-15719) TestMasterCoprocessorExceptionWithAbort#testExceptionFromCoprocessorWhenCreatingTable fails due to premature master abortion
[
https://issues.apache.org/jira/browse/HBASE-15719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259222#comment-15259222
]
Hadoop QA commented on HBASE-15719:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
52s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 19s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 3s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 6m
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
44s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 21s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 21s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 56s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 7m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
33m 11s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 0s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 44s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 47s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 24m 12s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 92m 56s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12800900/15719.v1.patch |
| JIRA Issue | HBASE-15719 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux asf910.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / e512c40 |
| Default Java | 1.7.0_79 |
| Multi-JDK versions | /home/jenkins/tools/java/jdk1.8.0:1.8.0
/usr/local/jenkins/java/jdk1.7.0_79:1.7.0_79 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/1623/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test
[jira] [Commented] (HBASE-15645) hbase.rpc.timeout is not used in operations of HTable
[
https://issues.apache.org/jira/browse/HBASE-15645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259221#comment-15259221
]
Hudson commented on HBASE-15645:
SUCCESS: Integrated in HBase-1.1-JDK7 #1705 (See
[https://builds.apache.org/job/HBase-1.1-JDK7/1705/])
HBASE-15645 hbase.rpc.timeout is not used in operations of HTable (stack: rev
e5c1a80aad4e0926f32221ea36e6de68a8da4c27)
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTablePool.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
*
hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestFastFailWithoutTestUtil.java
* hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/client/HTableWrapper.java
* hbase-common/src/main/resources/hbase-default.xml
* hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCaller.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerFactory.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/StatsTrackingRpcRetryingCaller.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/Table.java
> hbase.rpc.timeout is not used in operations of HTable
> -
>
> Key: HBASE-15645
> URL: https://issues.apache.org/jira/browse/HBASE-15645
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.3.0, 1.2.1, 1.0.3, 1.1.4
>Reporter: Phil Yang
>Assignee: Phil Yang
>Priority: Critical
> Fix For: 2.0.0, 1.3.0, 1.0.4, 1.4.0, 1.1.5, 1.2.2
>
> Attachments: HBASE-15645-branch-1-v1.patch,
> HBASE-15645-branch-1.0-v1.patch, HBASE-15645-branch-1.1-v1.patch,
> HBASE-15645-branch-1.2-v1.patch, HBASE-15645-v1.patch, HBASE-15645-v2.patch,
> HBASE-15645-v3.patch, HBASE-15645-v4.patch
>
>
> While fixing HBASE-15593, I find that we use operationTimeout as the timeout
> of Get operation rpc call (hbase.client.scanner.timeout.period is used in
> scan rpc), not the hbase.rpc.timeout.
> This can be verified by add one line in TestHCM.setUpBeforeClass():
> {code}
> TEST_UTIL.getConfiguration().setLong(HConstants.HBASE_RPC_TIMEOUT_KEY, 3000);
> {code}
> and then run testOperationTimeout(), the test passes but it should have
> failed because we should get rpc timeout first after 3 seconds then client
> should retry and timeout again and again until operationTimeout or max
> retries reached.
> If I port this test to 0.98, it will fail as expected.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15645) hbase.rpc.timeout is not used in operations of HTable
[
https://issues.apache.org/jira/browse/HBASE-15645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259210#comment-15259210
]
Hudson commented on HBASE-15645:
SUCCESS: Integrated in HBase-1.1-JDK8 #1791 (See
[https://builds.apache.org/job/HBase-1.1-JDK8/1791/])
HBASE-15645 hbase.rpc.timeout is not used in operations of HTable (stack: rev
e5c1a80aad4e0926f32221ea36e6de68a8da4c27)
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTablePool.java
* hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerFactory.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCaller.java
* hbase-common/src/main/resources/hbase-default.xml
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
*
hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestFastFailWithoutTestUtil.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/client/HTableWrapper.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/StatsTrackingRpcRetryingCaller.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/Table.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
> hbase.rpc.timeout is not used in operations of HTable
> -
>
> Key: HBASE-15645
> URL: https://issues.apache.org/jira/browse/HBASE-15645
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.3.0, 1.2.1, 1.0.3, 1.1.4
>Reporter: Phil Yang
>Assignee: Phil Yang
>Priority: Critical
> Fix For: 2.0.0, 1.3.0, 1.0.4, 1.4.0, 1.1.5, 1.2.2
>
> Attachments: HBASE-15645-branch-1-v1.patch,
> HBASE-15645-branch-1.0-v1.patch, HBASE-15645-branch-1.1-v1.patch,
> HBASE-15645-branch-1.2-v1.patch, HBASE-15645-v1.patch, HBASE-15645-v2.patch,
> HBASE-15645-v3.patch, HBASE-15645-v4.patch
>
>
> While fixing HBASE-15593, I find that we use operationTimeout as the timeout
> of Get operation rpc call (hbase.client.scanner.timeout.period is used in
> scan rpc), not the hbase.rpc.timeout.
> This can be verified by add one line in TestHCM.setUpBeforeClass():
> {code}
> TEST_UTIL.getConfiguration().setLong(HConstants.HBASE_RPC_TIMEOUT_KEY, 3000);
> {code}
> and then run testOperationTimeout(), the test passes but it should have
> failed because we should get rpc timeout first after 3 seconds then client
> should retry and timeout again and again until operationTimeout or max
> retries reached.
> If I port this test to 0.98, it will fail as expected.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15680) Examples in shell help message for TIMERANGE scanner specifications should use milliseconds instead of seconds
[
https://issues.apache.org/jira/browse/HBASE-15680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259176#comment-15259176
]
Hudson commented on HBASE-15680:
FAILURE: Integrated in HBase-Trunk_matrix #873 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/873/])
HBASE-15680 Examples in shell help message for TIMERANGE scanner (stack: rev
e512c40ad58da28db42bb99de23d467d8db296cb)
* hbase-shell/src/main/ruby/shell/commands/scan.rb
* hbase-shell/src/main/ruby/shell/commands/set_visibility.rb
> Examples in shell help message for TIMERANGE scanner specifications should
> use milliseconds instead of seconds
> --
>
> Key: HBASE-15680
> URL: https://issues.apache.org/jira/browse/HBASE-15680
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Li Fanxi
>Priority: Trivial
> Labels: beginner
> Fix For: 2.0.0
>
> Attachments: HBASE-15680.patch
>
> Original Estimate: 5m
> Remaining Estimate: 5m
>
> {code}
> hbase> scan 't1', {COLUMNS => 'c1', TIMERANGE => [1303668804, 1303668904]}
> hbase> set_visibility 't1', '(A)|C', {COLUMNS => 'c1',
> TIMERANGE => [1303668804, 1303668904]}
> {code}
> Examples for 'scan' and 'set_visibility' command use 10-digit timestamp (in
> seconds) as the parameter for the TIMERANGE scanner specifications. However,
> TIMERANGE only accepts milliseconds as its parameters.
> To avoid misleading, suggest to improve the examples by using 13-digit
> timestamps (in milliseconds).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15645) hbase.rpc.timeout is not used in operations of HTable
[
https://issues.apache.org/jira/browse/HBASE-15645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259177#comment-15259177
]
Hudson commented on HBASE-15645:
FAILURE: Integrated in HBase-Trunk_matrix #873 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/873/])
HBASE-15645 hbase.rpc.timeout is not used in operations of HTable (stack: rev
53d7316075b02e64673b05a3a4a0db49e3756127)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/RegionAsTable.java
* hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java
* hbase-common/src/main/resources/hbase-default.xml
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerFactory.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/client/HTableWrapper.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerImpl.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/Table.java
> hbase.rpc.timeout is not used in operations of HTable
> -
>
> Key: HBASE-15645
> URL: https://issues.apache.org/jira/browse/HBASE-15645
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.3.0, 1.2.1, 1.0.3, 1.1.4
>Reporter: Phil Yang
>Assignee: Phil Yang
>Priority: Critical
> Fix For: 2.0.0, 1.3.0, 1.0.4, 1.4.0, 1.1.5, 1.2.2
>
> Attachments: HBASE-15645-branch-1-v1.patch,
> HBASE-15645-branch-1.0-v1.patch, HBASE-15645-branch-1.1-v1.patch,
> HBASE-15645-branch-1.2-v1.patch, HBASE-15645-v1.patch, HBASE-15645-v2.patch,
> HBASE-15645-v3.patch, HBASE-15645-v4.patch
>
>
> While fixing HBASE-15593, I find that we use operationTimeout as the timeout
> of Get operation rpc call (hbase.client.scanner.timeout.period is used in
> scan rpc), not the hbase.rpc.timeout.
> This can be verified by add one line in TestHCM.setUpBeforeClass():
> {code}
> TEST_UTIL.getConfiguration().setLong(HConstants.HBASE_RPC_TIMEOUT_KEY, 3000);
> {code}
> and then run testOperationTimeout(), the test passes but it should have
> failed because we should get rpc timeout first after 3 seconds then client
> should retry and timeout again and again until operationTimeout or max
> retries reached.
> If I port this test to 0.98, it will fail as expected.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15707) ImportTSV bulk output does not support tags with hfile.format.version=3
[
https://issues.apache.org/jira/browse/HBASE-15707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259175#comment-15259175
]
Hudson commented on HBASE-15707:
FAILURE: Integrated in HBase-Trunk_matrix #873 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/873/])
HBASE-15707 ImportTSV bulk output does not support tags with (tedyu: rev
ebb5d421f96b83628cbfc5dd9f41ba714e2adf2b)
* hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/HBaseContext.scala
*
hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat2.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat2.java
> ImportTSV bulk output does not support tags with hfile.format.version=3
> ---
>
> Key: HBASE-15707
> URL: https://issues.apache.org/jira/browse/HBASE-15707
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
>Affects Versions: 2.0.0, 1.3.0, 1.4.0, 1.1.5, 1.2.2, 1.0.5
>Reporter: huaxiang sun
> Fix For: 2.0.0
>
> Attachments: HBASE-15707-v001.patch, HBASE-15707-v002.patch
>
>
> Running the following command:
> {code}
> hbase hbase org.apache.hadoop.hbase.mapreduce.ImportTsv \
> -Dhfile.format.version=3 \
> -Dmapreduce.map.combine.minspills=1 \
> -Dimporttsv.separator=, \
> -Dimporttsv.skip.bad.lines=false \
> -Dimporttsv.columns="HBASE_ROW_KEY,cf1:a,HBASE_CELL_TTL" \
> -Dimporttsv.bulk.output=/tmp/testttl/output/1 \
> testttl \
> /tmp/testttl/input
> {code}
> The content of input is like:
> {code}
> row1,data1,0060
> row2,data2,0660
> row3,data3,0060
> row4,data4,0660
> {code}
> When running hfile tool with the output hfile, there is no ttl tag.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15719) TestMasterCoprocessorExceptionWithAbort#testExceptionFromCoprocessorWhenCreatingTable fails due to premature master abortion
[
https://issues.apache.org/jira/browse/HBASE-15719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259171#comment-15259171
]
Hadoop QA commented on HBASE-15719:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
17s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 32s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
43s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
58s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 28s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 32s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 33s
{color} | {color:red} hbase-server in the patch failed with JDK v1.8.0. {color}
|
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 33s {color}
| {color:red} hbase-server in the patch failed with JDK v1.8.0. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 30s
{color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_79.
{color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 30s {color}
| {color:red} hbase-server in the patch failed with JDK v1.7.0_79. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 0m 54s
{color} | {color:red} Patch causes 18 errors with Hadoop v2.4.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m 48s
{color} | {color:red} Patch causes 18 errors with Hadoop v2.4.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m 42s
{color} | {color:red} Patch causes 18 errors with Hadoop v2.5.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 3m 37s
{color} | {color:red} Patch causes 18 errors with Hadoop v2.5.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m 32s
{color} | {color:red} Patch causes 18 errors with Hadoop v2.5.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 5m 27s
{color} | {color:red} Patch causes 18 errors with Hadoop v2.6.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m 21s
{color} | {color:red} Patch causes 18 errors with Hadoop v2.6.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 7m 15s
{color} | {color:red} Patch causes 18 errors with Hadoop v2.6.3. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 8m 11s
{color} | {color:red} Patch causes 18 errors with Hadoop v2.7.1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 23s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 26s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} unit {color}
[jira] [Updated] (HBASE-15719) TestMasterCoprocessorExceptionWithAbort#testExceptionFromCoprocessorWhenCreatingTable fails due to premature master abortion
[
https://issues.apache.org/jira/browse/HBASE-15719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15719:
---
Attachment: 15719.v2.patch
Updated patch v2 without change for 15717
> TestMasterCoprocessorExceptionWithAbort#testExceptionFromCoprocessorWhenCreatingTable
> fails due to premature master abortion
>
>
> Key: HBASE-15719
> URL: https://issues.apache.org/jira/browse/HBASE-15719
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Labels: backup
> Attachments: 15719.v1.patch, 15719.v2.patch
>
>
> In HBASE-7912 branch, I saw:
> {code}
> testExceptionFromCoprocessorWhenCreatingTable(org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort)
> Time elapsed: 0.055 sec <<< ERROR!
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort.testExceptionFromCoprocessorWhenCreatingTable(TestMasterCoprocessorExceptionWithAbort.java:163)
> {code}
> This was due to BuggyMasterObserver throwing NPE when hbase:backup was
> finishing creation, leading to premature master abortion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15719) TestMasterCoprocessorExceptionWithAbort#testExceptionFromCoprocessorWhenCreatingTable fails due to premature master abortion
[
https://issues.apache.org/jira/browse/HBASE-15719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15719:
---
Attachment: (was: 15719.v2.patch)
> TestMasterCoprocessorExceptionWithAbort#testExceptionFromCoprocessorWhenCreatingTable
> fails due to premature master abortion
>
>
> Key: HBASE-15719
> URL: https://issues.apache.org/jira/browse/HBASE-15719
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Labels: backup
> Attachments: 15719.v1.patch
>
>
> In HBASE-7912 branch, I saw:
> {code}
> testExceptionFromCoprocessorWhenCreatingTable(org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort)
> Time elapsed: 0.055 sec <<< ERROR!
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort.testExceptionFromCoprocessorWhenCreatingTable(TestMasterCoprocessorExceptionWithAbort.java:163)
> {code}
> This was due to BuggyMasterObserver throwing NPE when hbase:backup was
> finishing creation, leading to premature master abortion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15719) TestMasterCoprocessorExceptionWithAbort#testExceptionFromCoprocessorWhenCreatingTable fails due to premature master abortion
[
https://issues.apache.org/jira/browse/HBASE-15719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259161#comment-15259161
]
Vladimir Rodionov commented on HBASE-15719:
---
Ted, it seems that your patch includes HBASE-15717.
> TestMasterCoprocessorExceptionWithAbort#testExceptionFromCoprocessorWhenCreatingTable
> fails due to premature master abortion
>
>
> Key: HBASE-15719
> URL: https://issues.apache.org/jira/browse/HBASE-15719
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Labels: backup
> Attachments: 15719.v1.patch, 15719.v2.patch
>
>
> In HBASE-7912 branch, I saw:
> {code}
> testExceptionFromCoprocessorWhenCreatingTable(org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort)
> Time elapsed: 0.055 sec <<< ERROR!
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort.testExceptionFromCoprocessorWhenCreatingTable(TestMasterCoprocessorExceptionWithAbort.java:163)
> {code}
> This was due to BuggyMasterObserver throwing NPE when hbase:backup was
> finishing creation, leading to premature master abortion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15717) TestClassLoading#testMasterCoprocessorsReported fails due to Master Coprocessors mismatch
[
https://issues.apache.org/jira/browse/HBASE-15717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259154#comment-15259154
]
Vladimir Rodionov commented on HBASE-15717:
---
Looks good.
> TestClassLoading#testMasterCoprocessorsReported fails due to Master
> Coprocessors mismatch
> -
>
> Key: HBASE-15717
> URL: https://issues.apache.org/jira/browse/HBASE-15717
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Labels: backup
> Attachments: 15717.v1.txt
>
>
> BackupController was not accounted for:
> {code}
> testMasterCoprocessorsReported(org.apache.hadoop.hbase.coprocessor.TestClassLoading)
> Time elapsed: 0.042 sec <<< FAILURE!
> org.junit.ComparisonFailure: expected:<[Ba[]seMasterObserver]> but
> was:<[Ba[ckupController, Ba]seMasterObserver]>
> at org.junit.Assert.assertEquals(Assert.java:115)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hbase.coprocessor.TestClassLoading.testMasterCoprocessorsReported(TestClassLoading.java:538)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-15720) Print row locks at the debug dump page
Enis Soztutar created HBASE-15720: - Summary: Print row locks at the debug dump page Key: HBASE-15720 URL: https://issues.apache.org/jira/browse/HBASE-15720 Project: HBase Issue Type: Improvement Reporter: Enis Soztutar We had to debug cases where some handlers are holding row locks for an extended time (and maybe leak) and other handlers are getting timeouts for obtaining row locks. We should add row lock information at the debug page at the RS UI to be able to live-debug such cases. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15719) TestMasterCoprocessorExceptionWithAbort#testExceptionFromCoprocessorWhenCreatingTable fails due to premature master abortion
[
https://issues.apache.org/jira/browse/HBASE-15719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15719:
---
Attachment: 15719.v2.patch
Patch v2 applies similar fix to TestMasterCoprocessorExceptionWithRemove
> TestMasterCoprocessorExceptionWithAbort#testExceptionFromCoprocessorWhenCreatingTable
> fails due to premature master abortion
>
>
> Key: HBASE-15719
> URL: https://issues.apache.org/jira/browse/HBASE-15719
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Labels: backup
> Attachments: 15719.v1.patch, 15719.v2.patch
>
>
> In HBASE-7912 branch, I saw:
> {code}
> testExceptionFromCoprocessorWhenCreatingTable(org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort)
> Time elapsed: 0.055 sec <<< ERROR!
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort.testExceptionFromCoprocessorWhenCreatingTable(TestMasterCoprocessorExceptionWithAbort.java:163)
> {code}
> This was due to BuggyMasterObserver throwing NPE when hbase:backup was
> finishing creation, leading to premature master abortion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15708) Docker for dev-support scripts
[ https://issues.apache.org/jira/browse/HBASE-15708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259113#comment-15259113 ] Appy commented on HBASE-15708: -- Didn't understand. Elaborate a bit please? > Docker for dev-support scripts > -- > > Key: HBASE-15708 > URL: https://issues.apache.org/jira/browse/HBASE-15708 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Attachments: HBASE-15708-master-v2.patch, > HBASE-15708-master-v3.patch, HBASE-15708-master.patch > > > Scripts in dev-support are limited in terms of dependencies by what's > installed on apache machines. Installing new stuff is not easy. It can be > painful even importing simple python libraries like 'requests'. This jira is > to add a single docker instance which we can tweak to build the right > environment for dev-support scripts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15686) unable to dynamically load a table coprocessor if it belongs in org.apache.hadoop
[
https://issues.apache.org/jira/browse/HBASE-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259104#comment-15259104
]
Enis Soztutar commented on HBASE-15686:
---
if that is the case, we should name it included instead. Ted, please also
rename corresponding method names, etc.
> unable to dynamically load a table coprocessor if it belongs in
> org.apache.hadoop
> -
>
> Key: HBASE-15686
> URL: https://issues.apache.org/jira/browse/HBASE-15686
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
>Affects Versions: 1.0.1
>Reporter: Sangjin Lee
>Assignee: Ted Yu
> Attachments: 15686.v2.txt, 15686.v3.txt, 15686.wip
>
>
> As part of Hadoop's Timeline Service v.2 (YARN-2928), we're adding a table
> coprocessor (YARN-4062). However, we're finding that the coprocessor cannot
> be loaded dynamically. A relevant snippet for the exception:
> {noformat}
> java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1329)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1269)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:398)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:42436)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:324)
> at
> org.apache.hadoop.hbase.master.HMaster.checkClassLoading(HMaster.java:1483)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1327)
> ... 8 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
> org.apache.hadoop.hbase.util.CoprocessorClassLoader.loadClass(CoprocessorClassLoader.java:275)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:322)
> ... 10 more
> {noformat}
> We tracked it down to the fact that {{CoprocessorClassLoader}} regarding all
> hadoop classes as exempt from loading from the coprocessor jar. Since our
> coprocessor sits in the coprocessor jar, and yet the loading of this class is
> delegated to the parent which does not have this jar, the classloading fails.
> What would be nice is the ability to exclude certain classes from the exempt
> classes so that they can be loaded via table coprocessor classloader. See
> hadoop's {{ApplicationClassLoader}} for a similar feature.
> Is there any other way to load this coprocessor at the table scope?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15717) TestClassLoading#testMasterCoprocessorsReported fails due to Master Coprocessors mismatch
[
https://issues.apache.org/jira/browse/HBASE-15717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259085#comment-15259085
]
Hadoop QA commented on HBASE-15717:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 1s
{color} | {color:blue} The patch file was not named according to hbase's naming
conventions. Please see
https://yetus.apache.org/documentation/0.2.1/precommit-patchnames for
instructions. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
20s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 58s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 5s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 6m
27s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
30s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
45s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 19s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 11s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 59s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 1m 46s
{color} | {color:red} hbase-server in the patch failed with JDK v1.8.0. {color}
|
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 46s {color}
| {color:red} hbase-server in the patch failed with JDK v1.8.0. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 1m 8s
{color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_79.
{color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 8s {color}
| {color:red} hbase-server in the patch failed with JDK v1.7.0_79. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 6m
32s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
30s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m 6s
{color} | {color:red} Patch causes 18 errors with Hadoop v2.4.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m 0s
{color} | {color:red} Patch causes 18 errors with Hadoop v2.4.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m 1s
{color} | {color:red} Patch causes 18 errors with Hadoop v2.5.0. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 7m 58s
{color} | {color:red} Patch causes 18 errors with Hadoop v2.5.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 9m 50s
{color} | {color:red} Patch causes 18 errors with Hadoop v2.5.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 11m 52s
{color} | {color:red} Patch causes 18 errors with Hadoop v2.6.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 13m 56s
{color} | {color:red} Patch causes 18 errors with Hadoop v2.6.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 15m 54s
{color} | {color:red} Patch causes 18 errors with Hadoop v2.6.3. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 17m 53s
{color} | {color:red} Patch causes 18 errors with Hadoop v2.7.1. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 50s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
[jira] [Commented] (HBASE-15710) Include issue servers information in RetriesExhaustedWithDetailsException message
[
https://issues.apache.org/jira/browse/HBASE-15710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259075#comment-15259075
]
Hudson commented on HBASE-15710:
FAILURE: Integrated in HBase-1.4 #116 (See
[https://builds.apache.org/job/HBase-1.4/116/])
HBASE-15710 Include issue servers information in (stack: rev
9f222d59e1fb025fb5f4b86f26fca6dade6a982b)
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RetriesExhaustedWithDetailsException.java
> Include issue servers information in RetriesExhaustedWithDetailsException
> message
> -
>
> Key: HBASE-15710
> URL: https://issues.apache.org/jira/browse/HBASE-15710
> Project: HBase
> Issue Type: Bug
>Reporter: Yu Li
>Assignee: Yu Li
>Priority: Minor
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15710.patch
>
>
> In current {{RetriesExhaustedWithDetailsException#getDesc}}, we have
> constructed a StringBuilder to add information of issue servers but returned
> the wrong string:
> {code}
> public static String getDesc(List exceptions,
>List actions,
>List hostnamePort) {
> String s = getDesc(classifyExs(exceptions));
> StringBuilder addrs = new StringBuilder(s);
> addrs.append("servers with issues: ");
> Set uniqAddr = new HashSet();
> uniqAddr.addAll(hostnamePort);
> for(String addr : uniqAddr) {
> addrs.append(addr).append(", ");
> }
> return s;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15719) TestMasterCoprocessorExceptionWithAbort#testExceptionFromCoprocessorWhenCreatingTable fails due to premature master abortion
[
https://issues.apache.org/jira/browse/HBASE-15719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15719:
---
Attachment: 15719.v1.patch
> TestMasterCoprocessorExceptionWithAbort#testExceptionFromCoprocessorWhenCreatingTable
> fails due to premature master abortion
>
>
> Key: HBASE-15719
> URL: https://issues.apache.org/jira/browse/HBASE-15719
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Labels: backup
> Attachments: 15719.v1.patch
>
>
> In HBASE-7912 branch, I saw:
> {code}
> testExceptionFromCoprocessorWhenCreatingTable(org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort)
> Time elapsed: 0.055 sec <<< ERROR!
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort.testExceptionFromCoprocessorWhenCreatingTable(TestMasterCoprocessorExceptionWithAbort.java:163)
> {code}
> This was due to BuggyMasterObserver throwing NPE when hbase:backup was
> finishing creation, leading to premature master abortion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15719) TestMasterCoprocessorExceptionWithAbort#testExceptionFromCoprocessorWhenCreatingTable fails due to premature master abortion
[
https://issues.apache.org/jira/browse/HBASE-15719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15719:
---
Labels: backup (was: )
Status: Patch Available (was: Open)
> TestMasterCoprocessorExceptionWithAbort#testExceptionFromCoprocessorWhenCreatingTable
> fails due to premature master abortion
>
>
> Key: HBASE-15719
> URL: https://issues.apache.org/jira/browse/HBASE-15719
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Labels: backup
> Attachments: 15719.v1.patch
>
>
> In HBASE-7912 branch, I saw:
> {code}
> testExceptionFromCoprocessorWhenCreatingTable(org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort)
> Time elapsed: 0.055 sec <<< ERROR!
> java.lang.NullPointerException: null
> at
> org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort.testExceptionFromCoprocessorWhenCreatingTable(TestMasterCoprocessorExceptionWithAbort.java:163)
> {code}
> This was due to BuggyMasterObserver throwing NPE when hbase:backup was
> finishing creation, leading to premature master abortion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15645) hbase.rpc.timeout is not used in operations of HTable
[
https://issues.apache.org/jira/browse/HBASE-15645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259073#comment-15259073
]
Hudson commented on HBASE-15645:
FAILURE: Integrated in HBase-1.4 #116 (See
[https://builds.apache.org/job/HBase-1.4/116/])
HBASE-15645 hbase.rpc.timeout is not used in operations of HTable (stack: rev
d5acbdd1e4b9f073afe7de25747e5ed9e1436741)
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/Table.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
* hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/client/HTableWrapper.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTablePool.java
* hbase-common/src/main/resources/hbase-default.xml
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerFactory.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCaller.java
> hbase.rpc.timeout is not used in operations of HTable
> -
>
> Key: HBASE-15645
> URL: https://issues.apache.org/jira/browse/HBASE-15645
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.3.0, 1.2.1, 1.0.3, 1.1.4
>Reporter: Phil Yang
>Assignee: Phil Yang
>Priority: Critical
> Fix For: 2.0.0, 1.3.0, 1.0.4, 1.4.0, 1.1.5, 1.2.2
>
> Attachments: HBASE-15645-branch-1-v1.patch,
> HBASE-15645-branch-1.0-v1.patch, HBASE-15645-branch-1.1-v1.patch,
> HBASE-15645-branch-1.2-v1.patch, HBASE-15645-v1.patch, HBASE-15645-v2.patch,
> HBASE-15645-v3.patch, HBASE-15645-v4.patch
>
>
> While fixing HBASE-15593, I find that we use operationTimeout as the timeout
> of Get operation rpc call (hbase.client.scanner.timeout.period is used in
> scan rpc), not the hbase.rpc.timeout.
> This can be verified by add one line in TestHCM.setUpBeforeClass():
> {code}
> TEST_UTIL.getConfiguration().setLong(HConstants.HBASE_RPC_TIMEOUT_KEY, 3000);
> {code}
> and then run testOperationTimeout(), the test passes but it should have
> failed because we should get rpc timeout first after 3 seconds then client
> should retry and timeout again and again until operationTimeout or max
> retries reached.
> If I port this test to 0.98, it will fail as expected.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15634) TestDateTieredCompactionPolicy#negativeForMajor is flaky
[
https://issues.apache.org/jira/browse/HBASE-15634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259074#comment-15259074
]
Hudson commented on HBASE-15634:
FAILURE: Integrated in HBase-1.4 #116 (See
[https://builds.apache.org/job/HBase-1.4/116/])
HBASE-15634 TestDateTieredCompactionPolicy#negativeForMajor is flaky (tedyu:
rev 2562043e23f64cced60a858e98dbb607bcc44400)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DateTieredCompactionPolicy.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDateTieredCompactionPolicy.java
> TestDateTieredCompactionPolicy#negativeForMajor is flaky
>
>
> Key: HBASE-15634
> URL: https://issues.apache.org/jira/browse/HBASE-15634
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.4.0
>Reporter: Ted Yu
>Assignee: Duo Zhang
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: 15634.v1.patch, 15634.v2.patch, 15634.v3.patch,
> HBASE-15634.patch
>
>
> https://builds.apache.org/job/PreCommit-HBASE-Build/1365/artifact/patchprocess/patch-unit-hbase-server.txt
> :
> {code}
> negativeForMajor(org.apache.hadoop.hbase.regionserver.TestDateTieredCompactionPolicy)
> Time elapsed: 0.48 sec <<< FAILURE!
> java.lang.AssertionError: expected: but was:
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:834)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hbase.regionserver.TestDateTieredCompactionPolicy.compactEquals(TestDateTieredCompactionPolicy.java:94)
> at
> org.apache.hadoop.hbase.regionserver.TestDateTieredCompactionPolicy.negativeForMajor(TestDateTieredCompactionPolicy.java:301)
> {code}
> Similar test failure occurred in master JDK 8 build as well
> (https://builds.apache.org/job/HBase-TRUNK_matrix/839/jdk=latest1.8,label=yahoo-not-h2/testReport/org.apache.hadoop.hbase.regionserver/TestDateTieredCompactionPolicy/negativeForMajor/).
> Since TestDateTieredCompactionPolicy is a small test, its failure prevented
> large tests from running.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-15719) TestMasterCoprocessorExceptionWithAbort#testExceptionFromCoprocessorWhenCreatingTable fails due to premature master abortion
Ted Yu created HBASE-15719:
--
Summary:
TestMasterCoprocessorExceptionWithAbort#testExceptionFromCoprocessorWhenCreatingTable
fails due to premature master abortion
Key: HBASE-15719
URL: https://issues.apache.org/jira/browse/HBASE-15719
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
In HBASE-7912 branch, I saw:
{code}
testExceptionFromCoprocessorWhenCreatingTable(org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort)
Time elapsed: 0.055 sec <<< ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort.testExceptionFromCoprocessorWhenCreatingTable(TestMasterCoprocessorExceptionWithAbort.java:163)
{code}
This was due to BuggyMasterObserver throwing NPE when hbase:backup was
finishing creation, leading to premature master abortion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15615) Wrong sleep time when RegionServerCallable need retry
[
https://issues.apache.org/jira/browse/HBASE-15615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259061#comment-15259061
]
Hadoop QA commented on HBASE-15615:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
0s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 11s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
33s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
30s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
53s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s
{color} | {color:green} master passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 8s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 8s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 53s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 53s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
32s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 40s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
35s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 0s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 127m 45s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
35s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 183m 41s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.security.access.TestAccessController3 |
| | hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures |
| Timed out junit tests |
org.apache.hadoop.hbase.security.access.TestNamespaceCommands |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12800850/HBASE-15615-v1.patch |
| JIRA Issue | HBASE-15615 |
| Optional Tests |
[jira] [Commented] (HBASE-15718) Add on TableName implementation and tests
[
https://issues.apache.org/jira/browse/HBASE-15718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259044#comment-15259044
]
Hadoop QA commented on HBASE-15718:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s {color}
| {color:red} HBASE-15718 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.2.1/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12800895/HBASE-15718-v1.patch |
| JIRA Issue | HBASE-15718 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/1621/console |
| Powered by | Apache Yetus 0.2.1 http://yetus.apache.org |
This message was automatically generated.
> Add on TableName implementation and tests
> -
>
> Key: HBASE-15718
> URL: https://issues.apache.org/jira/browse/HBASE-15718
> Project: HBase
> Issue Type: Sub-task
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Attachments: HBASE-15718-v1.patch, HBASE-15718.patch
>
>
> Table name will be needed to look up rows from meta. Add the implementation
> and a test around TableName.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15713) Backport "HBASE-15477 Do not save 'next block header' when we cache hfileblocks"
[ https://issues.apache.org/jira/browse/HBASE-15713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15713: -- Attachment: 15477.backport.branch-1.v8.patch Remove test of the 0.94 to 0.96 migration of namespaces. > Backport "HBASE-15477 Do not save 'next block header' when we cache > hfileblocks" > > > Key: HBASE-15713 > URL: https://issues.apache.org/jira/browse/HBASE-15713 > Project: HBase > Issue Type: Sub-task > Components: BucketCache >Affects Versions: 1.3.0, 1.4.0 >Reporter: stack >Assignee: stack > Fix For: 1.3.0, 1.4.0 > > Attachments: 15477.backport.branch-1.v7.patch, > 15477.backport.branch-1.v8.patch > > > Backport "HBASE-15477 Do not save 'next block header' when we cache > hfileblocks" > The backport involves removing support for hfile v1/hfileblockv1 support > which was our default before 0.92. The backport patch also removes tests that > depended on hfile v1, tests that were deprecated post 0.96 and to be removed > (0.96 was the release that would only run if the whole cluster was hfiles > > v1). > [~mantonov] Ok to commit the above to 1.3? Thanks boss. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15718) Add on TableName implementation and tests
[ https://issues.apache.org/jira/browse/HBASE-15718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-15718: -- Attachment: HBASE-15718-v1.patch > Add on TableName implementation and tests > - > > Key: HBASE-15718 > URL: https://issues.apache.org/jira/browse/HBASE-15718 > Project: HBase > Issue Type: Sub-task >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-15718-v1.patch, HBASE-15718.patch > > > Table name will be needed to look up rows from meta. Add the implementation > and a test around TableName. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15708) Docker for dev-support scripts
[ https://issues.apache.org/jira/browse/HBASE-15708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259028#comment-15259028 ] Elliott Clark commented on HBASE-15708: --- Use a single run command. That will keep the layers down. > Docker for dev-support scripts > -- > > Key: HBASE-15708 > URL: https://issues.apache.org/jira/browse/HBASE-15708 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Attachments: HBASE-15708-master-v2.patch, > HBASE-15708-master-v3.patch, HBASE-15708-master.patch > > > Scripts in dev-support are limited in terms of dependencies by what's > installed on apache machines. Installing new stuff is not easy. It can be > painful even importing simple python libraries like 'requests'. This jira is > to add a single docker instance which we can tweak to build the right > environment for dev-support scripts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15708) Docker for dev-support scripts
[
https://issues.apache.org/jira/browse/HBASE-15708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259024#comment-15259024
]
Hadoop QA commented on HBASE-15708:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
4s {color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
2s {color} | {color:green} There were no new shelldocs issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
41m 44s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 42m 28s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12800879/HBASE-15708-master-v3.patch
|
| JIRA Issue | HBASE-15708 |
| Optional Tests | asflicense shellcheck shelldocs |
| uname | Linux pomona.apache.org 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT
Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / e512c40 |
| shellcheck | v0.3.3 (This is an old version that has serious bugs. Consider
upgrading.) |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/1619/console |
| Powered by | Apache Yetus 0.2.1 http://yetus.apache.org |
This message was automatically generated.
> Docker for dev-support scripts
> --
>
> Key: HBASE-15708
> URL: https://issues.apache.org/jira/browse/HBASE-15708
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-15708-master-v2.patch,
> HBASE-15708-master-v3.patch, HBASE-15708-master.patch
>
>
> Scripts in dev-support are limited in terms of dependencies by what's
> installed on apache machines. Installing new stuff is not easy. It can be
> painful even importing simple python libraries like 'requests'. This jira is
> to add a single docker instance which we can tweak to build the right
> environment for dev-support scripts.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15718) Add on TableName implementation and tests
[ https://issues.apache.org/jira/browse/HBASE-15718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-15718: -- Status: Patch Available (was: Open) > Add on TableName implementation and tests > - > > Key: HBASE-15718 > URL: https://issues.apache.org/jira/browse/HBASE-15718 > Project: HBase > Issue Type: Sub-task >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-15718.patch > > > Table name will be needed to look up rows from meta. Add the implementation > and a test around TableName. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15718) Add on TableName implementation and tests
[ https://issues.apache.org/jira/browse/HBASE-15718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-15718: -- Attachment: HBASE-15718.patch https://reviews.facebook.net/D57285 > Add on TableName implementation and tests > - > > Key: HBASE-15718 > URL: https://issues.apache.org/jira/browse/HBASE-15718 > Project: HBase > Issue Type: Sub-task >Reporter: Elliott Clark >Assignee: Elliott Clark > Attachments: HBASE-15718.patch > > > Table name will be needed to look up rows from meta. Add the implementation > and a test around TableName. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15658) RegionServerCallable / RpcRetryingCaller clear meta cache on retries
[ https://issues.apache.org/jira/browse/HBASE-15658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259021#comment-15259021 ] Gary Helmling commented on HBASE-15658: --- I'm running this through ITBLL right now. In addition, I've done testing of various scenarios on a cluster with live traffic: # simulating high load so that CQTBE is thrown, triggering non-meta-clearing retries. Verified that no additional calls to meta occurred during this. # graceful rolling / hard killing regionservers. This of course triggered calls to meta, but verified that clients recovered normally with region movement. # rolling masters (hosting meta). Again verified no clients were stuck. I should have ITBLL results soon. > RegionServerCallable / RpcRetryingCaller clear meta cache on retries > > > Key: HBASE-15658 > URL: https://issues.apache.org/jira/browse/HBASE-15658 > Project: HBase > Issue Type: Sub-task > Components: Client >Affects Versions: 1.2.1 >Reporter: Gary Helmling >Assignee: Gary Helmling >Priority: Critical > Fix For: 1.3.0 > > Attachments: hbase-15658.001.patch, hbase-15658.002.patch, > hbase-15658.branch-1.3.001.patch > > > When RpcRetryingCaller.callWithRetries() attempts a retry, it calls > RetryingCallable.prepare(tries != 0). For RegionServerCallable (and probably > others), this will wind up calling > RegionLocator.getRegionLocation(reload=true), which will drop the meta cache > for the given region and always go back to meta. > This is kind of silly, since in the case of exceptions, we already call > RetryingCallable.throwable(), which goes to great pains to only refresh the > meta cache when necessary. Since we are already doing this on failure, I > don't really understand why we are doing duplicate work to refresh the meta > cache on prepare() at all. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15718) Add on TableName implementation and tests
Elliott Clark created HBASE-15718: - Summary: Add on TableName implementation and tests Key: HBASE-15718 URL: https://issues.apache.org/jira/browse/HBASE-15718 Project: HBase Issue Type: Sub-task Reporter: Elliott Clark Assignee: Elliott Clark Table name will be needed to look up rows from meta. Add the implementation and a test around TableName. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15686) unable to dynamically load a table coprocessor if it belongs in org.apache.hadoop
[
https://issues.apache.org/jira/browse/HBASE-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258984#comment-15258984
]
Sangjin Lee commented on HBASE-15686:
-
The meaning of "excluded.classes" is these classes will be excluded from
exemption so that this is bit of a double negative. These classes will actually
be loaded by the coprocessor classloader. Could that be a source of confusion?
> unable to dynamically load a table coprocessor if it belongs in
> org.apache.hadoop
> -
>
> Key: HBASE-15686
> URL: https://issues.apache.org/jira/browse/HBASE-15686
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
>Affects Versions: 1.0.1
>Reporter: Sangjin Lee
>Assignee: Ted Yu
> Attachments: 15686.v2.txt, 15686.v3.txt, 15686.wip
>
>
> As part of Hadoop's Timeline Service v.2 (YARN-2928), we're adding a table
> coprocessor (YARN-4062). However, we're finding that the coprocessor cannot
> be loaded dynamically. A relevant snippet for the exception:
> {noformat}
> java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1329)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1269)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:398)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:42436)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:324)
> at
> org.apache.hadoop.hbase.master.HMaster.checkClassLoading(HMaster.java:1483)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1327)
> ... 8 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
> org.apache.hadoop.hbase.util.CoprocessorClassLoader.loadClass(CoprocessorClassLoader.java:275)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:322)
> ... 10 more
> {noformat}
> We tracked it down to the fact that {{CoprocessorClassLoader}} regarding all
> hadoop classes as exempt from loading from the coprocessor jar. Since our
> coprocessor sits in the coprocessor jar, and yet the loading of this class is
> delegated to the parent which does not have this jar, the classloading fails.
> What would be nice is the ability to exclude certain classes from the exempt
> classes so that they can be loaded via table coprocessor classloader. See
> hadoop's {{ApplicationClassLoader}} for a similar feature.
> Is there any other way to load this coprocessor at the table scope?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15716) HRegion#RegionScannerImpl scannerReadPoints synchronization costs
[ https://issues.apache.org/jira/browse/HBASE-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15716: -- Attachment: Screen Shot 2016-04-26 at 2.25.26 PM.png remove_cslm.patch This patch removes the CSLM in favor of a HM since we don't need CSLM especially given we are synchronizing currently... only, it doesn't help. FR still notes the synchronizes (and throughput is same as w/ CSLM..) So, [~lhofhansl]... do we need to synchronize in here? We lock because we want the smallest read point... the oldest outstanding scanner. Does the lock help here? The lock will ensure we see newer scanners but we don't care about newer scanners just the one w/ the oldest read point. Can we just remove this synchronization and just pivot on the content of the scannerReadPoints CSLM? Let me put up a patch. > HRegion#RegionScannerImpl scannerReadPoints synchronization costs > - > > Key: HBASE-15716 > URL: https://issues.apache.org/jira/browse/HBASE-15716 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: stack > Attachments: Screen Shot 2016-04-26 at 2.05.45 PM.png, Screen Shot > 2016-04-26 at 2.06.14 PM.png, Screen Shot 2016-04-26 at 2.07.06 PM.png, > Screen Shot 2016-04-26 at 2.25.26 PM.png, remove_cslm.patch > > > Here is a [~lhofhansl] special. > When we construct the region scanner, we get our read point and then store it > with the scanner instance in a Region scoped CSLM. This is done under a > synchronize on the CSLM. > This synchronize on a region-scoped Map creating region scanners is the > outstanding point of lock contention according to flight recorder (My work > load is workload c, random reads). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15648) Reduce number of concurrent region location lookups when MetaCache entry is cleared
[ https://issues.apache.org/jira/browse/HBASE-15648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258967#comment-15258967 ] Elliott Clark commented on HBASE-15648: --- This one probably needs to be un-scheduled. It's a good idea. Using a lock to make sure that there aren't too many outstanding meta requests. However since the keys used are user specified the current implementation doesn't work. > Reduce number of concurrent region location lookups when MetaCache entry is > cleared > --- > > Key: HBASE-15648 > URL: https://issues.apache.org/jira/browse/HBASE-15648 > Project: HBase > Issue Type: Sub-task > Components: Client >Affects Versions: 1.3.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov > Attachments: HBASE-15648-branch-1.3.v1.patch > > > It seems in HConnectionImplementation#locateRegionInMeta if region location > is removed from the cache, with large number of client threads we could have > many of them getting cache miss and doing meta scan, which looks unnecessary > - we could empty mechanism similar to what we have in IdLock in HFileReader > to fetch the block to cache, do ensure that if one thread is already looking > up location for region R1, other threads who need it's location wait until > first thread finishes his work. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15648) Reduce number of concurrent region location lookups when MetaCache entry is cleared
[ https://issues.apache.org/jira/browse/HBASE-15648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-15648: -- Fix Version/s: (was: 1.3.0) > Reduce number of concurrent region location lookups when MetaCache entry is > cleared > --- > > Key: HBASE-15648 > URL: https://issues.apache.org/jira/browse/HBASE-15648 > Project: HBase > Issue Type: Sub-task > Components: Client >Affects Versions: 1.3.0 >Reporter: Mikhail Antonov >Assignee: Mikhail Antonov > Attachments: HBASE-15648-branch-1.3.v1.patch > > > It seems in HConnectionImplementation#locateRegionInMeta if region location > is removed from the cache, with large number of client threads we could have > many of them getting cache miss and doing meta scan, which looks unnecessary > - we could empty mechanism similar to what we have in IdLock in HFileReader > to fetch the block to cache, do ensure that if one thread is already looking > up location for region R1, other threads who need it's location wait until > first thread finishes his work. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15717) TestClassLoading#testMasterCoprocessorsReported fails due to Master Coprocessors mismatch
[
https://issues.apache.org/jira/browse/HBASE-15717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15717:
---
Status: Patch Available (was: Open)
> TestClassLoading#testMasterCoprocessorsReported fails due to Master
> Coprocessors mismatch
> -
>
> Key: HBASE-15717
> URL: https://issues.apache.org/jira/browse/HBASE-15717
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Labels: backup
> Attachments: 15717.v1.txt
>
>
> BackupController was not accounted for:
> {code}
> testMasterCoprocessorsReported(org.apache.hadoop.hbase.coprocessor.TestClassLoading)
> Time elapsed: 0.042 sec <<< FAILURE!
> org.junit.ComparisonFailure: expected:<[Ba[]seMasterObserver]> but
> was:<[Ba[ckupController, Ba]seMasterObserver]>
> at org.junit.Assert.assertEquals(Assert.java:115)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hbase.coprocessor.TestClassLoading.testMasterCoprocessorsReported(TestClassLoading.java:538)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15717) TestClassLoading#testMasterCoprocessorsReported fails due to Master Coprocessors mismatch
[
https://issues.apache.org/jira/browse/HBASE-15717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15717:
---
Attachment: 15717.v1.txt
> TestClassLoading#testMasterCoprocessorsReported fails due to Master
> Coprocessors mismatch
> -
>
> Key: HBASE-15717
> URL: https://issues.apache.org/jira/browse/HBASE-15717
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Labels: backup
> Attachments: 15717.v1.txt
>
>
> BackupController was not accounted for:
> {code}
> testMasterCoprocessorsReported(org.apache.hadoop.hbase.coprocessor.TestClassLoading)
> Time elapsed: 0.042 sec <<< FAILURE!
> org.junit.ComparisonFailure: expected:<[Ba[]seMasterObserver]> but
> was:<[Ba[ckupController, Ba]seMasterObserver]>
> at org.junit.Assert.assertEquals(Assert.java:115)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hbase.coprocessor.TestClassLoading.testMasterCoprocessorsReported(TestClassLoading.java:538)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-15717) TestClassLoading#testMasterCoprocessorsReported fails due to Master Coprocessors mismatch
Ted Yu created HBASE-15717:
--
Summary: TestClassLoading#testMasterCoprocessorsReported fails due
to Master Coprocessors mismatch
Key: HBASE-15717
URL: https://issues.apache.org/jira/browse/HBASE-15717
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
BackupController was not accounted for:
{code}
testMasterCoprocessorsReported(org.apache.hadoop.hbase.coprocessor.TestClassLoading)
Time elapsed: 0.042 sec <<< FAILURE!
org.junit.ComparisonFailure: expected:<[Ba[]seMasterObserver]> but
was:<[Ba[ckupController, Ba]seMasterObserver]>
at org.junit.Assert.assertEquals(Assert.java:115)
at org.junit.Assert.assertEquals(Assert.java:144)
at
org.apache.hadoop.hbase.coprocessor.TestClassLoading.testMasterCoprocessorsReported(TestClassLoading.java:538)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15715) BackupType misses @InterfaceAudience annotation
[
https://issues.apache.org/jira/browse/HBASE-15715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15715:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks for the review.
> BackupType misses @InterfaceAudience annotation
> ---
>
> Key: HBASE-15715
> URL: https://issues.apache.org/jira/browse/HBASE-15715
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Attachments: 15715.v1.patch
>
>
> I was running test suite in HBASE-7912 branch and stumbled over the following:
> {code}
> testInterfaceAudienceAnnotation(org.apache.hadoop.hbase.TestInterfaceAudienceAnnotations)
> Time elapsed: 0.344 sec <<< FAILURE!
> java.lang.AssertionError: All classes should have @InterfaceAudience
> annotation expected:<0> but was:<1>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at
> org.apache.hadoop.hbase.TestInterfaceAudienceAnnotations.testInterfaceAudienceAnnotation(TestInterfaceAudienceAnnotations.java:257)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15708) Docker for dev-support scripts
[ https://issues.apache.org/jira/browse/HBASE-15708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-15708: - Attachment: HBASE-15708-master-v3.patch > Docker for dev-support scripts > -- > > Key: HBASE-15708 > URL: https://issues.apache.org/jira/browse/HBASE-15708 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Attachments: HBASE-15708-master-v2.patch, > HBASE-15708-master-v3.patch, HBASE-15708-master.patch > > > Scripts in dev-support are limited in terms of dependencies by what's > installed on apache machines. Installing new stuff is not easy. It can be > painful even importing simple python libraries like 'requests'. This jira is > to add a single docker instance which we can tweak to build the right > environment for dev-support scripts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15716) HRegion#RegionScannerImpl scannerReadPoints synchronization costs
[ https://issues.apache.org/jira/browse/HBASE-15716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-15716: -- Attachment: Screen Shot 2016-04-26 at 2.07.06 PM.png Screen Shot 2016-04-26 at 2.05.45 PM.png Screen Shot 2016-04-26 at 2.06.14 PM.png Here are flight recordings of before and after. The before is current state of branch-1. The workload is ycsb c -- pure random read -- and the hit rate is about 220k/second. The after is my eliding the synchronization. See how our lock instances drops radically from 200k per thread per minute of my sample down to nothing (miscelllenous locking allocating BBs out of BucketCache). The speed up seen is not that much but nothing to sneer at... from 220k to 240k. > HRegion#RegionScannerImpl scannerReadPoints synchronization costs > - > > Key: HBASE-15716 > URL: https://issues.apache.org/jira/browse/HBASE-15716 > Project: HBase > Issue Type: Bug > Components: Performance >Reporter: stack > Attachments: Screen Shot 2016-04-26 at 2.05.45 PM.png, Screen Shot > 2016-04-26 at 2.06.14 PM.png, Screen Shot 2016-04-26 at 2.07.06 PM.png > > > Here is a [~lhofhansl] special. > When we construct the region scanner, we get our read point and then store it > with the scanner instance in a Region scoped CSLM. This is done under a > synchronize on the CSLM. > This synchronize on a region-scoped Map creating region scanners is the > outstanding point of lock contention according to flight recorder (My work > load is workload c, random reads). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15716) HRegion#RegionScannerImpl scannerReadPoints synchronization costs
stack created HBASE-15716: - Summary: HRegion#RegionScannerImpl scannerReadPoints synchronization costs Key: HBASE-15716 URL: https://issues.apache.org/jira/browse/HBASE-15716 Project: HBase Issue Type: Bug Components: Performance Reporter: stack Here is a [~lhofhansl] special. When we construct the region scanner, we get our read point and then store it with the scanner instance in a Region scoped CSLM. This is done under a synchronize on the CSLM. This synchronize on a region-scoped Map creating region scanners is the outstanding point of lock contention according to flight recorder (My work load is workload c, random reads). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15715) BackupType misses @InterfaceAudience annotation
[
https://issues.apache.org/jira/browse/HBASE-15715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258917#comment-15258917
]
Hadoop QA commented on HBASE-15715:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s {color}
| {color:red} HBASE-15715 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.2.1/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12800869/15715.v1.patch |
| JIRA Issue | HBASE-15715 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/1618/console |
| Powered by | Apache Yetus 0.2.1 http://yetus.apache.org |
This message was automatically generated.
> BackupType misses @InterfaceAudience annotation
> ---
>
> Key: HBASE-15715
> URL: https://issues.apache.org/jira/browse/HBASE-15715
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Attachments: 15715.v1.patch
>
>
> I was running test suite in HBASE-7912 branch and stumbled over the following:
> {code}
> testInterfaceAudienceAnnotation(org.apache.hadoop.hbase.TestInterfaceAudienceAnnotations)
> Time elapsed: 0.344 sec <<< FAILURE!
> java.lang.AssertionError: All classes should have @InterfaceAudience
> annotation expected:<0> but was:<1>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at
> org.apache.hadoop.hbase.TestInterfaceAudienceAnnotations.testInterfaceAudienceAnnotation(TestInterfaceAudienceAnnotations.java:257)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15454) Archive store files older than max age
[ https://issues.apache.org/jira/browse/HBASE-15454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258913#comment-15258913 ] Dave Latham commented on HBASE-15454: - Hi Duo, I posted a review up on reviewboard. I'm still a bit skeptical of this extension of DTCP, but I can see value for some cases. I'm worried about the level of complexity and wonder if there is a simpler way to get the desired outcome. I'm unsure about all the edge cases. > Archive store files older than max age > -- > > Key: HBASE-15454 > URL: https://issues.apache.org/jira/browse/HBASE-15454 > Project: HBase > Issue Type: Sub-task > Components: Compaction >Affects Versions: 2.0.0, 1.3.0, 0.98.18, 1.4.0 >Reporter: Duo Zhang >Assignee: Duo Zhang > Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.20 > > Attachments: HBASE-15454-v1.patch, HBASE-15454-v2.patch, > HBASE-15454-v3.patch, HBASE-15454-v4.patch, HBASE-15454-v5.patch, > HBASE-15454-v6.patch, HBASE-15454.patch > > > In date tiered compaction, the store files older than max age are never > touched by minor compactions. Here we introduce a 'freeze window' operation, > which does the follow things: > 1. Find all store files that contains cells whose timestamp are in the give > window. > 2. Compaction all these files and output one file for each window that these > files covered. > After the compaction, we will have only one in the give window, and all cells > whose timestamp are in the give window are in the only file. And if you do > not write new cells with an older timestamp in this window, the file will > never be changed. This makes it easier to do erasure coding on the freezed > file to reduce redundence. And also, it makes it possible to check > consistency between master and peer cluster incrementally. > And why use the word 'freeze'? > Because there is already an 'HFileArchiver' class. I want to use a different > word to prevent confusing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15715) BackupType misses @InterfaceAudience annotation
[
https://issues.apache.org/jira/browse/HBASE-15715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258914#comment-15258914
]
Vladimir Rodionov commented on HBASE-15715:
---
OK.
> BackupType misses @InterfaceAudience annotation
> ---
>
> Key: HBASE-15715
> URL: https://issues.apache.org/jira/browse/HBASE-15715
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Attachments: 15715.v1.patch
>
>
> I was running test suite in HBASE-7912 branch and stumbled over the following:
> {code}
> testInterfaceAudienceAnnotation(org.apache.hadoop.hbase.TestInterfaceAudienceAnnotations)
> Time elapsed: 0.344 sec <<< FAILURE!
> java.lang.AssertionError: All classes should have @InterfaceAudience
> annotation expected:<0> but was:<1>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at
> org.apache.hadoop.hbase.TestInterfaceAudienceAnnotations.testInterfaceAudienceAnnotation(TestInterfaceAudienceAnnotations.java:257)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15686) unable to dynamically load a table coprocessor if it belongs in org.apache.hadoop
[
https://issues.apache.org/jira/browse/HBASE-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15686:
---
Attachment: 15686.v3.txt
Renamed config as suggested
> unable to dynamically load a table coprocessor if it belongs in
> org.apache.hadoop
> -
>
> Key: HBASE-15686
> URL: https://issues.apache.org/jira/browse/HBASE-15686
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
>Affects Versions: 1.0.1
>Reporter: Sangjin Lee
>Assignee: Ted Yu
> Attachments: 15686.v2.txt, 15686.v3.txt, 15686.wip
>
>
> As part of Hadoop's Timeline Service v.2 (YARN-2928), we're adding a table
> coprocessor (YARN-4062). However, we're finding that the coprocessor cannot
> be loaded dynamically. A relevant snippet for the exception:
> {noformat}
> java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1329)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1269)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:398)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:42436)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:324)
> at
> org.apache.hadoop.hbase.master.HMaster.checkClassLoading(HMaster.java:1483)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1327)
> ... 8 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
> org.apache.hadoop.hbase.util.CoprocessorClassLoader.loadClass(CoprocessorClassLoader.java:275)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:322)
> ... 10 more
> {noformat}
> We tracked it down to the fact that {{CoprocessorClassLoader}} regarding all
> hadoop classes as exempt from loading from the coprocessor jar. Since our
> coprocessor sits in the coprocessor jar, and yet the loading of this class is
> delegated to the parent which does not have this jar, the classloading fails.
> What would be nice is the ability to exclude certain classes from the exempt
> classes so that they can be loaded via table coprocessor classloader. See
> hadoop's {{ApplicationClassLoader}} for a similar feature.
> Is there any other way to load this coprocessor at the table scope?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15715) BackupType misses @InterfaceAudience annotation
[
https://issues.apache.org/jira/browse/HBASE-15715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15715:
---
Attachment: 15715.v1.patch
> BackupType misses @InterfaceAudience annotation
> ---
>
> Key: HBASE-15715
> URL: https://issues.apache.org/jira/browse/HBASE-15715
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Attachments: 15715.v1.patch
>
>
> I was running test suite in HBASE-7912 branch and stumbled over the following:
> {code}
> testInterfaceAudienceAnnotation(org.apache.hadoop.hbase.TestInterfaceAudienceAnnotations)
> Time elapsed: 0.344 sec <<< FAILURE!
> java.lang.AssertionError: All classes should have @InterfaceAudience
> annotation expected:<0> but was:<1>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at
> org.apache.hadoop.hbase.TestInterfaceAudienceAnnotations.testInterfaceAudienceAnnotation(TestInterfaceAudienceAnnotations.java:257)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15715) BackupType misses @InterfaceAudience annotation
[
https://issues.apache.org/jira/browse/HBASE-15715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-15715:
---
Status: Patch Available (was: Open)
> BackupType misses @InterfaceAudience annotation
> ---
>
> Key: HBASE-15715
> URL: https://issues.apache.org/jira/browse/HBASE-15715
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
>Priority: Minor
> Attachments: 15715.v1.patch
>
>
> I was running test suite in HBASE-7912 branch and stumbled over the following:
> {code}
> testInterfaceAudienceAnnotation(org.apache.hadoop.hbase.TestInterfaceAudienceAnnotations)
> Time elapsed: 0.344 sec <<< FAILURE!
> java.lang.AssertionError: All classes should have @InterfaceAudience
> annotation expected:<0> but was:<1>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at
> org.apache.hadoop.hbase.TestInterfaceAudienceAnnotations.testInterfaceAudienceAnnotation(TestInterfaceAudienceAnnotations.java:257)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-15715) BackupType misses @InterfaceAudience annotation
Ted Yu created HBASE-15715:
--
Summary: BackupType misses @InterfaceAudience annotation
Key: HBASE-15715
URL: https://issues.apache.org/jira/browse/HBASE-15715
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
I was running test suite in HBASE-7912 branch and stumbled over the following:
{code}
testInterfaceAudienceAnnotation(org.apache.hadoop.hbase.TestInterfaceAudienceAnnotations)
Time elapsed: 0.344 sec <<< FAILURE!
java.lang.AssertionError: All classes should have @InterfaceAudience annotation
expected:<0> but was:<1>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at
org.apache.hadoop.hbase.TestInterfaceAudienceAnnotations.testInterfaceAudienceAnnotation(TestInterfaceAudienceAnnotations.java:257)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15645) hbase.rpc.timeout is not used in operations of HTable
[
https://issues.apache.org/jira/browse/HBASE-15645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258823#comment-15258823
]
Hudson commented on HBASE-15645:
SUCCESS: Integrated in HBase-1.2-IT #491 (See
[https://builds.apache.org/job/HBase-1.2-IT/491/])
HBASE-15645 hbase.rpc.timeout is not used in operations of HTable (stack: rev
15ee4040aa3f2b7a65209eae3b47e2d6c82891e8)
* hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/Table.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCaller.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTablePool.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/client/HTableWrapper.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/StatsTrackingRpcRetryingCaller.java
* hbase-common/src/main/resources/hbase-default.xml
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerFactory.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
> hbase.rpc.timeout is not used in operations of HTable
> -
>
> Key: HBASE-15645
> URL: https://issues.apache.org/jira/browse/HBASE-15645
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.3.0, 1.2.1, 1.0.3, 1.1.4
>Reporter: Phil Yang
>Assignee: Phil Yang
>Priority: Critical
> Fix For: 2.0.0, 1.3.0, 1.0.4, 1.4.0, 1.1.5, 1.2.2
>
> Attachments: HBASE-15645-branch-1-v1.patch,
> HBASE-15645-branch-1.0-v1.patch, HBASE-15645-branch-1.1-v1.patch,
> HBASE-15645-branch-1.2-v1.patch, HBASE-15645-v1.patch, HBASE-15645-v2.patch,
> HBASE-15645-v3.patch, HBASE-15645-v4.patch
>
>
> While fixing HBASE-15593, I find that we use operationTimeout as the timeout
> of Get operation rpc call (hbase.client.scanner.timeout.period is used in
> scan rpc), not the hbase.rpc.timeout.
> This can be verified by add one line in TestHCM.setUpBeforeClass():
> {code}
> TEST_UTIL.getConfiguration().setLong(HConstants.HBASE_RPC_TIMEOUT_KEY, 3000);
> {code}
> and then run testOperationTimeout(), the test passes but it should have
> failed because we should get rpc timeout first after 3 seconds then client
> should retry and timeout again and again until operationTimeout or max
> retries reached.
> If I port this test to 0.98, it will fail as expected.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15708) Docker for dev-support scripts
[ https://issues.apache.org/jira/browse/HBASE-15708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258819#comment-15258819 ] Nick Dimiduk commented on HBASE-15708: -- I didn't realize the intended target environment was jenkins. This seems reasonable for jenkins. > Docker for dev-support scripts > -- > > Key: HBASE-15708 > URL: https://issues.apache.org/jira/browse/HBASE-15708 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Attachments: HBASE-15708-master-v2.patch, HBASE-15708-master.patch > > > Scripts in dev-support are limited in terms of dependencies by what's > installed on apache machines. Installing new stuff is not easy. It can be > painful even importing simple python libraries like 'requests'. This jira is > to add a single docker instance which we can tweak to build the right > environment for dev-support scripts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15634) TestDateTieredCompactionPolicy#negativeForMajor is flaky
[
https://issues.apache.org/jira/browse/HBASE-15634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258812#comment-15258812
]
Hudson commented on HBASE-15634:
FAILURE: Integrated in HBase-Trunk_matrix #872 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/872/])
HBASE-15634 TestDateTieredCompactionPolicy#negativeForMajor is flaky (tedyu:
rev dce59294fffb0005db74ff6532fe8b356180d2a5)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDateTieredCompactionPolicy.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DateTieredCompactionPolicy.java
> TestDateTieredCompactionPolicy#negativeForMajor is flaky
>
>
> Key: HBASE-15634
> URL: https://issues.apache.org/jira/browse/HBASE-15634
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.4.0
>Reporter: Ted Yu
>Assignee: Duo Zhang
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: 15634.v1.patch, 15634.v2.patch, 15634.v3.patch,
> HBASE-15634.patch
>
>
> https://builds.apache.org/job/PreCommit-HBASE-Build/1365/artifact/patchprocess/patch-unit-hbase-server.txt
> :
> {code}
> negativeForMajor(org.apache.hadoop.hbase.regionserver.TestDateTieredCompactionPolicy)
> Time elapsed: 0.48 sec <<< FAILURE!
> java.lang.AssertionError: expected: but was:
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:834)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hbase.regionserver.TestDateTieredCompactionPolicy.compactEquals(TestDateTieredCompactionPolicy.java:94)
> at
> org.apache.hadoop.hbase.regionserver.TestDateTieredCompactionPolicy.negativeForMajor(TestDateTieredCompactionPolicy.java:301)
> {code}
> Similar test failure occurred in master JDK 8 build as well
> (https://builds.apache.org/job/HBase-TRUNK_matrix/839/jdk=latest1.8,label=yahoo-not-h2/testReport/org.apache.hadoop.hbase.regionserver/TestDateTieredCompactionPolicy/negativeForMajor/).
> Since TestDateTieredCompactionPolicy is a small test, its failure prevented
> large tests from running.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15477) Do not save 'next block header' when we cache hfileblocks
[
https://issues.apache.org/jira/browse/HBASE-15477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258811#comment-15258811
]
Hadoop QA commented on HBASE-15477:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 14 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 15s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
53s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 42s
{color} | {color:green} branch-1 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 14s
{color} | {color:green} branch-1 passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
49s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
43s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 47s
{color} | {color:green} branch-1 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 13s
{color} | {color:green} branch-1 passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
29s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 45s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 45s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 14s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 14s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 25s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
40s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 36s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 42s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 17s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 121m 50s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 25s
{color} | {color:green} hbase-external-blockcache in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
38s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 169m 39s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.snapshot.TestExportSnapshot |
| | hadoop.hbase.migration.TestNamespaceUpgrade |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
[jira] [Commented] (HBASE-15710) Include issue servers information in RetriesExhaustedWithDetailsException message
[
https://issues.apache.org/jira/browse/HBASE-15710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258813#comment-15258813
]
Hudson commented on HBASE-15710:
FAILURE: Integrated in HBase-Trunk_matrix #872 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/872/])
HBASE-15710 Include issue servers information in (stack: rev
4bdd47c52c62340de1ce707e6ca2c95b3660e5e4)
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RetriesExhaustedWithDetailsException.java
> Include issue servers information in RetriesExhaustedWithDetailsException
> message
> -
>
> Key: HBASE-15710
> URL: https://issues.apache.org/jira/browse/HBASE-15710
> Project: HBase
> Issue Type: Bug
>Reporter: Yu Li
>Assignee: Yu Li
>Priority: Minor
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15710.patch
>
>
> In current {{RetriesExhaustedWithDetailsException#getDesc}}, we have
> constructed a StringBuilder to add information of issue servers but returned
> the wrong string:
> {code}
> public static String getDesc(List exceptions,
>List actions,
>List hostnamePort) {
> String s = getDesc(classifyExs(exceptions));
> StringBuilder addrs = new StringBuilder(s);
> addrs.append("servers with issues: ");
> Set uniqAddr = new HashSet();
> uniqAddr.addAll(hostnamePort);
> for(String addr : uniqAddr) {
> addrs.append(addr).append(", ");
> }
> return s;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15686) unable to dynamically load a table coprocessor if it belongs in org.apache.hadoop
[
https://issues.apache.org/jira/browse/HBASE-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258789#comment-15258789
]
Enis Soztutar commented on HBASE-15686:
---
The patch for this adds a configuration option for excluded classes. If that is
the target goal, it should be backportable. There is no BC implications and it
is a low risk patch.
We should rename the config option though, instead of "coproc.exclusion", it
should be something like {{hbase.coprocessor.classloader.excluded.classes}}
> unable to dynamically load a table coprocessor if it belongs in
> org.apache.hadoop
> -
>
> Key: HBASE-15686
> URL: https://issues.apache.org/jira/browse/HBASE-15686
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors
>Affects Versions: 1.0.1
>Reporter: Sangjin Lee
>Assignee: Ted Yu
> Attachments: 15686.v2.txt, 15686.wip
>
>
> As part of Hadoop's Timeline Service v.2 (YARN-2928), we're adding a table
> coprocessor (YARN-4062). However, we're finding that the coprocessor cannot
> be loaded dynamically. A relevant snippet for the exception:
> {noformat}
> java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1329)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1269)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:398)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:42436)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: Class
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> cannot be loaded
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:324)
> at
> org.apache.hadoop.hbase.master.HMaster.checkClassLoading(HMaster.java:1483)
> at
> org.apache.hadoop.hbase.master.HMaster.sanityCheckTableDescriptor(HMaster.java:1327)
> ... 8 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.yarn.server.timelineservice.storage.flow.FlowRunCoprocessor
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
> org.apache.hadoop.hbase.util.CoprocessorClassLoader.loadClass(CoprocessorClassLoader.java:275)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs(RegionCoprocessorHost.java:322)
> ... 10 more
> {noformat}
> We tracked it down to the fact that {{CoprocessorClassLoader}} regarding all
> hadoop classes as exempt from loading from the coprocessor jar. Since our
> coprocessor sits in the coprocessor jar, and yet the loading of this class is
> delegated to the parent which does not have this jar, the classloading fails.
> What would be nice is the ability to exclude certain classes from the exempt
> classes so that they can be loaded via table coprocessor classloader. See
> hadoop's {{ApplicationClassLoader}} for a similar feature.
> Is there any other way to load this coprocessor at the table scope?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15600) Add provision for adding mutations to memstore or able to write to same region in batchMutate coprocessor hooks
[
https://issues.apache.org/jira/browse/HBASE-15600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258770#comment-15258770
]
Enis Soztutar commented on HBASE-15600:
---
Checked the code again. prepareDeleteTimestamps() and prepareDelete() are doing
different things and they are getting called in different times.
prepareDelete() is checking the "Delete row" operation, and fills in families
so that it turns into a set of DeleteFamily cells. It also checks the families
in the delete. prepareDeleteTimestamps() is doing an actual Get, in case delete
timestamps are LATEST_TIMESTAMP.
In the patch, we only call {{prepareDelete()}} for Delete's and we never call
{{prepareDeleteTimestamps()}} for Mutations injected from the coprocessor. This
assumes that the timestamps are already set by the coprocessor (see the javadoc
in addOperation()). Similarly for Puts, we do not call updateCellTimestamps().
Should we also consider calling these for Mutations from CP? It is not
immediately clear for non-Phoenix use cases whether this behavior is needed.
> Add provision for adding mutations to memstore or able to write to same
> region in batchMutate coprocessor hooks
> ---
>
> Key: HBASE-15600
> URL: https://issues.apache.org/jira/browse/HBASE-15600
> Project: HBase
> Issue Type: Improvement
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Rajeshbabu Chintaguntla
> Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.20
>
> Attachments: HBASE-15600.patch, HBASE-15600_v1.patch,
> HBASE-15600_v2.patch, hbase-15600_v3.patch, hbase-15600_v4.patch
>
>
> As part of PHOENIX-1734 we need to write the index updates to same region
> from coprocessors but writing from batchMutate API is not allowed because of
> mvcc.
> Raised PHOENIX-2742 to discuss any alternative way to write to the same
> region directly or not but not having any proper solution there.
> Currently we have provision to write wal edits from coprocessors. We can set
> wal edits in MiniBatchOperationInProgress.
> {noformat}
> /**
>* Sets the walEdit for the operation(Mutation) at the specified position.
>* @param index
>* @param walEdit
>*/
> public void setWalEdit(int index, WALEdit walEdit) {
> this.walEditsFromCoprocessors[getAbsoluteIndex(index)] = walEdit;
> }
> {noformat}
> Similarly we can allow to write mutations from coprocessors to memstore as
> well. Or else we should provide the batch mutation API allow write in batch
> mutate coprocessors.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-15714) We are calling checkRow() twice in doMiniBatchMutation()
Enis Soztutar created HBASE-15714:
-
Summary: We are calling checkRow() twice in doMiniBatchMutation()
Key: HBASE-15714
URL: https://issues.apache.org/jira/browse/HBASE-15714
Project: HBase
Issue Type: Bug
Reporter: Enis Soztutar
Fix For: 2.0.0, 1.4.0
In {{multi()}} -> {{doMiniBatchMutation()}} code path, we end up calling
{{checkRow()}} twice, once from {{checkBatchOp()}} and once from
{{getRowLock()}}.
See [~anoop.hbase]'s comments at
https://issues.apache.org/jira/browse/HBASE-15600?focusedCommentId=15257636=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15257636.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15600) Add provision for adding mutations to memstore or able to write to same region in batchMutate coprocessor hooks
[
https://issues.apache.org/jira/browse/HBASE-15600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258751#comment-15258751
]
Enis Soztutar commented on HBASE-15600:
---
bq. I can see there is a checkRow() call within checkAndPrepareMutation() and
getRowLock() again have checkRow() call.. Can we avoid?
This is even true for Mutations in regular doMiniBatchMutation(). Let's do the
fix in another jira since the fix is more generic and touches regular code
paths.
bq. This is within doMiniBatchMutate(). checkAndPrepareMutation seems doing
some thing else?
Yes, I had noticed this as well. We are calling checkRow(),
updateCellTimestamps() and prepareDeleteTimestamps() twice in regular
doMiniBatchMutation(). I would not complicate this patch for fixing those
though, because they are existing issues that affect non-cp code paths. Let me
open a follow up jira.
> Add provision for adding mutations to memstore or able to write to same
> region in batchMutate coprocessor hooks
> ---
>
> Key: HBASE-15600
> URL: https://issues.apache.org/jira/browse/HBASE-15600
> Project: HBase
> Issue Type: Improvement
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Rajeshbabu Chintaguntla
> Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.20
>
> Attachments: HBASE-15600.patch, HBASE-15600_v1.patch,
> HBASE-15600_v2.patch, hbase-15600_v3.patch, hbase-15600_v4.patch
>
>
> As part of PHOENIX-1734 we need to write the index updates to same region
> from coprocessors but writing from batchMutate API is not allowed because of
> mvcc.
> Raised PHOENIX-2742 to discuss any alternative way to write to the same
> region directly or not but not having any proper solution there.
> Currently we have provision to write wal edits from coprocessors. We can set
> wal edits in MiniBatchOperationInProgress.
> {noformat}
> /**
>* Sets the walEdit for the operation(Mutation) at the specified position.
>* @param index
>* @param walEdit
>*/
> public void setWalEdit(int index, WALEdit walEdit) {
> this.walEditsFromCoprocessors[getAbsoluteIndex(index)] = walEdit;
> }
> {noformat}
> Similarly we can allow to write mutations from coprocessors to memstore as
> well. Or else we should provide the batch mutation API allow write in batch
> mutate coprocessors.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15713) Backport "HBASE-15477 Do not save 'next block header' when we cache hfileblocks"
[
https://issues.apache.org/jira/browse/HBASE-15713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258741#comment-15258741
]
Hadoop QA commented on HBASE-15713:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 14 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 50s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
40s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 13s
{color} | {color:green} branch-1 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s
{color} | {color:green} branch-1 passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
46s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
45s {color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 11s
{color} | {color:green} branch-1 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s
{color} | {color:green} branch-1 passed with JDK v1.7.0_79 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 8s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
9s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 18s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 18s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
35s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
12m 1s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
38s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s
{color} | {color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 47s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 82m 29s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 18s
{color} | {color:green} hbase-external-blockcache in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
39s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 121m 15s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.migration.TestNamespaceUpgrade |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12800823/15477.backport.branch-1.v7.patch
|
| JIRA Issue | HBASE-15713 |
| Optional
[jira] [Commented] (HBASE-15600) Add provision for adding mutations to memstore or able to write to same region in batchMutate coprocessor hooks
[
https://issues.apache.org/jira/browse/HBASE-15600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258740#comment-15258740
]
Enis Soztutar commented on HBASE-15600:
---
Good catch. Let me fix it.
> Add provision for adding mutations to memstore or able to write to same
> region in batchMutate coprocessor hooks
> ---
>
> Key: HBASE-15600
> URL: https://issues.apache.org/jira/browse/HBASE-15600
> Project: HBase
> Issue Type: Improvement
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Rajeshbabu Chintaguntla
> Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.20
>
> Attachments: HBASE-15600.patch, HBASE-15600_v1.patch,
> HBASE-15600_v2.patch, hbase-15600_v3.patch, hbase-15600_v4.patch
>
>
> As part of PHOENIX-1734 we need to write the index updates to same region
> from coprocessors but writing from batchMutate API is not allowed because of
> mvcc.
> Raised PHOENIX-2742 to discuss any alternative way to write to the same
> region directly or not but not having any proper solution there.
> Currently we have provision to write wal edits from coprocessors. We can set
> wal edits in MiniBatchOperationInProgress.
> {noformat}
> /**
>* Sets the walEdit for the operation(Mutation) at the specified position.
>* @param index
>* @param walEdit
>*/
> public void setWalEdit(int index, WALEdit walEdit) {
> this.walEditsFromCoprocessors[getAbsoluteIndex(index)] = walEdit;
> }
> {noformat}
> Similarly we can allow to write mutations from coprocessors to memstore as
> well. Or else we should provide the batch mutation API allow write in batch
> mutate coprocessors.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15710) Include issue servers information in RetriesExhaustedWithDetailsException message
[
https://issues.apache.org/jira/browse/HBASE-15710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258733#comment-15258733
]
Hudson commented on HBASE-15710:
SUCCESS: Integrated in HBase-1.3-IT #635 (See
[https://builds.apache.org/job/HBase-1.3-IT/635/])
HBASE-15710 Include issue servers information in (stack: rev
3240c0e1ba4aab6a80e495efd36403a23d289781)
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RetriesExhaustedWithDetailsException.java
> Include issue servers information in RetriesExhaustedWithDetailsException
> message
> -
>
> Key: HBASE-15710
> URL: https://issues.apache.org/jira/browse/HBASE-15710
> Project: HBase
> Issue Type: Bug
>Reporter: Yu Li
>Assignee: Yu Li
>Priority: Minor
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: HBASE-15710.patch
>
>
> In current {{RetriesExhaustedWithDetailsException#getDesc}}, we have
> constructed a StringBuilder to add information of issue servers but returned
> the wrong string:
> {code}
> public static String getDesc(List exceptions,
>List actions,
>List hostnamePort) {
> String s = getDesc(classifyExs(exceptions));
> StringBuilder addrs = new StringBuilder(s);
> addrs.append("servers with issues: ");
> Set uniqAddr = new HashSet();
> uniqAddr.addAll(hostnamePort);
> for(String addr : uniqAddr) {
> addrs.append(addr).append(", ");
> }
> return s;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15634) TestDateTieredCompactionPolicy#negativeForMajor is flaky
[
https://issues.apache.org/jira/browse/HBASE-15634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258732#comment-15258732
]
Hudson commented on HBASE-15634:
SUCCESS: Integrated in HBase-1.3-IT #635 (See
[https://builds.apache.org/job/HBase-1.3-IT/635/])
HBASE-15634 TestDateTieredCompactionPolicy#negativeForMajor is flaky (tedyu:
rev 82544f47bbfde64a0c5b1f974e9fd782d2050fdc)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDateTieredCompactionPolicy.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DateTieredCompactionPolicy.java
> TestDateTieredCompactionPolicy#negativeForMajor is flaky
>
>
> Key: HBASE-15634
> URL: https://issues.apache.org/jira/browse/HBASE-15634
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.4.0
>Reporter: Ted Yu
>Assignee: Duo Zhang
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: 15634.v1.patch, 15634.v2.patch, 15634.v3.patch,
> HBASE-15634.patch
>
>
> https://builds.apache.org/job/PreCommit-HBASE-Build/1365/artifact/patchprocess/patch-unit-hbase-server.txt
> :
> {code}
> negativeForMajor(org.apache.hadoop.hbase.regionserver.TestDateTieredCompactionPolicy)
> Time elapsed: 0.48 sec <<< FAILURE!
> java.lang.AssertionError: expected: but was:
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:834)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.hadoop.hbase.regionserver.TestDateTieredCompactionPolicy.compactEquals(TestDateTieredCompactionPolicy.java:94)
> at
> org.apache.hadoop.hbase.regionserver.TestDateTieredCompactionPolicy.negativeForMajor(TestDateTieredCompactionPolicy.java:301)
> {code}
> Similar test failure occurred in master JDK 8 build as well
> (https://builds.apache.org/job/HBase-TRUNK_matrix/839/jdk=latest1.8,label=yahoo-not-h2/testReport/org.apache.hadoop.hbase.regionserver/TestDateTieredCompactionPolicy/negativeForMajor/).
> Since TestDateTieredCompactionPolicy is a small test, its failure prevented
> large tests from running.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15645) hbase.rpc.timeout is not used in operations of HTable
[
https://issues.apache.org/jira/browse/HBASE-15645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258731#comment-15258731
]
Hudson commented on HBASE-15645:
SUCCESS: Integrated in HBase-1.3-IT #635 (See
[https://builds.apache.org/job/HBase-1.3-IT/635/])
HBASE-15645 hbase.rpc.timeout is not used in operations of HTable (stack: rev
e917a74f077e1822ae9dc351c8e4f7e5d7cf1b30)
* hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
* hbase-common/src/main/resources/hbase-default.xml
* hbase-server/src/main/java/org/apache/hadoop/hbase/client/HTableWrapper.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCaller.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java
*
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerFactory.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/Table.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTablePool.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
> hbase.rpc.timeout is not used in operations of HTable
> -
>
> Key: HBASE-15645
> URL: https://issues.apache.org/jira/browse/HBASE-15645
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0, 1.3.0, 1.2.1, 1.0.3, 1.1.4
>Reporter: Phil Yang
>Assignee: Phil Yang
>Priority: Critical
> Fix For: 2.0.0, 1.3.0, 1.0.4, 1.4.0, 1.1.5, 1.2.2
>
> Attachments: HBASE-15645-branch-1-v1.patch,
> HBASE-15645-branch-1.0-v1.patch, HBASE-15645-branch-1.1-v1.patch,
> HBASE-15645-branch-1.2-v1.patch, HBASE-15645-v1.patch, HBASE-15645-v2.patch,
> HBASE-15645-v3.patch, HBASE-15645-v4.patch
>
>
> While fixing HBASE-15593, I find that we use operationTimeout as the timeout
> of Get operation rpc call (hbase.client.scanner.timeout.period is used in
> scan rpc), not the hbase.rpc.timeout.
> This can be verified by add one line in TestHCM.setUpBeforeClass():
> {code}
> TEST_UTIL.getConfiguration().setLong(HConstants.HBASE_RPC_TIMEOUT_KEY, 3000);
> {code}
> and then run testOperationTimeout(), the test passes but it should have
> failed because we should get rpc timeout first after 3 seconds then client
> should retry and timeout again and again until operationTimeout or max
> retries reached.
> If I port this test to 0.98, it will fail as expected.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15704) Refactoring: Move HFileArchiver from backup to its own package
[ https://issues.apache.org/jira/browse/HBASE-15704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258729#comment-15258729 ] Enis Soztutar commented on HBASE-15704: --- bq. I don't know if we need to keep it around. Thanks [~jesse_yates]. Let's remove the code in master then. It is not good to keep un-used code around. Vladimir, can we remove the backup.example classes in master, and do the rename only in HFileArchiver. In branch-1, we can keep the example classes and do the HFileArchiver rename only. > Refactoring: Move HFileArchiver from backup to its own package > -- > > Key: HBASE-15704 > URL: https://issues.apache.org/jira/browse/HBASE-15704 > Project: HBase > Issue Type: Task >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Fix For: 2.0.0 > > Attachments: HBASE-15704-v2.patch > > > This class is in backup package (as well as backup/examples classes) but is > not backup - related. Move examples classes to hbase-examples package. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
