[jira] [Commented] (HBASE-16176) Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of contention when many threads reading a StoreFile
[
https://issues.apache.org/jira/browse/HBASE-16176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368941#comment-15368941
]
stack commented on HBASE-16176:
---
Or, sorry, its a loop job and its been through 6 or so loops so far passing.
> Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of
> contention when many threads reading a StoreFile
> --
>
> Key: HBASE-16176
> URL: https://issues.apache.org/jira/browse/HBASE-16176
> Project: HBase
> Issue Type: Sub-task
> Components: Performance
>Affects Versions: 1.3.0, 0.98.20
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.21
>
> Attachments: HBASE-16176.branch-1.3.001.patch,
> HBASE-16176.branch-1.3.002.patch, HBASE-16176.branch-1.3.002.patch,
> HBASE-16176.branch-1.3.003.patch, HBASE-16176.master.001.patch
>
>
> Debugging the parent issue, came up with some improvements on old HBASE-15650
> "Remove TimeRangeTracker as point of contention when many threads reading a
> StoreFile". Lets get them in. Here are the changes:
> {code}
> 6 Change HFile Writer constructor so we pass in the TimeRangeTracker,
> if one,
> 7 on construction rather than set later (the flag and reference were
> not
> 8 volatile so could have made for issues in concurrent case) 2. Make
> sure the
> 9 construction of a TimeRange from a TimeRangeTracer on open of an
> HFile Reader
> 10 never makes a bad minimum value, one that would preclude us reading
> any
> 11 values from a file (add a log and set min to 0)
> 12 M hbase-common/src/main/java/org/apache/hadoop/hbase/io/TimeRange.java
> 13 Call through to next constructor (if minStamp was 0, we'd skip
> setting allTime=true)
> 14 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
> 15 Add constructor override that takes a TimeRangeTracker (set when
> flushing but
> 16 not when compacting)
> 17 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
> 18 Add override creating an HFile in tmp that takes a TimeRangeTracker
> 19 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
> 20 Add override for HFile Writer that takes a TimeRangeTracker
> 21 Take it on construction instead of having it passed by a setter
> later (flags
> 22 and reference set by the setter were not volatile... could have been
> prob
> 23 in concurrent case)
> 24 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TimeRangeTracker.java
> 25 Log WARN if bad initial TimeRange value (and then 'fix' it)
> 26 M
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTimeRangeTracker.java
> 27 A few tests to prove serialization works as expected and that we'll
> get a bad min if
> 28 not constructed properly.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16176) Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of contention when many threads reading a StoreFile
[
https://issues.apache.org/jira/browse/HBASE-16176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368938#comment-15368938
]
stack commented on HBASE-16176:
---
Ran a more vicious, bigger ITBLL job today and it passed on 1.3 just now.
> Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of
> contention when many threads reading a StoreFile
> --
>
> Key: HBASE-16176
> URL: https://issues.apache.org/jira/browse/HBASE-16176
> Project: HBase
> Issue Type: Sub-task
> Components: Performance
>Affects Versions: 1.3.0, 0.98.20
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.21
>
> Attachments: HBASE-16176.branch-1.3.001.patch,
> HBASE-16176.branch-1.3.002.patch, HBASE-16176.branch-1.3.002.patch,
> HBASE-16176.branch-1.3.003.patch, HBASE-16176.master.001.patch
>
>
> Debugging the parent issue, came up with some improvements on old HBASE-15650
> "Remove TimeRangeTracker as point of contention when many threads reading a
> StoreFile". Lets get them in. Here are the changes:
> {code}
> 6 Change HFile Writer constructor so we pass in the TimeRangeTracker,
> if one,
> 7 on construction rather than set later (the flag and reference were
> not
> 8 volatile so could have made for issues in concurrent case) 2. Make
> sure the
> 9 construction of a TimeRange from a TimeRangeTracer on open of an
> HFile Reader
> 10 never makes a bad minimum value, one that would preclude us reading
> any
> 11 values from a file (add a log and set min to 0)
> 12 M hbase-common/src/main/java/org/apache/hadoop/hbase/io/TimeRange.java
> 13 Call through to next constructor (if minStamp was 0, we'd skip
> setting allTime=true)
> 14 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
> 15 Add constructor override that takes a TimeRangeTracker (set when
> flushing but
> 16 not when compacting)
> 17 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
> 18 Add override creating an HFile in tmp that takes a TimeRangeTracker
> 19 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
> 20 Add override for HFile Writer that takes a TimeRangeTracker
> 21 Take it on construction instead of having it passed by a setter
> later (flags
> 22 and reference set by the setter were not volatile... could have been
> prob
> 23 in concurrent case)
> 24 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TimeRangeTracker.java
> 25 Log WARN if bad initial TimeRange value (and then 'fix' it)
> 26 M
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTimeRangeTracker.java
> 27 A few tests to prove serialization works as expected and that we'll
> get a bad min if
> 28 not constructed properly.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16161) Remove a few unnecessary (uncontended) synchronizes
[ https://issues.apache.org/jira/browse/HBASE-16161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368936#comment-15368936 ] stack commented on HBASE-16161: --- Over in review @heng chen suggestion that I add a test to prove concurrency on next in RegionScanner is good absent synchronize seems good to me. Let me do it. > Remove a few unnecessary (uncontended) synchronizes > --- > > Key: HBASE-16161 > URL: https://issues.apache.org/jira/browse/HBASE-16161 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack >Priority: Minor > Attachments: HBASE-16161.branch-1.001.patch > > > This is followon from HBASE-15716. We have a few odd looking synchronizes. A > few are probably elided after analysis concludes single thread consumer but > lets remove to be clear. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16176) Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of contention when many threads reading a StoreFile
[
https://issues.apache.org/jira/browse/HBASE-16176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368915#comment-15368915
]
stack commented on HBASE-16176:
---
Pushed to branch-1.3+
Just submitted new patch for master because it is a bit different just to make
sure hadoop qa comes back good.
> Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of
> contention when many threads reading a StoreFile
> --
>
> Key: HBASE-16176
> URL: https://issues.apache.org/jira/browse/HBASE-16176
> Project: HBase
> Issue Type: Sub-task
> Components: Performance
>Affects Versions: 1.3.0, 0.98.20
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.21
>
> Attachments: HBASE-16176.branch-1.3.001.patch,
> HBASE-16176.branch-1.3.002.patch, HBASE-16176.branch-1.3.002.patch,
> HBASE-16176.branch-1.3.003.patch, HBASE-16176.master.001.patch
>
>
> Debugging the parent issue, came up with some improvements on old HBASE-15650
> "Remove TimeRangeTracker as point of contention when many threads reading a
> StoreFile". Lets get them in. Here are the changes:
> {code}
> 6 Change HFile Writer constructor so we pass in the TimeRangeTracker,
> if one,
> 7 on construction rather than set later (the flag and reference were
> not
> 8 volatile so could have made for issues in concurrent case) 2. Make
> sure the
> 9 construction of a TimeRange from a TimeRangeTracer on open of an
> HFile Reader
> 10 never makes a bad minimum value, one that would preclude us reading
> any
> 11 values from a file (add a log and set min to 0)
> 12 M hbase-common/src/main/java/org/apache/hadoop/hbase/io/TimeRange.java
> 13 Call through to next constructor (if minStamp was 0, we'd skip
> setting allTime=true)
> 14 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
> 15 Add constructor override that takes a TimeRangeTracker (set when
> flushing but
> 16 not when compacting)
> 17 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
> 18 Add override creating an HFile in tmp that takes a TimeRangeTracker
> 19 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
> 20 Add override for HFile Writer that takes a TimeRangeTracker
> 21 Take it on construction instead of having it passed by a setter
> later (flags
> 22 and reference set by the setter were not volatile... could have been
> prob
> 23 in concurrent case)
> 24 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TimeRangeTracker.java
> 25 Log WARN if bad initial TimeRange value (and then 'fix' it)
> 26 M
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTimeRangeTracker.java
> 27 A few tests to prove serialization works as expected and that we'll
> get a bad min if
> 28 not constructed properly.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16176) Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of contention when many threads reading a StoreFile
[
https://issues.apache.org/jira/browse/HBASE-16176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-16176:
--
Attachment: HBASE-16176.master.001.patch
> Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of
> contention when many threads reading a StoreFile
> --
>
> Key: HBASE-16176
> URL: https://issues.apache.org/jira/browse/HBASE-16176
> Project: HBase
> Issue Type: Sub-task
> Components: Performance
>Affects Versions: 1.3.0, 0.98.20
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.21
>
> Attachments: HBASE-16176.branch-1.3.001.patch,
> HBASE-16176.branch-1.3.002.patch, HBASE-16176.branch-1.3.002.patch,
> HBASE-16176.branch-1.3.003.patch, HBASE-16176.master.001.patch
>
>
> Debugging the parent issue, came up with some improvements on old HBASE-15650
> "Remove TimeRangeTracker as point of contention when many threads reading a
> StoreFile". Lets get them in. Here are the changes:
> {code}
> 6 Change HFile Writer constructor so we pass in the TimeRangeTracker,
> if one,
> 7 on construction rather than set later (the flag and reference were
> not
> 8 volatile so could have made for issues in concurrent case) 2. Make
> sure the
> 9 construction of a TimeRange from a TimeRangeTracer on open of an
> HFile Reader
> 10 never makes a bad minimum value, one that would preclude us reading
> any
> 11 values from a file (add a log and set min to 0)
> 12 M hbase-common/src/main/java/org/apache/hadoop/hbase/io/TimeRange.java
> 13 Call through to next constructor (if minStamp was 0, we'd skip
> setting allTime=true)
> 14 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
> 15 Add constructor override that takes a TimeRangeTracker (set when
> flushing but
> 16 not when compacting)
> 17 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
> 18 Add override creating an HFile in tmp that takes a TimeRangeTracker
> 19 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
> 20 Add override for HFile Writer that takes a TimeRangeTracker
> 21 Take it on construction instead of having it passed by a setter
> later (flags
> 22 and reference set by the setter were not volatile... could have been
> prob
> 23 in concurrent case)
> 24 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TimeRangeTracker.java
> 25 Log WARN if bad initial TimeRange value (and then 'fix' it)
> 26 M
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTimeRangeTracker.java
> 27 A few tests to prove serialization works as expected and that we'll
> get a bad min if
> 28 not constructed properly.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16017) HBase TableOutputFormat has connection leak in getRecordWriter
[ https://issues.apache.org/jira/browse/HBASE-16017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368898#comment-15368898 ] Hudson commented on HBASE-16017: SUCCESS: Integrated in HBase-1.3 #775 (See [https://builds.apache.org/job/HBase-1.3/775/]) HBASE-16017 HBase TableOutputFormat has connection leak in (tedyu: rev e7513e447e2b8f299a39f06d35d872e0c8de3e7e) * hbase-server/src/main/java/org/apache/hadoop/hbase/mapred/TableOutputFormat.java > HBase TableOutputFormat has connection leak in getRecordWriter > -- > > Key: HBASE-16017 > URL: https://issues.apache.org/jira/browse/HBASE-16017 > Project: HBase > Issue Type: Bug > Components: mapreduce >Reporter: Zhan Zhang >Assignee: Zhan Zhang > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-16017-1.patch > > > Currently getRecordWriter will not release the connection until jvm > terminate, which is not a right assumption given that the function may be > invoked many times in the same jvm lifecycle. Inside of mapreduce, the issue > has already fixed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15650) Remove TimeRangeTracker as point of contention when many threads reading a StoreFile
[ https://issues.apache.org/jira/browse/HBASE-15650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368900#comment-15368900 ] Hudson commented on HBASE-15650: SUCCESS: Integrated in HBase-1.3 #775 (See [https://builds.apache.org/job/HBase-1.3/775/]) HBASE-16176 Bug fixes/improvements on HBASE-15650 Remove (stack: rev 2e2a3fda40df8b24928f57a6c8ea089b988ed0ce) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java * hbase-common/src/main/java/org/apache/hadoop/hbase/io/TimeRange.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestQueryMatcher.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StripeStoreFlusher.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTimeRangeTracker.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultStoreFlusher.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/Query.java * hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValueUtil.java * hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ExplicitColumnTracker.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TimeRangeTracker.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java > Remove TimeRangeTracker as point of contention when many threads reading a > StoreFile > > > Key: HBASE-15650 > URL: https://issues.apache.org/jira/browse/HBASE-15650 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack > Fix For: 2.0.0, 1.3.0, 0.98.19, 1.4.0 > > Attachments: 15650.branch-1.2.patch, 15650.branch-1.patch, > 15650.branch-1.patch, 15650.patch, 15650.patch, 15650v2.branch-1.patch, > 15650v2.patch, 15650v3.patch, 15650v4.patch, 15650v5.patch, 15650v6.patch, > Point-of-contention-on-random-read.png > > > HBASE-12148 is about "Remove TimeRangeTracker as point of contention when > many threads writing a Store". It is also a point of contention when reading. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15353) Add metric for number of CallQueueTooBigException's
[ https://issues.apache.org/jira/browse/HBASE-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368897#comment-15368897 ] Hudson commented on HBASE-15353: SUCCESS: Integrated in HBase-1.3 #775 (See [https://builds.apache.org/job/HBase-1.3/775/]) HBASE-15353 Add metric for number of CallQueueTooBigException's (garyh: rev 687c70c96339fccace6db1e0f327d11fe9f0c2a4) * hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerSource.java * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerSourceImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServer.java > Add metric for number of CallQueueTooBigException's > --- > > Key: HBASE-15353 > URL: https://issues.apache.org/jira/browse/HBASE-15353 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC, metrics >Reporter: Elliott Clark >Assignee: Jingcheng Du >Priority: Minor > Labels: beginner, easy > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-15353.patch > > > This exception is being thrown more. We should add a metric for this one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16176) Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of contention when many threads reading a StoreFile
[
https://issues.apache.org/jira/browse/HBASE-16176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368899#comment-15368899
]
Hudson commented on HBASE-16176:
SUCCESS: Integrated in HBase-1.3 #775 (See
[https://builds.apache.org/job/HBase-1.3/775/])
HBASE-16176 Bug fixes/improvements on HBASE-15650 Remove (stack: rev
2e2a3fda40df8b24928f57a6c8ea089b988ed0ce)
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/io/TimeRange.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestQueryMatcher.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StripeStoreFlusher.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTimeRangeTracker.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultStoreFlusher.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/Query.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValueUtil.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ExplicitColumnTracker.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TimeRangeTracker.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
> Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of
> contention when many threads reading a StoreFile
> --
>
> Key: HBASE-16176
> URL: https://issues.apache.org/jira/browse/HBASE-16176
> Project: HBase
> Issue Type: Sub-task
> Components: Performance
>Affects Versions: 1.3.0, 0.98.20
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.21
>
> Attachments: HBASE-16176.branch-1.3.001.patch,
> HBASE-16176.branch-1.3.002.patch, HBASE-16176.branch-1.3.002.patch,
> HBASE-16176.branch-1.3.003.patch
>
>
> Debugging the parent issue, came up with some improvements on old HBASE-15650
> "Remove TimeRangeTracker as point of contention when many threads reading a
> StoreFile". Lets get them in. Here are the changes:
> {code}
> 6 Change HFile Writer constructor so we pass in the TimeRangeTracker,
> if one,
> 7 on construction rather than set later (the flag and reference were
> not
> 8 volatile so could have made for issues in concurrent case) 2. Make
> sure the
> 9 construction of a TimeRange from a TimeRangeTracer on open of an
> HFile Reader
> 10 never makes a bad minimum value, one that would preclude us reading
> any
> 11 values from a file (add a log and set min to 0)
> 12 M hbase-common/src/main/java/org/apache/hadoop/hbase/io/TimeRange.java
> 13 Call through to next constructor (if minStamp was 0, we'd skip
> setting allTime=true)
> 14 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
> 15 Add constructor override that takes a TimeRangeTracker (set when
> flushing but
> 16 not when compacting)
> 17 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
> 18 Add override creating an HFile in tmp that takes a TimeRangeTracker
> 19 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
> 20 Add override for HFile Writer that takes a TimeRangeTracker
> 21 Take it on construction instead of having it passed by a setter
> later (flags
> 22 and reference set by the setter were not volatile... could have been
> prob
> 23 in concurrent case)
> 24 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TimeRangeTracker.java
> 25 Log WARN if bad initial TimeRange value (and then 'fix' it)
> 26 M
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTimeRangeTracker.java
> 27 A few tests to prove serialization works as expected and that we'll
> get a bad min if
> 28 not constructed properly.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16176) Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of contention when many threads reading a StoreFile
[
https://issues.apache.org/jira/browse/HBASE-16176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368890#comment-15368890
]
Hudson commented on HBASE-16176:
SUCCESS: Integrated in HBase-1.4 #277 (See
[https://builds.apache.org/job/HBase-1.4/277/])
HBASE-16176 Bug fixes/improvements on HBASE-15650 Remove (stack: rev
9563ab4ca10b600d4072e87896f09f8cf84d1a2a)
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultStoreFlusher.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/Query.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StripeStoreFlusher.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValueUtil.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TimeRangeTracker.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestQueryMatcher.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ExplicitColumnTracker.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/io/TimeRange.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTimeRangeTracker.java
> Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of
> contention when many threads reading a StoreFile
> --
>
> Key: HBASE-16176
> URL: https://issues.apache.org/jira/browse/HBASE-16176
> Project: HBase
> Issue Type: Sub-task
> Components: Performance
>Affects Versions: 1.3.0, 0.98.20
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.21
>
> Attachments: HBASE-16176.branch-1.3.001.patch,
> HBASE-16176.branch-1.3.002.patch, HBASE-16176.branch-1.3.002.patch,
> HBASE-16176.branch-1.3.003.patch
>
>
> Debugging the parent issue, came up with some improvements on old HBASE-15650
> "Remove TimeRangeTracker as point of contention when many threads reading a
> StoreFile". Lets get them in. Here are the changes:
> {code}
> 6 Change HFile Writer constructor so we pass in the TimeRangeTracker,
> if one,
> 7 on construction rather than set later (the flag and reference were
> not
> 8 volatile so could have made for issues in concurrent case) 2. Make
> sure the
> 9 construction of a TimeRange from a TimeRangeTracer on open of an
> HFile Reader
> 10 never makes a bad minimum value, one that would preclude us reading
> any
> 11 values from a file (add a log and set min to 0)
> 12 M hbase-common/src/main/java/org/apache/hadoop/hbase/io/TimeRange.java
> 13 Call through to next constructor (if minStamp was 0, we'd skip
> setting allTime=true)
> 14 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
> 15 Add constructor override that takes a TimeRangeTracker (set when
> flushing but
> 16 not when compacting)
> 17 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
> 18 Add override creating an HFile in tmp that takes a TimeRangeTracker
> 19 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
> 20 Add override for HFile Writer that takes a TimeRangeTracker
> 21 Take it on construction instead of having it passed by a setter
> later (flags
> 22 and reference set by the setter were not volatile... could have been
> prob
> 23 in concurrent case)
> 24 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TimeRangeTracker.java
> 25 Log WARN if bad initial TimeRange value (and then 'fix' it)
> 26 M
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTimeRangeTracker.java
> 27 A few tests to prove serialization works as expected and that we'll
> get a bad min if
> 28 not constructed properly.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15650) Remove TimeRangeTracker as point of contention when many threads reading a StoreFile
[ https://issues.apache.org/jira/browse/HBASE-15650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368891#comment-15368891 ] Hudson commented on HBASE-15650: SUCCESS: Integrated in HBase-1.4 #277 (See [https://builds.apache.org/job/HBase-1.4/277/]) HBASE-16176 Bug fixes/improvements on HBASE-15650 Remove (stack: rev 9563ab4ca10b600d4072e87896f09f8cf84d1a2a) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultStoreFlusher.java * hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/Query.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StripeStoreFlusher.java * hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValueUtil.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TimeRangeTracker.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestQueryMatcher.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ExplicitColumnTracker.java * hbase-common/src/main/java/org/apache/hadoop/hbase/io/TimeRange.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTimeRangeTracker.java > Remove TimeRangeTracker as point of contention when many threads reading a > StoreFile > > > Key: HBASE-15650 > URL: https://issues.apache.org/jira/browse/HBASE-15650 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack > Fix For: 2.0.0, 1.3.0, 0.98.19, 1.4.0 > > Attachments: 15650.branch-1.2.patch, 15650.branch-1.patch, > 15650.branch-1.patch, 15650.patch, 15650.patch, 15650v2.branch-1.patch, > 15650v2.patch, 15650v3.patch, 15650v4.patch, 15650v5.patch, 15650v6.patch, > Point-of-contention-on-random-read.png > > > HBASE-12148 is about "Remove TimeRangeTracker as point of contention when > many threads writing a Store". It is also a point of contention when reading. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-3727) MultiHFileOutputFormat
[
https://issues.apache.org/jira/browse/HBASE-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368887#comment-15368887
]
Hadoop QA commented on HBASE-3727:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
38s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
59s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 28s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 40s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 40s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 44s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 29s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 115m 17s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 159m 16s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.master.procedure.TestMasterFailoverWithProcedures |
| | hadoop.hbase.replication.TestMasterReplication |
| | hadoop.hbase.mapreduce.TestLoadIncrementalHFilesUseSecurityEndPoint |
| | hadoop.hbase.regionserver.TestRegionServerMetrics |
| | hadoop.hbase.replication.TestReplicationSmallTests |
| Timed out junit tests |
org.apache.hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFiles |
| | org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles |
| | org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan2 |
| | org.apache.hadoop.hbase.filter.TestFuzzyRowFilterEndToEnd |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12816941/HBASE-3727-V4.patch |
| JIRA Issue | HBASE-3727 |
| Optional Tests | asflicense javac javadoc unit findbugs
[jira] [Commented] (HBASE-16148) Hybrid Logical Clocks(placeholder for running tests)
[
https://issues.apache.org/jira/browse/HBASE-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368885#comment-15368885
]
Hadoop QA commented on HBASE-16148:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 25s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
28s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m 42s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 46s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 7m
5s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
33s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 2m
29s {color} | {color:green} the patch passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s
{color} | {color:blue} Skipped branch modules with no Java source: . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 53s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m 19s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 11s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m 18s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 4m 18s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m 18s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 47s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 3m 47s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m 47s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 7m
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
4s {color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
2s {color} | {color:green} There were no new shelldocs issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 38s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 2m
13s {color} | {color:green} the patch passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s
{color} | {color:blue} Skipped patch modules with no Java source: . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
20s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green}
[jira] [Commented] (HBASE-15650) Remove TimeRangeTracker as point of contention when many threads reading a StoreFile
[ https://issues.apache.org/jira/browse/HBASE-15650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368862#comment-15368862 ] Mikhail Antonov commented on HBASE-15650: - Any reason to not commit on branch-1 and master? want to get it to work with that fake cell interface? > Remove TimeRangeTracker as point of contention when many threads reading a > StoreFile > > > Key: HBASE-15650 > URL: https://issues.apache.org/jira/browse/HBASE-15650 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack > Fix For: 2.0.0, 1.3.0, 0.98.19, 1.4.0 > > Attachments: 15650.branch-1.2.patch, 15650.branch-1.patch, > 15650.branch-1.patch, 15650.patch, 15650.patch, 15650v2.branch-1.patch, > 15650v2.patch, 15650v3.patch, 15650v4.patch, 15650v5.patch, 15650v6.patch, > Point-of-contention-on-random-read.png > > > HBASE-12148 is about "Remove TimeRangeTracker as point of contention when > many threads writing a Store". It is also a point of contention when reading. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16201) NPE in RpcServer causing intermittent UT failure of TestMasterReplication#testHFileCyclicReplication
[
https://issues.apache.org/jira/browse/HBASE-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368823#comment-15368823
]
Hudson commented on HBASE-16201:
SUCCESS: Integrated in HBase-1.1-JDK7 #1743 (See
[https://builds.apache.org/job/HBase-1.1-JDK7/1743/])
HBASE-16201 fix a NPE issue in RpcServer (liyu: rev
73189eb801f1c49e738e8a79838b1cd17b1fcff5)
* hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
> NPE in RpcServer causing intermittent UT failure of
> TestMasterReplication#testHFileCyclicReplication
>
>
> Key: HBASE-16201
> URL: https://issues.apache.org/jira/browse/HBASE-16201
> Project: HBase
> Issue Type: Bug
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0, 1.3.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-16201.patch
>
>
> Every several rounds of {{TestMasterReplication#testHFileCyclicReplication}},
> we could observe below NPE in UT log:
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2257)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:118)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:189)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:169)
> {noformat}
> And related codes at RpcServer line 2257 are:
> {code}
> if (e instanceof ServiceException) {
> e = e.getCause();
> }
> // increment the number of requests that were exceptions.
> metrics.exception(e);
> if (e instanceof LinkageError) throw new DoNotRetryIOException(e);
> if (e instanceof IOException) throw (IOException)e;
> {code}
> And after some debugging, we could find several places that constructing
> ServiceException with no cause, such as in
> {{RsRpcServices#replicateWALEntry}}:
> {code}
> if (regionServer.replicationSinkHandler != null) {
> ...
> } else {
> throw new ServiceException("Replication services are not initialized
> yet");
> }
> {code}
> So we should firstly check and only reset {{e=e.getCause()}} when the cause
> is not null
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16201) NPE in RpcServer causing intermittent UT failure of TestMasterReplication#testHFileCyclicReplication
[
https://issues.apache.org/jira/browse/HBASE-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368788#comment-15368788
]
Hudson commented on HBASE-16201:
SUCCESS: Integrated in HBase-1.1-JDK8 #1830 (See
[https://builds.apache.org/job/HBase-1.1-JDK8/1830/])
HBASE-16201 fix a NPE issue in RpcServer (liyu: rev
73189eb801f1c49e738e8a79838b1cd17b1fcff5)
* hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
> NPE in RpcServer causing intermittent UT failure of
> TestMasterReplication#testHFileCyclicReplication
>
>
> Key: HBASE-16201
> URL: https://issues.apache.org/jira/browse/HBASE-16201
> Project: HBase
> Issue Type: Bug
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0, 1.3.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-16201.patch
>
>
> Every several rounds of {{TestMasterReplication#testHFileCyclicReplication}},
> we could observe below NPE in UT log:
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2257)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:118)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:189)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:169)
> {noformat}
> And related codes at RpcServer line 2257 are:
> {code}
> if (e instanceof ServiceException) {
> e = e.getCause();
> }
> // increment the number of requests that were exceptions.
> metrics.exception(e);
> if (e instanceof LinkageError) throw new DoNotRetryIOException(e);
> if (e instanceof IOException) throw (IOException)e;
> {code}
> And after some debugging, we could find several places that constructing
> ServiceException with no cause, such as in
> {{RsRpcServices#replicateWALEntry}}:
> {code}
> if (regionServer.replicationSinkHandler != null) {
> ...
> } else {
> throw new ServiceException("Replication services are not initialized
> yet");
> }
> {code}
> So we should firstly check and only reset {{e=e.getCause()}} when the cause
> is not null
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16096) Replication keeps accumulating znodes
[
https://issues.apache.org/jira/browse/HBASE-16096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368785#comment-15368785
]
Hadoop QA commented on HBASE-16096:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 28s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
32s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 13s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
53s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s
{color} | {color:green} master passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} master passed with JDK v1.7.0_80 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
5s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 5s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 5s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 52s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
28m 48s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
31s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 7s
{color} | {color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s
{color} | {color:green} the patch passed with JDK v1.7.0_80 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 0s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 130m 12s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
44s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 184m 52s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.replication.TestReplicationDisableInactivePeer |
| |
hadoop.hbase.replication.multiwal.TestReplicationKillMasterRSCompressedWithMultipleWAL
|
| | hadoop.hbase.snapshot.TestMobFlushSnapshotFromClient |
| | hadoop.hbase.snapshot.TestSecureExportSnapshot |
| | hadoop.hbase.replication.TestReplicationChangingPeerRegionservers |
| |
[jira] [Commented] (HBASE-16201) NPE in RpcServer causing intermittent UT failure of TestMasterReplication#testHFileCyclicReplication
[
https://issues.apache.org/jira/browse/HBASE-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368738#comment-15368738
]
Hudson commented on HBASE-16201:
FAILURE: Integrated in HBase-Trunk_matrix #1193 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/1193/])
HBASE-16201 NPE in RpcServer causing intermittent UT failure of (liyu: rev
3c39cbd92c3f309c98ca01bfb70ca89bc046a228)
* hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
> NPE in RpcServer causing intermittent UT failure of
> TestMasterReplication#testHFileCyclicReplication
>
>
> Key: HBASE-16201
> URL: https://issues.apache.org/jira/browse/HBASE-16201
> Project: HBase
> Issue Type: Bug
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0, 1.3.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-16201.patch
>
>
> Every several rounds of {{TestMasterReplication#testHFileCyclicReplication}},
> we could observe below NPE in UT log:
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2257)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:118)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:189)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:169)
> {noformat}
> And related codes at RpcServer line 2257 are:
> {code}
> if (e instanceof ServiceException) {
> e = e.getCause();
> }
> // increment the number of requests that were exceptions.
> metrics.exception(e);
> if (e instanceof LinkageError) throw new DoNotRetryIOException(e);
> if (e instanceof IOException) throw (IOException)e;
> {code}
> And after some debugging, we could find several places that constructing
> ServiceException with no cause, such as in
> {{RsRpcServices#replicateWALEntry}}:
> {code}
> if (regionServer.replicationSinkHandler != null) {
> ...
> } else {
> throw new ServiceException("Replication services are not initialized
> yet");
> }
> {code}
> So we should firstly check and only reset {{e=e.getCause()}} when the cause
> is not null
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-3727) MultiHFileOutputFormat
[ https://issues.apache.org/jira/browse/HBASE-3727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yi liang updated HBASE-3727: Attachment: HBASE-3727-V4.patch > MultiHFileOutputFormat > -- > > Key: HBASE-3727 > URL: https://issues.apache.org/jira/browse/HBASE-3727 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: Andrew Purtell >Assignee: yi liang >Priority: Minor > Attachments: HBASE-3727-V3.patch, HBASE-3727-V4.patch, MH2.patch, > MultiHFileOutputFormat.java, MultiHFileOutputFormat.java, > MultiHFileOutputFormat.java, TestMultiHFileOutputFormat.java > > > Like MultiTableOutputFormat, but outputting HFiles. Key is tablename as an > IBW. Creates sub-writers (code cut and pasted from HFileOutputFormat) on > demand that produce HFiles in per-table subdirectories of the configured > output path. Does not currently support partitioning for existing tables / > incremental update. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16201) NPE in RpcServer causing intermittent UT failure of TestMasterReplication#testHFileCyclicReplication
[
https://issues.apache.org/jira/browse/HBASE-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368708#comment-15368708
]
Hudson commented on HBASE-16201:
SUCCESS: Integrated in HBase-1.3 #774 (See
[https://builds.apache.org/job/HBase-1.3/774/])
HBASE-16201 NPE in RpcServer causing intermittent UT failure of (liyu: rev
43626fc06e895433cce304b5cee97999a106e0ac)
* hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
> NPE in RpcServer causing intermittent UT failure of
> TestMasterReplication#testHFileCyclicReplication
>
>
> Key: HBASE-16201
> URL: https://issues.apache.org/jira/browse/HBASE-16201
> Project: HBase
> Issue Type: Bug
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0, 1.3.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-16201.patch
>
>
> Every several rounds of {{TestMasterReplication#testHFileCyclicReplication}},
> we could observe below NPE in UT log:
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2257)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:118)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:189)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:169)
> {noformat}
> And related codes at RpcServer line 2257 are:
> {code}
> if (e instanceof ServiceException) {
> e = e.getCause();
> }
> // increment the number of requests that were exceptions.
> metrics.exception(e);
> if (e instanceof LinkageError) throw new DoNotRetryIOException(e);
> if (e instanceof IOException) throw (IOException)e;
> {code}
> And after some debugging, we could find several places that constructing
> ServiceException with no cause, such as in
> {{RsRpcServices#replicateWALEntry}}:
> {code}
> if (regionServer.replicationSinkHandler != null) {
> ...
> } else {
> throw new ServiceException("Replication services are not initialized
> yet");
> }
> {code}
> So we should firstly check and only reset {{e=e.getCause()}} when the cause
> is not null
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15775) Canary launches two AuthUtil Chores
[ https://issues.apache.org/jira/browse/HBASE-15775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368707#comment-15368707 ] Hudson commented on HBASE-15775: SUCCESS: Integrated in HBase-1.3 #774 (See [https://builds.apache.org/job/HBase-1.3/774/]) HBASE-15775 Canary launches two AuthUtil Chores (stack: rev 966e5f2cbb41718faa53cd7a843e6d562e866b2c) * hbase-server/src/main/java/org/apache/hadoop/hbase/tool/Canary.java > Canary launches two AuthUtil Chores > --- > > Key: HBASE-15775 > URL: https://issues.apache.org/jira/browse/HBASE-15775 > Project: HBase > Issue Type: Bug > Components: canary >Affects Versions: 0.98.13, 0.98.14, 1.2.0, 1.3.0, 0.98.15, 1.2.1, 0.98.16, > 0.98.17, 0.98.18, 0.98.16.1, 0.98.19, 1.4.0 >Reporter: Sean Busbey >Assignee: Vishal Khandelwal >Priority: Minor > Labels: beginner > Fix For: 1.3.0, 1.2.3 > > Attachments: HBASE-15775.0.98.00.patch, HBASE-15775.branch-1.00.patch > > > Looks like a result of an error in backport done in HBASE-13712. We have a > AuthUtil chore both in main() and in run(). > The one in main() should be removed so that the code is consistent with other > branches. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15650) Remove TimeRangeTracker as point of contention when many threads reading a StoreFile
[ https://issues.apache.org/jira/browse/HBASE-15650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368663#comment-15368663 ] Hudson commented on HBASE-15650: SUCCESS: Integrated in HBase-1.3-IT #747 (See [https://builds.apache.org/job/HBase-1.3-IT/747/]) HBASE-16176 Bug fixes/improvements on HBASE-15650 Remove (stack: rev 2e2a3fda40df8b24928f57a6c8ea089b988ed0ce) * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTimeRangeTracker.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TimeRangeTracker.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ExplicitColumnTracker.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StripeStoreFlusher.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/Query.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestQueryMatcher.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultStoreFlusher.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java * hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java * hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValueUtil.java * hbase-common/src/main/java/org/apache/hadoop/hbase/io/TimeRange.java > Remove TimeRangeTracker as point of contention when many threads reading a > StoreFile > > > Key: HBASE-15650 > URL: https://issues.apache.org/jira/browse/HBASE-15650 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 2.0.0 >Reporter: stack >Assignee: stack > Fix For: 2.0.0, 1.3.0, 0.98.19, 1.4.0 > > Attachments: 15650.branch-1.2.patch, 15650.branch-1.patch, > 15650.branch-1.patch, 15650.patch, 15650.patch, 15650v2.branch-1.patch, > 15650v2.patch, 15650v3.patch, 15650v4.patch, 15650v5.patch, 15650v6.patch, > Point-of-contention-on-random-read.png > > > HBASE-12148 is about "Remove TimeRangeTracker as point of contention when > many threads writing a Store". It is also a point of contention when reading. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15353) Add metric for number of CallQueueTooBigException's
[ https://issues.apache.org/jira/browse/HBASE-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368660#comment-15368660 ] Hudson commented on HBASE-15353: SUCCESS: Integrated in HBase-1.3-IT #747 (See [https://builds.apache.org/job/HBase-1.3-IT/747/]) HBASE-15353 Add metric for number of CallQueueTooBigException's (garyh: rev 687c70c96339fccace6db1e0f327d11fe9f0c2a4) * hbase-hadoop2-compat/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerSourceImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServer.java * hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/ipc/MetricsHBaseServerSource.java > Add metric for number of CallQueueTooBigException's > --- > > Key: HBASE-15353 > URL: https://issues.apache.org/jira/browse/HBASE-15353 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC, metrics >Reporter: Elliott Clark >Assignee: Jingcheng Du >Priority: Minor > Labels: beginner, easy > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-15353.patch > > > This exception is being thrown more. We should add a metric for this one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16176) Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of contention when many threads reading a StoreFile
[
https://issues.apache.org/jira/browse/HBASE-16176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368662#comment-15368662
]
Hudson commented on HBASE-16176:
SUCCESS: Integrated in HBase-1.3-IT #747 (See
[https://builds.apache.org/job/HBase-1.3-IT/747/])
HBASE-16176 Bug fixes/improvements on HBASE-15650 Remove (stack: rev
2e2a3fda40df8b24928f57a6c8ea089b988ed0ce)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTimeRangeTracker.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TimeRangeTracker.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ExplicitColumnTracker.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StripeStoreFlusher.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
* hbase-client/src/main/java/org/apache/hadoop/hbase/client/Query.java
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestQueryMatcher.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DefaultStoreFlusher.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValueUtil.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/io/TimeRange.java
> Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of
> contention when many threads reading a StoreFile
> --
>
> Key: HBASE-16176
> URL: https://issues.apache.org/jira/browse/HBASE-16176
> Project: HBase
> Issue Type: Sub-task
> Components: Performance
>Affects Versions: 1.3.0, 0.98.20
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.21
>
> Attachments: HBASE-16176.branch-1.3.001.patch,
> HBASE-16176.branch-1.3.002.patch, HBASE-16176.branch-1.3.002.patch,
> HBASE-16176.branch-1.3.003.patch
>
>
> Debugging the parent issue, came up with some improvements on old HBASE-15650
> "Remove TimeRangeTracker as point of contention when many threads reading a
> StoreFile". Lets get them in. Here are the changes:
> {code}
> 6 Change HFile Writer constructor so we pass in the TimeRangeTracker,
> if one,
> 7 on construction rather than set later (the flag and reference were
> not
> 8 volatile so could have made for issues in concurrent case) 2. Make
> sure the
> 9 construction of a TimeRange from a TimeRangeTracer on open of an
> HFile Reader
> 10 never makes a bad minimum value, one that would preclude us reading
> any
> 11 values from a file (add a log and set min to 0)
> 12 M hbase-common/src/main/java/org/apache/hadoop/hbase/io/TimeRange.java
> 13 Call through to next constructor (if minStamp was 0, we'd skip
> setting allTime=true)
> 14 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
> 15 Add constructor override that takes a TimeRangeTracker (set when
> flushing but
> 16 not when compacting)
> 17 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
> 18 Add override creating an HFile in tmp that takes a TimeRangeTracker
> 19 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
> 20 Add override for HFile Writer that takes a TimeRangeTracker
> 21 Take it on construction instead of having it passed by a setter
> later (flags
> 22 and reference set by the setter were not volatile... could have been
> prob
> 23 in concurrent case)
> 24 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TimeRangeTracker.java
> 25 Log WARN if bad initial TimeRange value (and then 'fix' it)
> 26 M
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTimeRangeTracker.java
> 27 A few tests to prove serialization works as expected and that we'll
> get a bad min if
> 28 not constructed properly.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16017) HBase TableOutputFormat has connection leak in getRecordWriter
[ https://issues.apache.org/jira/browse/HBASE-16017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368661#comment-15368661 ] Hudson commented on HBASE-16017: SUCCESS: Integrated in HBase-1.3-IT #747 (See [https://builds.apache.org/job/HBase-1.3-IT/747/]) HBASE-16017 HBase TableOutputFormat has connection leak in (tedyu: rev e7513e447e2b8f299a39f06d35d872e0c8de3e7e) * hbase-server/src/main/java/org/apache/hadoop/hbase/mapred/TableOutputFormat.java > HBase TableOutputFormat has connection leak in getRecordWriter > -- > > Key: HBASE-16017 > URL: https://issues.apache.org/jira/browse/HBASE-16017 > Project: HBase > Issue Type: Bug > Components: mapreduce >Reporter: Zhan Zhang >Assignee: Zhan Zhang > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-16017-1.patch > > > Currently getRecordWriter will not release the connection until jvm > terminate, which is not a right assumption given that the function may be > invoked many times in the same jvm lifecycle. Inside of mapreduce, the issue > has already fixed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16201) NPE in RpcServer causing intermittent UT failure of TestMasterReplication#testHFileCyclicReplication
[
https://issues.apache.org/jira/browse/HBASE-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368651#comment-15368651
]
Hudson commented on HBASE-16201:
FAILURE: Integrated in HBase-1.4 #276 (See
[https://builds.apache.org/job/HBase-1.4/276/])
HBASE-16201 NPE in RpcServer causing intermittent UT failure of (liyu: rev
09d9dc4b594674d77fb344466daaabc3eab21da0)
* hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
> NPE in RpcServer causing intermittent UT failure of
> TestMasterReplication#testHFileCyclicReplication
>
>
> Key: HBASE-16201
> URL: https://issues.apache.org/jira/browse/HBASE-16201
> Project: HBase
> Issue Type: Bug
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0, 1.3.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-16201.patch
>
>
> Every several rounds of {{TestMasterReplication#testHFileCyclicReplication}},
> we could observe below NPE in UT log:
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2257)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:118)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:189)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:169)
> {noformat}
> And related codes at RpcServer line 2257 are:
> {code}
> if (e instanceof ServiceException) {
> e = e.getCause();
> }
> // increment the number of requests that were exceptions.
> metrics.exception(e);
> if (e instanceof LinkageError) throw new DoNotRetryIOException(e);
> if (e instanceof IOException) throw (IOException)e;
> {code}
> And after some debugging, we could find several places that constructing
> ServiceException with no cause, such as in
> {{RsRpcServices#replicateWALEntry}}:
> {code}
> if (regionServer.replicationSinkHandler != null) {
> ...
> } else {
> throw new ServiceException("Replication services are not initialized
> yet");
> }
> {code}
> So we should firstly check and only reset {{e=e.getCause()}} when the cause
> is not null
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15935) Have a separate Walker task running concurrently with Generator
[ https://issues.apache.org/jira/browse/HBASE-15935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368650#comment-15368650 ] Hudson commented on HBASE-15935: FAILURE: Integrated in HBase-1.4 #276 (See [https://builds.apache.org/job/HBase-1.4/276/]) HBASE-15935 Set up a concurrent walker that walks flushed circular (eclark: rev 0d8e7b2f631b5b5bd9d0c1a4d202b6178bfd1b72) * hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestBigLinkedList.java * hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestBigLinkedListWithVisibility.java * hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestReplication.java > Have a separate Walker task running concurrently with Generator > -- > > Key: HBASE-15935 > URL: https://issues.apache.org/jira/browse/HBASE-15935 > Project: HBase > Issue Type: Sub-task > Components: integration tests >Reporter: Joseph >Assignee: Joseph >Priority: Minor > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-15935.patch > > > Keep track of which linked lists have been flushed in an HBase table, so that > we can concurrently Walk these lists during the Generation phase. This will > allow us to test: > 1. HBase under concurrent read/writes > 2. Availability of data immediately after flushes (as opposed to waiting till > the Verification phase) > The review diff can be found at: > https://reviews.apache.org/r/48294/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15775) Canary launches two AuthUtil Chores
[ https://issues.apache.org/jira/browse/HBASE-15775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368649#comment-15368649 ] Hudson commented on HBASE-15775: FAILURE: Integrated in HBase-1.4 #276 (See [https://builds.apache.org/job/HBase-1.4/276/]) HBASE-15775 Canary launches two AuthUtil Chores (stack: rev 37040bfc42883a29fc9ff5edb4fe2ab32b81ad63) * hbase-server/src/main/java/org/apache/hadoop/hbase/tool/Canary.java > Canary launches two AuthUtil Chores > --- > > Key: HBASE-15775 > URL: https://issues.apache.org/jira/browse/HBASE-15775 > Project: HBase > Issue Type: Bug > Components: canary >Affects Versions: 0.98.13, 0.98.14, 1.2.0, 1.3.0, 0.98.15, 1.2.1, 0.98.16, > 0.98.17, 0.98.18, 0.98.16.1, 0.98.19, 1.4.0 >Reporter: Sean Busbey >Assignee: Vishal Khandelwal >Priority: Minor > Labels: beginner > Fix For: 1.3.0, 1.2.3 > > Attachments: HBASE-15775.0.98.00.patch, HBASE-15775.branch-1.00.patch > > > Looks like a result of an error in backport done in HBASE-13712. We have a > AuthUtil chore both in main() and in run(). > The one in main() should be removed so that the code is consistent with other > branches. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15682) HBase Backup Phase 3: Possible data loss during incremental WAL files copy
[ https://issues.apache.org/jira/browse/HBASE-15682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-15682: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks for the patch, Vlad. > HBase Backup Phase 3: Possible data loss during incremental WAL files copy > -- > > Key: HBASE-15682 > URL: https://issues.apache.org/jira/browse/HBASE-15682 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Labels: backup > Fix For: 2.0.0 > > Attachments: HBASE-15682-v2.patch, HBASE-15682-v3.patch, > HBASE-15682-v4.patch > > > We collect list of files in WAL and oldWALs directory and launch DistCp job. > Some files can be moved from WALs to oldWALs directory by RS during job's > run, what can result in potential data loss. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15775) Canary launches two AuthUtil Chores
[ https://issues.apache.org/jira/browse/HBASE-15775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368632#comment-15368632 ] Hudson commented on HBASE-15775: SUCCESS: Integrated in HBase-1.2 #667 (See [https://builds.apache.org/job/HBase-1.2/667/]) HBASE-15775 Canary launches two AuthUtil Chores (stack: rev 801d4333cb705c0fd13b7ab06417aeb2349cf3a7) * hbase-server/src/main/java/org/apache/hadoop/hbase/tool/Canary.java > Canary launches two AuthUtil Chores > --- > > Key: HBASE-15775 > URL: https://issues.apache.org/jira/browse/HBASE-15775 > Project: HBase > Issue Type: Bug > Components: canary >Affects Versions: 0.98.13, 0.98.14, 1.2.0, 1.3.0, 0.98.15, 1.2.1, 0.98.16, > 0.98.17, 0.98.18, 0.98.16.1, 0.98.19, 1.4.0 >Reporter: Sean Busbey >Assignee: Vishal Khandelwal >Priority: Minor > Labels: beginner > Fix For: 1.3.0, 1.2.3 > > Attachments: HBASE-15775.0.98.00.patch, HBASE-15775.branch-1.00.patch > > > Looks like a result of an error in backport done in HBASE-13712. We have a > AuthUtil chore both in main() and in run(). > The one in main() should be removed so that the code is consistent with other > branches. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16201) NPE in RpcServer causing intermittent UT failure of TestMasterReplication#testHFileCyclicReplication
[
https://issues.apache.org/jira/browse/HBASE-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368634#comment-15368634
]
Hudson commented on HBASE-16201:
SUCCESS: Integrated in HBase-1.2 #667 (See
[https://builds.apache.org/job/HBase-1.2/667/])
HBASE-16201 fix a NPE issue in RpcServer (liyu: rev
b99efe65e900299271b1e2a0c5feabd23930eb70)
* hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
> NPE in RpcServer causing intermittent UT failure of
> TestMasterReplication#testHFileCyclicReplication
>
>
> Key: HBASE-16201
> URL: https://issues.apache.org/jira/browse/HBASE-16201
> Project: HBase
> Issue Type: Bug
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0, 1.3.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-16201.patch
>
>
> Every several rounds of {{TestMasterReplication#testHFileCyclicReplication}},
> we could observe below NPE in UT log:
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2257)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:118)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:189)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:169)
> {noformat}
> And related codes at RpcServer line 2257 are:
> {code}
> if (e instanceof ServiceException) {
> e = e.getCause();
> }
> // increment the number of requests that were exceptions.
> metrics.exception(e);
> if (e instanceof LinkageError) throw new DoNotRetryIOException(e);
> if (e instanceof IOException) throw (IOException)e;
> {code}
> And after some debugging, we could find several places that constructing
> ServiceException with no cause, such as in
> {{RsRpcServices#replicateWALEntry}}:
> {code}
> if (regionServer.replicationSinkHandler != null) {
> ...
> } else {
> throw new ServiceException("Replication services are not initialized
> yet");
> }
> {code}
> So we should firstly check and only reset {{e=e.getCause()}} when the cause
> is not null
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16172) Unify the retry logic in ScannerCallableWithReplicas and RpcRetryingCallerWithReadReplicas
[
https://issues.apache.org/jira/browse/HBASE-16172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368631#comment-15368631
]
Heng Chen commented on HBASE-16172:
---
+1 on patch v4.
> Unify the retry logic in ScannerCallableWithReplicas and
> RpcRetryingCallerWithReadReplicas
> --
>
> Key: HBASE-16172
> URL: https://issues.apache.org/jira/browse/HBASE-16172
> Project: HBase
> Issue Type: Bug
>Reporter: Yu Li
>Assignee: Ted Yu
> Attachments: 16172.branch-1.v4.txt, 16172.v1.txt, 16172.v2.txt,
> 16172.v2.txt, 16172.v3.txt, 16172.v4.txt, 16172.v4.txt
>
>
> The issue is pointed out by [~devaraj] in HBASE-16132 (Thanks D.D.), that in
> {{RpcRetryingCallerWithReadReplicas#call}} we will call
> {{ResultBoundedCompletionService#take}} instead of {{poll}} to dead-wait on
> the second one if the first replica timed out, while in
> {{ScannerCallableWithReplicas#call}} we still use
> {{ResultBoundedCompletionService#poll}} with some timeout for the 2nd replica.
> This JIRA aims at discussing whether to unify the logic in these two kinds of
> caller with region replica and taking action if necessary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15682) HBase Backup Phase 3: Possible data loss during incremental WAL files copy
[ https://issues.apache.org/jira/browse/HBASE-15682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-15682: -- Attachment: HBASE-15682-v4.patch v4. Yes, I have run tests. > HBase Backup Phase 3: Possible data loss during incremental WAL files copy > -- > > Key: HBASE-15682 > URL: https://issues.apache.org/jira/browse/HBASE-15682 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Labels: backup > Fix For: 2.0.0 > > Attachments: HBASE-15682-v2.patch, HBASE-15682-v3.patch, > HBASE-15682-v4.patch > > > We collect list of files in WAL and oldWALs directory and launch DistCp job. > Some files can be moved from WALs to oldWALs directory by RS during job's > run, what can result in potential data loss. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15682) HBase Backup Phase 3: Possible data loss during incremental WAL files copy
[
https://issues.apache.org/jira/browse/HBASE-15682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368615#comment-15368615
]
Hadoop QA commented on HBASE-15682:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 3s {color}
| {color:red} HBASE-15682 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.2.1/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12816923/HBASE-15682-v3.patch |
| JIRA Issue | HBASE-15682 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/2571/console |
| Powered by | Apache Yetus 0.2.1 http://yetus.apache.org |
This message was automatically generated.
> HBase Backup Phase 3: Possible data loss during incremental WAL files copy
> --
>
> Key: HBASE-15682
> URL: https://issues.apache.org/jira/browse/HBASE-15682
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Labels: backup
> Fix For: 2.0.0
>
> Attachments: HBASE-15682-v2.patch, HBASE-15682-v3.patch
>
>
> We collect list of files in WAL and oldWALs directory and launch DistCp job.
> Some files can be moved from WALs to oldWALs directory by RS during job's
> run, what can result in potential data loss.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16148) Hybrid Logical Clocks(placeholder for running tests)
[ https://issues.apache.org/jira/browse/HBASE-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sai Teja Ranuva updated HBASE-16148: Labels: test-patch (was: ) Status: Patch Available (was: In Progress) Test adding items to Protobuf > Hybrid Logical Clocks(placeholder for running tests) > > > Key: HBASE-16148 > URL: https://issues.apache.org/jira/browse/HBASE-16148 > Project: HBase > Issue Type: Sub-task > Components: API >Reporter: Sai Teja Ranuva >Assignee: Sai Teja Ranuva >Priority: Minor > Labels: test-patch > Attachments: HBASE-16148.master.001.patch > > > This JIRA is just a placeholder to test Hybrid Logical Clocks code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16117) Fix Connection leak in mapred.TableOutputFormat
[
https://issues.apache.org/jira/browse/HBASE-16117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368597#comment-15368597
]
Mikhail Antonov commented on HBASE-16117:
-
Looking at the patch I have few questions.
3167
zkw.getRecoverableZooKeeper().getZooKeeper().exists(zkw.baseZNode, false);
3179 connection.isMasterRunning(); 3168
connection.isMasterRunning();
3169} catch (IOException e) {
3170 throw new ZooKeeperConnectionException("Can't connect to
ZooKeeper", e);
3171} catch (InterruptedException e) {
So we moved up isMasterRunning() check in the try{} block, this one can throw
MNRE which extends IOE and we would masquerade master not running exception
with ZK exception?
Also not sure it's good to add more IOE to Registry interface, those are too
ubiquitous already..any way to avoid that?
> Fix Connection leak in mapred.TableOutputFormat
>
>
> Key: HBASE-16117
> URL: https://issues.apache.org/jira/browse/HBASE-16117
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
>Affects Versions: 2.0.0, 1.3.0, 1.2.2, 1.1.6
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 2.0.0, 1.3.0, 1.1.6, 1.2.3
>
> Attachments: hbase-16117.branch-1.patch, hbase-16117.patch,
> hbase-16117.v2.branch-1.patch, hbase-16117.v2.patch,
> hbase-16117.v3.branch-1.patch, hbase-16117.v3.patch, hbase-16117.v4.patch
>
>
> Spark seems to instantiate multiple instances of output formats within a
> single process. When mapred.TableOutputFormat (not
> mapreduce.TableOutputFormat) is used, this may cause connection leaks that
> slowly exhaust the cluster's zk connections.
> This patch fixes that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14417) Incremental backup and bulk loading
[ https://issues.apache.org/jira/browse/HBASE-14417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368585#comment-15368585 ] Ted Yu commented on HBASE-14417: Looks like bulk data loading should be applied to target table(s) as well. > Incremental backup and bulk loading > --- > > Key: HBASE-14417 > URL: https://issues.apache.org/jira/browse/HBASE-14417 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Critical > Labels: backup > Fix For: 2.0.0 > > > Currently, incremental backup is based on WAL files. Bulk data loading > bypasses WALs for obvious reasons, breaking incremental backups. The only way > to continue backups after bulk loading is to create new full backup of a > table. This may not be feasible for customers who do bulk loading regularly > (say, every day). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15925) compat-module maven variable not evaluated
[
https://issues.apache.org/jira/browse/HBASE-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368584#comment-15368584
]
David Portabella commented on HBASE-15925:
--
Ok, yes, it works :)
> compat-module maven variable not evaluated
> --
>
> Key: HBASE-15925
> URL: https://issues.apache.org/jira/browse/HBASE-15925
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 1.0.0, 1.1.0, 1.2.0, 1.2.1, 1.0.3, 1.1.5
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-15925.1.patch
>
>
> Looks like we've regressed on HBASE-8488. Have a look at the dependency
> artifacts list on
> http://mvnrepository.com/artifact/org.apache.hbase/hbase-testing-util/1.2.1.
> Notice the direct dependency's artifactId is {{$\{compat.module\}}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16176) Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of contention when many threads reading a StoreFile
[
https://issues.apache.org/jira/browse/HBASE-16176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368579#comment-15368579
]
Mikhail Antonov commented on HBASE-16176:
-
[~apurtell] I think it's also affected 0.98.20?
> Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of
> contention when many threads reading a StoreFile
> --
>
> Key: HBASE-16176
> URL: https://issues.apache.org/jira/browse/HBASE-16176
> Project: HBase
> Issue Type: Sub-task
> Components: Performance
>Affects Versions: 1.3.0, 0.98.20
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.21
>
> Attachments: HBASE-16176.branch-1.3.001.patch,
> HBASE-16176.branch-1.3.002.patch, HBASE-16176.branch-1.3.002.patch,
> HBASE-16176.branch-1.3.003.patch
>
>
> Debugging the parent issue, came up with some improvements on old HBASE-15650
> "Remove TimeRangeTracker as point of contention when many threads reading a
> StoreFile". Lets get them in. Here are the changes:
> {code}
> 6 Change HFile Writer constructor so we pass in the TimeRangeTracker,
> if one,
> 7 on construction rather than set later (the flag and reference were
> not
> 8 volatile so could have made for issues in concurrent case) 2. Make
> sure the
> 9 construction of a TimeRange from a TimeRangeTracer on open of an
> HFile Reader
> 10 never makes a bad minimum value, one that would preclude us reading
> any
> 11 values from a file (add a log and set min to 0)
> 12 M hbase-common/src/main/java/org/apache/hadoop/hbase/io/TimeRange.java
> 13 Call through to next constructor (if minStamp was 0, we'd skip
> setting allTime=true)
> 14 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
> 15 Add constructor override that takes a TimeRangeTracker (set when
> flushing but
> 16 not when compacting)
> 17 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
> 18 Add override creating an HFile in tmp that takes a TimeRangeTracker
> 19 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
> 20 Add override for HFile Writer that takes a TimeRangeTracker
> 21 Take it on construction instead of having it passed by a setter
> later (flags
> 22 and reference set by the setter were not volatile... could have been
> prob
> 23 in concurrent case)
> 24 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TimeRangeTracker.java
> 25 Log WARN if bad initial TimeRange value (and then 'fix' it)
> 26 M
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTimeRangeTracker.java
> 27 A few tests to prove serialization works as expected and that we'll
> get a bad min if
> 28 not constructed properly.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16176) Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of contention when many threads reading a StoreFile
[
https://issues.apache.org/jira/browse/HBASE-16176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-16176:
Affects Version/s: 0.98.20
> Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of
> contention when many threads reading a StoreFile
> --
>
> Key: HBASE-16176
> URL: https://issues.apache.org/jira/browse/HBASE-16176
> Project: HBase
> Issue Type: Sub-task
> Components: Performance
>Affects Versions: 1.3.0, 0.98.20
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.21
>
> Attachments: HBASE-16176.branch-1.3.001.patch,
> HBASE-16176.branch-1.3.002.patch, HBASE-16176.branch-1.3.002.patch,
> HBASE-16176.branch-1.3.003.patch
>
>
> Debugging the parent issue, came up with some improvements on old HBASE-15650
> "Remove TimeRangeTracker as point of contention when many threads reading a
> StoreFile". Lets get them in. Here are the changes:
> {code}
> 6 Change HFile Writer constructor so we pass in the TimeRangeTracker,
> if one,
> 7 on construction rather than set later (the flag and reference were
> not
> 8 volatile so could have made for issues in concurrent case) 2. Make
> sure the
> 9 construction of a TimeRange from a TimeRangeTracer on open of an
> HFile Reader
> 10 never makes a bad minimum value, one that would preclude us reading
> any
> 11 values from a file (add a log and set min to 0)
> 12 M hbase-common/src/main/java/org/apache/hadoop/hbase/io/TimeRange.java
> 13 Call through to next constructor (if minStamp was 0, we'd skip
> setting allTime=true)
> 14 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
> 15 Add constructor override that takes a TimeRangeTracker (set when
> flushing but
> 16 not when compacting)
> 17 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
> 18 Add override creating an HFile in tmp that takes a TimeRangeTracker
> 19 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
> 20 Add override for HFile Writer that takes a TimeRangeTracker
> 21 Take it on construction instead of having it passed by a setter
> later (flags
> 22 and reference set by the setter were not volatile... could have been
> prob
> 23 in concurrent case)
> 24 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TimeRangeTracker.java
> 25 Log WARN if bad initial TimeRange value (and then 'fix' it)
> 26 M
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTimeRangeTracker.java
> 27 A few tests to prove serialization works as expected and that we'll
> get a bad min if
> 28 not constructed properly.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16176) Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of contention when many threads reading a StoreFile
[
https://issues.apache.org/jira/browse/HBASE-16176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-16176:
Affects Version/s: 1.3.0
> Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of
> contention when many threads reading a StoreFile
> --
>
> Key: HBASE-16176
> URL: https://issues.apache.org/jira/browse/HBASE-16176
> Project: HBase
> Issue Type: Sub-task
> Components: Performance
>Affects Versions: 1.3.0, 0.98.20
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.21
>
> Attachments: HBASE-16176.branch-1.3.001.patch,
> HBASE-16176.branch-1.3.002.patch, HBASE-16176.branch-1.3.002.patch,
> HBASE-16176.branch-1.3.003.patch
>
>
> Debugging the parent issue, came up with some improvements on old HBASE-15650
> "Remove TimeRangeTracker as point of contention when many threads reading a
> StoreFile". Lets get them in. Here are the changes:
> {code}
> 6 Change HFile Writer constructor so we pass in the TimeRangeTracker,
> if one,
> 7 on construction rather than set later (the flag and reference were
> not
> 8 volatile so could have made for issues in concurrent case) 2. Make
> sure the
> 9 construction of a TimeRange from a TimeRangeTracer on open of an
> HFile Reader
> 10 never makes a bad minimum value, one that would preclude us reading
> any
> 11 values from a file (add a log and set min to 0)
> 12 M hbase-common/src/main/java/org/apache/hadoop/hbase/io/TimeRange.java
> 13 Call through to next constructor (if minStamp was 0, we'd skip
> setting allTime=true)
> 14 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
> 15 Add constructor override that takes a TimeRangeTracker (set when
> flushing but
> 16 not when compacting)
> 17 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
> 18 Add override creating an HFile in tmp that takes a TimeRangeTracker
> 19 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
> 20 Add override for HFile Writer that takes a TimeRangeTracker
> 21 Take it on construction instead of having it passed by a setter
> later (flags
> 22 and reference set by the setter were not volatile... could have been
> prob
> 23 in concurrent case)
> 24 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TimeRangeTracker.java
> 25 Log WARN if bad initial TimeRange value (and then 'fix' it)
> 26 M
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTimeRangeTracker.java
> 27 A few tests to prove serialization works as expected and that we'll
> get a bad min if
> 28 not constructed properly.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16017) HBase TableOutputFormat has connection leak in getRecordWriter
[ https://issues.apache.org/jira/browse/HBASE-16017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-16017: --- Fix Version/s: 1.3.0 Integrated to branch-1.3 as well. > HBase TableOutputFormat has connection leak in getRecordWriter > -- > > Key: HBASE-16017 > URL: https://issues.apache.org/jira/browse/HBASE-16017 > Project: HBase > Issue Type: Bug > Components: mapreduce >Reporter: Zhan Zhang >Assignee: Zhan Zhang > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-16017-1.patch > > > Currently getRecordWriter will not release the connection until jvm > terminate, which is not a right assumption given that the function may be > invoked many times in the same jvm lifecycle. Inside of mapreduce, the issue > has already fixed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16017) HBase TableOutputFormat has connection leak in getRecordWriter
[ https://issues.apache.org/jira/browse/HBASE-16017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368568#comment-15368568 ] Mikhail Antonov commented on HBASE-16017: - +1 for 1.3 [~jmhsieh] [~te...@apache.org] > HBase TableOutputFormat has connection leak in getRecordWriter > -- > > Key: HBASE-16017 > URL: https://issues.apache.org/jira/browse/HBASE-16017 > Project: HBase > Issue Type: Bug > Components: mapreduce >Reporter: Zhan Zhang >Assignee: Zhan Zhang > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-16017-1.patch > > > Currently getRecordWriter will not release the connection until jvm > terminate, which is not a right assumption given that the function may be > invoked many times in the same jvm lifecycle. Inside of mapreduce, the issue > has already fixed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15682) HBase Backup Phase 3: Possible data loss during incremental WAL files copy
[
https://issues.apache.org/jira/browse/HBASE-15682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368559#comment-15368559
]
Ted Yu commented on HBASE-15682:
{code}
+while (counter++ < 2) {
...
+if(counter == 2){
{code}
Can you define a constant prior to the loop and reference the constant ?
> HBase Backup Phase 3: Possible data loss during incremental WAL files copy
> --
>
> Key: HBASE-15682
> URL: https://issues.apache.org/jira/browse/HBASE-15682
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Labels: backup
> Fix For: 2.0.0
>
> Attachments: HBASE-15682-v2.patch, HBASE-15682-v3.patch
>
>
> We collect list of files in WAL and oldWALs directory and launch DistCp job.
> Some files can be moved from WALs to oldWALs directory by RS during job's
> run, what can result in potential data loss.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15682) HBase Backup Phase 3: Possible data loss during incremental WAL files copy
[ https://issues.apache.org/jira/browse/HBASE-15682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368554#comment-15368554 ] Ted Yu commented on HBASE-15682: Have you run backup test suites based on v3 ? > HBase Backup Phase 3: Possible data loss during incremental WAL files copy > -- > > Key: HBASE-15682 > URL: https://issues.apache.org/jira/browse/HBASE-15682 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Labels: backup > Fix For: 2.0.0 > > Attachments: HBASE-15682-v2.patch, HBASE-15682-v3.patch > > > We collect list of files in WAL and oldWALs directory and launch DistCp job. > Some files can be moved from WALs to oldWALs directory by RS during job's > run, what can result in potential data loss. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (HBASE-16198) Enhance backup history command
[ https://issues.apache.org/jira/browse/HBASE-16198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-16198 started by Vladimir Rodionov. - > Enhance backup history command > -- > > Key: HBASE-16198 > URL: https://issues.apache.org/jira/browse/HBASE-16198 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Fix For: 2.0.0 > > > We need history per table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15682) HBase Backup Phase 3: Possible data loss during incremental WAL files copy
[ https://issues.apache.org/jira/browse/HBASE-15682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Rodionov updated HBASE-15682: -- Attachment: HBASE-15682-v3.patch v3. > HBase Backup Phase 3: Possible data loss during incremental WAL files copy > -- > > Key: HBASE-15682 > URL: https://issues.apache.org/jira/browse/HBASE-15682 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Labels: backup > Fix For: 2.0.0 > > Attachments: HBASE-15682-v2.patch, HBASE-15682-v3.patch > > > We collect list of files in WAL and oldWALs directory and launch DistCp job. > Some files can be moved from WALs to oldWALs directory by RS during job's > run, what can result in potential data loss. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16079) TestFailedAppendAndSync fails
[ https://issues.apache.org/jira/browse/HBASE-16079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-16079: Fix Version/s: (was: 1.3.0) > TestFailedAppendAndSync fails > - > > Key: HBASE-16079 > URL: https://issues.apache.org/jira/browse/HBASE-16079 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 1.4.0 >Reporter: Mikhail Antonov > > #testLockupAroundBadAssignSync > https://builds.apache.org/view/All/job/HBase-1.3/751/jdk=latest1.8,label=yahoo-not-h2/testReport/junit/org.apache.hadoop.hbase.regionserver/TestFailedAppendAndSync/testLockupAroundBadAssignSync/ > Error Message > test timed out after 30 milliseconds > Stacktrace > org.junit.runners.model.TestTimedOutException: test timed out after 30 > milliseconds > at > org.mockito.internal.debugging.LocationImpl.toString(LocationImpl.java:29) > at java.lang.String.valueOf(String.java:2994) > at java.lang.StringBuilder.append(StringBuilder.java:131) > at > org.mockito.exceptions.Reporter.wantedButNotInvoked(Reporter.java:320) > at > org.mockito.internal.verification.checkers.MissingInvocationChecker.check(MissingInvocationChecker.java:42) > at org.mockito.internal.verification.AtLeast.verify(AtLeast.java:38) > at > org.mockito.internal.verification.MockAwareVerificationMode.verify(MockAwareVerificationMode.java:21) > at > org.mockito.internal.handler.MockHandlerImpl.handle(MockHandlerImpl.java:72) > at > org.mockito.internal.handler.NullResultGuardian.handle(NullResultGuardian.java:29) > at > org.mockito.internal.handler.InvocationNotifierHandler.handle(InvocationNotifierHandler.java:38) > at > org.mockito.internal.creation.MethodInterceptorFilter.intercept(MethodInterceptorFilter.java:61) > at > org.apache.hadoop.hbase.Server$$EnhancerByMockitoWithCGLIB$$323c5f5b.abort() > at > org.apache.hadoop.hbase.regionserver.TestFailedAppendAndSync.testLockupAroundBadAssignSync(TestFailedAppendAndSync.java:239) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16079) TestFailedAppendAndSync fails
[ https://issues.apache.org/jira/browse/HBASE-16079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-16079: Affects Version/s: (was: 1.3.0) 1.4.0 > TestFailedAppendAndSync fails > - > > Key: HBASE-16079 > URL: https://issues.apache.org/jira/browse/HBASE-16079 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 1.4.0 >Reporter: Mikhail Antonov > > #testLockupAroundBadAssignSync > https://builds.apache.org/view/All/job/HBase-1.3/751/jdk=latest1.8,label=yahoo-not-h2/testReport/junit/org.apache.hadoop.hbase.regionserver/TestFailedAppendAndSync/testLockupAroundBadAssignSync/ > Error Message > test timed out after 30 milliseconds > Stacktrace > org.junit.runners.model.TestTimedOutException: test timed out after 30 > milliseconds > at > org.mockito.internal.debugging.LocationImpl.toString(LocationImpl.java:29) > at java.lang.String.valueOf(String.java:2994) > at java.lang.StringBuilder.append(StringBuilder.java:131) > at > org.mockito.exceptions.Reporter.wantedButNotInvoked(Reporter.java:320) > at > org.mockito.internal.verification.checkers.MissingInvocationChecker.check(MissingInvocationChecker.java:42) > at org.mockito.internal.verification.AtLeast.verify(AtLeast.java:38) > at > org.mockito.internal.verification.MockAwareVerificationMode.verify(MockAwareVerificationMode.java:21) > at > org.mockito.internal.handler.MockHandlerImpl.handle(MockHandlerImpl.java:72) > at > org.mockito.internal.handler.NullResultGuardian.handle(NullResultGuardian.java:29) > at > org.mockito.internal.handler.InvocationNotifierHandler.handle(InvocationNotifierHandler.java:38) > at > org.mockito.internal.creation.MethodInterceptorFilter.intercept(MethodInterceptorFilter.java:61) > at > org.apache.hadoop.hbase.Server$$EnhancerByMockitoWithCGLIB$$323c5f5b.abort() > at > org.apache.hadoop.hbase.regionserver.TestFailedAppendAndSync.testLockupAroundBadAssignSync(TestFailedAppendAndSync.java:239) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16176) Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of contention when many threads reading a StoreFile
[
https://issues.apache.org/jira/browse/HBASE-16176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368512#comment-15368512
]
Mikhail Antonov commented on HBASE-16176:
-
[~stack] +1
> Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of
> contention when many threads reading a StoreFile
> --
>
> Key: HBASE-16176
> URL: https://issues.apache.org/jira/browse/HBASE-16176
> Project: HBase
> Issue Type: Sub-task
> Components: Performance
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.21
>
> Attachments: HBASE-16176.branch-1.3.001.patch,
> HBASE-16176.branch-1.3.002.patch, HBASE-16176.branch-1.3.002.patch,
> HBASE-16176.branch-1.3.003.patch
>
>
> Debugging the parent issue, came up with some improvements on old HBASE-15650
> "Remove TimeRangeTracker as point of contention when many threads reading a
> StoreFile". Lets get them in. Here are the changes:
> {code}
> 6 Change HFile Writer constructor so we pass in the TimeRangeTracker,
> if one,
> 7 on construction rather than set later (the flag and reference were
> not
> 8 volatile so could have made for issues in concurrent case) 2. Make
> sure the
> 9 construction of a TimeRange from a TimeRangeTracer on open of an
> HFile Reader
> 10 never makes a bad minimum value, one that would preclude us reading
> any
> 11 values from a file (add a log and set min to 0)
> 12 M hbase-common/src/main/java/org/apache/hadoop/hbase/io/TimeRange.java
> 13 Call through to next constructor (if minStamp was 0, we'd skip
> setting allTime=true)
> 14 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
> 15 Add constructor override that takes a TimeRangeTracker (set when
> flushing but
> 16 not when compacting)
> 17 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
> 18 Add override creating an HFile in tmp that takes a TimeRangeTracker
> 19 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
> 20 Add override for HFile Writer that takes a TimeRangeTracker
> 21 Take it on construction instead of having it passed by a setter
> later (flags
> 22 and reference set by the setter were not volatile... could have been
> prob
> 23 in concurrent case)
> 24 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TimeRangeTracker.java
> 25 Log WARN if bad initial TimeRange value (and then 'fix' it)
> 26 M
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTimeRangeTracker.java
> 27 A few tests to prove serialization works as expected and that we'll
> get a bad min if
> 28 not constructed properly.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15682) HBase Backup Phase 3: Possible data loss during incremental WAL files copy
[
https://issues.apache.org/jira/browse/HBASE-15682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368500#comment-15368500
]
Vladimir Rodionov commented on HBASE-15682:
---
{quote}
Separator goes first.
{quote}
Why? What is the difference between both versions?
> HBase Backup Phase 3: Possible data loss during incremental WAL files copy
> --
>
> Key: HBASE-15682
> URL: https://issues.apache.org/jira/browse/HBASE-15682
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Vladimir Rodionov
> Labels: backup
> Fix For: 2.0.0
>
> Attachments: HBASE-15682-v2.patch
>
>
> We collect list of files in WAL and oldWALs directory and launch DistCp job.
> Some files can be moved from WALs to oldWALs directory by RS during job's
> run, what can result in potential data loss.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15996) Asynchronous execution mode
[ https://issues.apache.org/jira/browse/HBASE-15996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368498#comment-15368498 ] Vladimir Rodionov commented on HBASE-15996: --- This is for command - line commands. They are all synchronous now. > Asynchronous execution mode > --- > > Key: HBASE-15996 > URL: https://issues.apache.org/jira/browse/HBASE-15996 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Labels: backup > Fix For: 2.0.0 > > > Async mode for backup and restore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15448) HBase Backup Phase 3: Restore optimization 2
[ https://issues.apache.org/jira/browse/HBASE-15448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368495#comment-15368495 ] Vladimir Rodionov commented on HBASE-15448: --- Yes. > HBase Backup Phase 3: Restore optimization 2 > > > Key: HBASE-15448 > URL: https://issues.apache.org/jira/browse/HBASE-15448 > Project: HBase > Issue Type: Task >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Critical > Labels: backup > Fix For: 2.0.0 > > > JIRA opened to continue work on restore optimization. > This will focus on the following > # During incremental backup image restore - restoring full backup into region > boundaries of the most recent incremental backup image. > # Combining multiple tables into single M/R job -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15353) Add metric for number of CallQueueTooBigException's
[ https://issues.apache.org/jira/browse/HBASE-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368485#comment-15368485 ] Mikhail Antonov commented on HBASE-15353: - Thanks [~ghelmling] > Add metric for number of CallQueueTooBigException's > --- > > Key: HBASE-15353 > URL: https://issues.apache.org/jira/browse/HBASE-15353 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC, metrics >Reporter: Elliott Clark >Assignee: Jingcheng Du >Priority: Minor > Labels: beginner, easy > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-15353.patch > > > This exception is being thrown more. We should add a metric for this one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16160) Get the UnsupportedOperationException when using delegation token with encryption
[ https://issues.apache.org/jira/browse/HBASE-16160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368477#comment-15368477 ] Gary Helmling commented on HBASE-16160: --- Changing to blocker, since it means that security + RPC protection + direct byte buffer pool (the default) is broken for 1.3. > Get the UnsupportedOperationException when using delegation token with > encryption > - > > Key: HBASE-16160 > URL: https://issues.apache.org/jira/browse/HBASE-16160 > Project: HBase > Issue Type: Bug >Reporter: Colin Ma >Assignee: Colin Ma >Priority: Blocker > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-16160.001.patch, HBASE-16160.002.patch > > > Using delegation token with encryption, when do the Put operation, will get > the following exception: > [RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345] > ipc.CallRunner(161): > RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345: caught: > java.lang.UnsupportedOperationException > at java.nio.ByteBuffer.array(ByteBuffer.java:959) > at > org.apache.hadoop.hbase.ipc.BufferChain.getBytes(BufferChain.java:66) > at > org.apache.hadoop.hbase.ipc.RpcServer$Call.wrapWithSasl(RpcServer.java:547) > at > org.apache.hadoop.hbase.ipc.RpcServer$Call.setResponse(RpcServer.java:467) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:140) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:189) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:169) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16160) Get the UnsupportedOperationException when using delegation token with encryption
[ https://issues.apache.org/jira/browse/HBASE-16160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368481#comment-15368481 ] Mikhail Antonov commented on HBASE-16160: - +1 for 1.3, thanks for the heads up > Get the UnsupportedOperationException when using delegation token with > encryption > - > > Key: HBASE-16160 > URL: https://issues.apache.org/jira/browse/HBASE-16160 > Project: HBase > Issue Type: Bug >Reporter: Colin Ma >Assignee: Colin Ma >Priority: Blocker > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-16160.001.patch, HBASE-16160.002.patch > > > Using delegation token with encryption, when do the Put operation, will get > the following exception: > [RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345] > ipc.CallRunner(161): > RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345: caught: > java.lang.UnsupportedOperationException > at java.nio.ByteBuffer.array(ByteBuffer.java:959) > at > org.apache.hadoop.hbase.ipc.BufferChain.getBytes(BufferChain.java:66) > at > org.apache.hadoop.hbase.ipc.RpcServer$Call.wrapWithSasl(RpcServer.java:547) > at > org.apache.hadoop.hbase.ipc.RpcServer$Call.setResponse(RpcServer.java:467) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:140) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:189) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:169) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16160) Get the UnsupportedOperationException when using delegation token with encryption
[ https://issues.apache.org/jira/browse/HBASE-16160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Helmling updated HBASE-16160: -- Priority: Blocker (was: Major) > Get the UnsupportedOperationException when using delegation token with > encryption > - > > Key: HBASE-16160 > URL: https://issues.apache.org/jira/browse/HBASE-16160 > Project: HBase > Issue Type: Bug >Reporter: Colin Ma >Assignee: Colin Ma >Priority: Blocker > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-16160.001.patch, HBASE-16160.002.patch > > > Using delegation token with encryption, when do the Put operation, will get > the following exception: > [RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345] > ipc.CallRunner(161): > RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345: caught: > java.lang.UnsupportedOperationException > at java.nio.ByteBuffer.array(ByteBuffer.java:959) > at > org.apache.hadoop.hbase.ipc.BufferChain.getBytes(BufferChain.java:66) > at > org.apache.hadoop.hbase.ipc.RpcServer$Call.wrapWithSasl(RpcServer.java:547) > at > org.apache.hadoop.hbase.ipc.RpcServer$Call.setResponse(RpcServer.java:467) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:140) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:189) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:169) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16160) Get the UnsupportedOperationException when using delegation token with encryption
[ https://issues.apache.org/jira/browse/HBASE-16160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Helmling updated HBASE-16160: -- Fix Version/s: 1.4.0 1.3.0 2.0.0 > Get the UnsupportedOperationException when using delegation token with > encryption > - > > Key: HBASE-16160 > URL: https://issues.apache.org/jira/browse/HBASE-16160 > Project: HBase > Issue Type: Bug >Reporter: Colin Ma >Assignee: Colin Ma > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-16160.001.patch, HBASE-16160.002.patch > > > Using delegation token with encryption, when do the Put operation, will get > the following exception: > [RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345] > ipc.CallRunner(161): > RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345: caught: > java.lang.UnsupportedOperationException > at java.nio.ByteBuffer.array(ByteBuffer.java:959) > at > org.apache.hadoop.hbase.ipc.BufferChain.getBytes(BufferChain.java:66) > at > org.apache.hadoop.hbase.ipc.RpcServer$Call.wrapWithSasl(RpcServer.java:547) > at > org.apache.hadoop.hbase.ipc.RpcServer$Call.setResponse(RpcServer.java:467) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:140) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:189) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:169) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15353) Add metric for number of CallQueueTooBigException's
[ https://issues.apache.org/jira/browse/HBASE-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Helmling updated HBASE-15353: -- Fix Version/s: 1.3.0 > Add metric for number of CallQueueTooBigException's > --- > > Key: HBASE-15353 > URL: https://issues.apache.org/jira/browse/HBASE-15353 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC, metrics >Reporter: Elliott Clark >Assignee: Jingcheng Du >Priority: Minor > Labels: beginner, easy > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-15353.patch > > > This exception is being thrown more. We should add a metric for this one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15353) Add metric for number of CallQueueTooBigException's
[ https://issues.apache.org/jira/browse/HBASE-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368474#comment-15368474 ] Gary Helmling commented on HBASE-15353: --- Cherry-picked to branch-1.3 as well. > Add metric for number of CallQueueTooBigException's > --- > > Key: HBASE-15353 > URL: https://issues.apache.org/jira/browse/HBASE-15353 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC, metrics >Reporter: Elliott Clark >Assignee: Jingcheng Du >Priority: Minor > Labels: beginner, easy > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-15353.patch > > > This exception is being thrown more. We should add a metric for this one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-16202) Backport metric for CallQueueTooBigException to 1.3
[ https://issues.apache.org/jira/browse/HBASE-16202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Helmling resolved HBASE-16202. --- Resolution: Invalid No backport needed, I just did a simple cherry-pick to branch-1.3. > Backport metric for CallQueueTooBigException to 1.3 > --- > > Key: HBASE-16202 > URL: https://issues.apache.org/jira/browse/HBASE-16202 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC, metrics >Reporter: Gary Helmling >Assignee: Gary Helmling > > HBASE-15353 added a separate metric for tracking the number of > CallQueueTooBigExceptions, but only went in to 1.4+. Since CQTBE is already > in 1.2+, it would be nice to at least get this in the upcoming 1.3.0 release. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15448) HBase Backup Phase 3: Restore optimization 2
[ https://issues.apache.org/jira/browse/HBASE-15448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368450#comment-15368450 ] Ted Yu commented on HBASE-15448: For #1, can you clarify (whether restoring full backup or restoring incremental backup is to be optimized) ? Thanks > HBase Backup Phase 3: Restore optimization 2 > > > Key: HBASE-15448 > URL: https://issues.apache.org/jira/browse/HBASE-15448 > Project: HBase > Issue Type: Task >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Critical > Labels: backup > Fix For: 2.0.0 > > > JIRA opened to continue work on restore optimization. > This will focus on the following > # During incremental backup image restore - restoring full backup into region > boundaries of the most recent incremental backup image. > # Combining multiple tables into single M/R job -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16202) Backport metric for CallQueueTooBigException to 1.3
Gary Helmling created HBASE-16202: - Summary: Backport metric for CallQueueTooBigException to 1.3 Key: HBASE-16202 URL: https://issues.apache.org/jira/browse/HBASE-16202 Project: HBase Issue Type: Improvement Components: IPC/RPC, metrics Reporter: Gary Helmling Assignee: Gary Helmling HBASE-15353 added a separate metric for tracking the number of CallQueueTooBigExceptions, but only went in to 1.4+. Since CQTBE is already in 1.2+, it would be nice to at least get this in the upcoming 1.3.0 release. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15996) Asynchronous execution mode
[ https://issues.apache.org/jira/browse/HBASE-15996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368444#comment-15368444 ] Ted Yu commented on HBASE-15996: Is this dup of HBASE-15565 (restore using proc v2) ? There is already async version for table backup. > Asynchronous execution mode > --- > > Key: HBASE-15996 > URL: https://issues.apache.org/jira/browse/HBASE-15996 > Project: HBase > Issue Type: New Feature >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov > Labels: backup > Fix For: 2.0.0 > > > Async mode for backup and restore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16096) Replication keeps accumulating znodes
[ https://issues.apache.org/jira/browse/HBASE-16096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-16096: --- Attachment: HBASE-16096-v2.patch > Replication keeps accumulating znodes > - > > Key: HBASE-16096 > URL: https://issues.apache.org/jira/browse/HBASE-16096 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Ashu Pachauri >Assignee: Joseph > Attachments: HBASE-16096-v1.patch, HBASE-16096-v2.patch, > HBASE-16096.patch > > > If there is an error while creating the replication source on adding the > peer, the source if not added to the in memory list of sources but the > replication peer is. > However, in such a scenario, when you remove the peer, it is deleted from > zookeeper successfully but for removing the in memory list of peers, we wait > for the corresponding sources to get deleted (which as we said don't exist > because of error creating the source). > The problem here is the ordering of operations for adding/removing source and > peer. > Modifying the code to always remove queues from the underlying storage, even > if there exists no sources also requires a small refactoring of > TableBasedReplicationQueuesImpl to not abort on removeQueues() of an empty > queue -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16096) Replication keeps accumulating znodes
[ https://issues.apache.org/jira/browse/HBASE-16096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-16096: --- Attachment: (was: HBASE-16096-v2.patch) > Replication keeps accumulating znodes > - > > Key: HBASE-16096 > URL: https://issues.apache.org/jira/browse/HBASE-16096 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 2.0.0, 1.2.0, 1.3.0 >Reporter: Ashu Pachauri >Assignee: Joseph > Attachments: HBASE-16096-v1.patch, HBASE-16096-v2.patch, > HBASE-16096.patch > > > If there is an error while creating the replication source on adding the > peer, the source if not added to the in memory list of sources but the > replication peer is. > However, in such a scenario, when you remove the peer, it is deleted from > zookeeper successfully but for removing the in memory list of peers, we wait > for the corresponding sources to get deleted (which as we said don't exist > because of error creating the source). > The problem here is the ordering of operations for adding/removing source and > peer. > Modifying the code to always remove queues from the underlying storage, even > if there exists no sources also requires a small refactoring of > TableBasedReplicationQueuesImpl to not abort on removeQueues() of an empty > queue -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16096) Replication keeps accumulating znodes
[ https://issues.apache.org/jira/browse/HBASE-16096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-16096: --- Status: Patch Available (was: Open) > Replication keeps accumulating znodes > - > > Key: HBASE-16096 > URL: https://issues.apache.org/jira/browse/HBASE-16096 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 1.2.0, 2.0.0, 1.3.0 >Reporter: Ashu Pachauri >Assignee: Joseph > Attachments: HBASE-16096-v1.patch, HBASE-16096-v2.patch, > HBASE-16096.patch > > > If there is an error while creating the replication source on adding the > peer, the source if not added to the in memory list of sources but the > replication peer is. > However, in such a scenario, when you remove the peer, it is deleted from > zookeeper successfully but for removing the in memory list of peers, we wait > for the corresponding sources to get deleted (which as we said don't exist > because of error creating the source). > The problem here is the ordering of operations for adding/removing source and > peer. > Modifying the code to always remove queues from the underlying storage, even > if there exists no sources also requires a small refactoring of > TableBasedReplicationQueuesImpl to not abort on removeQueues() of an empty > queue -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16096) Replication keeps accumulating znodes
[ https://issues.apache.org/jira/browse/HBASE-16096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph updated HBASE-16096: --- Status: Open (was: Patch Available) > Replication keeps accumulating znodes > - > > Key: HBASE-16096 > URL: https://issues.apache.org/jira/browse/HBASE-16096 > Project: HBase > Issue Type: Bug > Components: Replication >Affects Versions: 1.2.0, 2.0.0, 1.3.0 >Reporter: Ashu Pachauri >Assignee: Joseph > Attachments: HBASE-16096-v1.patch, HBASE-16096-v2.patch, > HBASE-16096.patch > > > If there is an error while creating the replication source on adding the > peer, the source if not added to the in memory list of sources but the > replication peer is. > However, in such a scenario, when you remove the peer, it is deleted from > zookeeper successfully but for removing the in memory list of peers, we wait > for the corresponding sources to get deleted (which as we said don't exist > because of error creating the source). > The problem here is the ordering of operations for adding/removing source and > peer. > Modifying the code to always remove queues from the underlying storage, even > if there exists no sources also requires a small refactoring of > TableBasedReplicationQueuesImpl to not abort on removeQueues() of an empty > queue -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16160) Get the UnsupportedOperationException when using delegation token with encryption
[ https://issues.apache.org/jira/browse/HBASE-16160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368374#comment-15368374 ] Gary Helmling commented on HBASE-16160: --- Sorry, I missed the review board comments. +1 from me, pending those being addressed. > Get the UnsupportedOperationException when using delegation token with > encryption > - > > Key: HBASE-16160 > URL: https://issues.apache.org/jira/browse/HBASE-16160 > Project: HBase > Issue Type: Bug >Reporter: Colin Ma >Assignee: Colin Ma > Attachments: HBASE-16160.001.patch, HBASE-16160.002.patch > > > Using delegation token with encryption, when do the Put operation, will get > the following exception: > [RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345] > ipc.CallRunner(161): > RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345: caught: > java.lang.UnsupportedOperationException > at java.nio.ByteBuffer.array(ByteBuffer.java:959) > at > org.apache.hadoop.hbase.ipc.BufferChain.getBytes(BufferChain.java:66) > at > org.apache.hadoop.hbase.ipc.RpcServer$Call.wrapWithSasl(RpcServer.java:547) > at > org.apache.hadoop.hbase.ipc.RpcServer$Call.setResponse(RpcServer.java:467) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:140) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:189) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:169) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16201) NPE in RpcServer causing intermittent UT failure of TestMasterReplication#testHFileCyclicReplication
[
https://issues.apache.org/jira/browse/HBASE-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368353#comment-15368353
]
Hudson commented on HBASE-16201:
SUCCESS: Integrated in HBase-1.2-IT #548 (See
[https://builds.apache.org/job/HBase-1.2-IT/548/])
HBASE-16201 fix a NPE issue in RpcServer (liyu: rev
b99efe65e900299271b1e2a0c5feabd23930eb70)
* hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
> NPE in RpcServer causing intermittent UT failure of
> TestMasterReplication#testHFileCyclicReplication
>
>
> Key: HBASE-16201
> URL: https://issues.apache.org/jira/browse/HBASE-16201
> Project: HBase
> Issue Type: Bug
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0, 1.3.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-16201.patch
>
>
> Every several rounds of {{TestMasterReplication#testHFileCyclicReplication}},
> we could observe below NPE in UT log:
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2257)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:118)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:189)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:169)
> {noformat}
> And related codes at RpcServer line 2257 are:
> {code}
> if (e instanceof ServiceException) {
> e = e.getCause();
> }
> // increment the number of requests that were exceptions.
> metrics.exception(e);
> if (e instanceof LinkageError) throw new DoNotRetryIOException(e);
> if (e instanceof IOException) throw (IOException)e;
> {code}
> And after some debugging, we could find several places that constructing
> ServiceException with no cause, such as in
> {{RsRpcServices#replicateWALEntry}}:
> {code}
> if (regionServer.replicationSinkHandler != null) {
> ...
> } else {
> throw new ServiceException("Replication services are not initialized
> yet");
> }
> {code}
> So we should firstly check and only reset {{e=e.getCause()}} when the cause
> is not null
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15775) Canary launches two AuthUtil Chores
[ https://issues.apache.org/jira/browse/HBASE-15775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368352#comment-15368352 ] Hudson commented on HBASE-15775: SUCCESS: Integrated in HBase-1.2-IT #548 (See [https://builds.apache.org/job/HBase-1.2-IT/548/]) HBASE-15775 Canary launches two AuthUtil Chores (stack: rev 801d4333cb705c0fd13b7ab06417aeb2349cf3a7) * hbase-server/src/main/java/org/apache/hadoop/hbase/tool/Canary.java > Canary launches two AuthUtil Chores > --- > > Key: HBASE-15775 > URL: https://issues.apache.org/jira/browse/HBASE-15775 > Project: HBase > Issue Type: Bug > Components: canary >Affects Versions: 0.98.13, 0.98.14, 1.2.0, 1.3.0, 0.98.15, 1.2.1, 0.98.16, > 0.98.17, 0.98.18, 0.98.16.1, 0.98.19, 1.4.0 >Reporter: Sean Busbey >Assignee: Vishal Khandelwal >Priority: Minor > Labels: beginner > Fix For: 1.3.0, 1.2.3 > > Attachments: HBASE-15775.0.98.00.patch, HBASE-15775.branch-1.00.patch > > > Looks like a result of an error in backport done in HBASE-13712. We have a > AuthUtil chore both in main() and in run(). > The one in main() should be removed so that the code is consistent with other > branches. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16160) Get the UnsupportedOperationException when using delegation token with encryption
[ https://issues.apache.org/jira/browse/HBASE-16160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368338#comment-15368338 ] Gary Helmling commented on HBASE-16160: --- cc: [~mantonov], looks like this is also needed in 1.3. > Get the UnsupportedOperationException when using delegation token with > encryption > - > > Key: HBASE-16160 > URL: https://issues.apache.org/jira/browse/HBASE-16160 > Project: HBase > Issue Type: Bug >Reporter: Colin Ma >Assignee: Colin Ma > Attachments: HBASE-16160.001.patch, HBASE-16160.002.patch > > > Using delegation token with encryption, when do the Put operation, will get > the following exception: > [RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345] > ipc.CallRunner(161): > RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345: caught: > java.lang.UnsupportedOperationException > at java.nio.ByteBuffer.array(ByteBuffer.java:959) > at > org.apache.hadoop.hbase.ipc.BufferChain.getBytes(BufferChain.java:66) > at > org.apache.hadoop.hbase.ipc.RpcServer$Call.wrapWithSasl(RpcServer.java:547) > at > org.apache.hadoop.hbase.ipc.RpcServer$Call.setResponse(RpcServer.java:467) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:140) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:189) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:169) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15925) compat-module maven variable not evaluated
[
https://issues.apache.org/jira/browse/HBASE-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368336#comment-15368336
]
Nick Dimiduk commented on HBASE-15925:
--
Yeah, with that repository URL, the version is just 1.2.2 -- no rc part.
> compat-module maven variable not evaluated
> --
>
> Key: HBASE-15925
> URL: https://issues.apache.org/jira/browse/HBASE-15925
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 1.0.0, 1.1.0, 1.2.0, 1.2.1, 1.0.3, 1.1.5
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-15925.1.patch
>
>
> Looks like we've regressed on HBASE-8488. Have a look at the dependency
> artifacts list on
> http://mvnrepository.com/artifact/org.apache.hbase/hbase-testing-util/1.2.1.
> Notice the direct dependency's artifactId is {{$\{compat.module\}}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16176) Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of contention when many threads reading a StoreFile
[
https://issues.apache.org/jira/browse/HBASE-16176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368331#comment-15368331
]
Ted Yu commented on HBASE-16176:
Skimmed the patch.
+1
Nice catch.
> Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of
> contention when many threads reading a StoreFile
> --
>
> Key: HBASE-16176
> URL: https://issues.apache.org/jira/browse/HBASE-16176
> Project: HBase
> Issue Type: Sub-task
> Components: Performance
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.21
>
> Attachments: HBASE-16176.branch-1.3.001.patch,
> HBASE-16176.branch-1.3.002.patch, HBASE-16176.branch-1.3.002.patch,
> HBASE-16176.branch-1.3.003.patch
>
>
> Debugging the parent issue, came up with some improvements on old HBASE-15650
> "Remove TimeRangeTracker as point of contention when many threads reading a
> StoreFile". Lets get them in. Here are the changes:
> {code}
> 6 Change HFile Writer constructor so we pass in the TimeRangeTracker,
> if one,
> 7 on construction rather than set later (the flag and reference were
> not
> 8 volatile so could have made for issues in concurrent case) 2. Make
> sure the
> 9 construction of a TimeRange from a TimeRangeTracer on open of an
> HFile Reader
> 10 never makes a bad minimum value, one that would preclude us reading
> any
> 11 values from a file (add a log and set min to 0)
> 12 M hbase-common/src/main/java/org/apache/hadoop/hbase/io/TimeRange.java
> 13 Call through to next constructor (if minStamp was 0, we'd skip
> setting allTime=true)
> 14 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
> 15 Add constructor override that takes a TimeRangeTracker (set when
> flushing but
> 16 not when compacting)
> 17 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
> 18 Add override creating an HFile in tmp that takes a TimeRangeTracker
> 19 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
> 20 Add override for HFile Writer that takes a TimeRangeTracker
> 21 Take it on construction instead of having it passed by a setter
> later (flags
> 22 and reference set by the setter were not volatile... could have been
> prob
> 23 in concurrent case)
> 24 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TimeRangeTracker.java
> 25 Log WARN if bad initial TimeRange value (and then 'fix' it)
> 26 M
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTimeRangeTracker.java
> 27 A few tests to prove serialization works as expected and that we'll
> get a bad min if
> 28 not constructed properly.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15925) compat-module maven variable not evaluated
[
https://issues.apache.org/jira/browse/HBASE-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368327#comment-15368327
]
David Portabella commented on HBASE-15925:
--
{quote}
David Portabella can you verify 1.2.2rc2 specifically with it's repository:
https://repository.apache.org/content/repositories/orgapachehbase-1142/
{quote}
I try as follows:
{code}
$ git clone https://github.com/dportabella/spark-examples.git
# edit build.sbt:
replace: val hbaseVersion = "1.2.2rc2"
add: resolvers += "Apache Staging" at
"https://repository.apache.org/content/repositories/orgapachehbase-1142/;
$ rm -Rf /david/.ivy2/cache/org.apache.hbase
$ rm -Rf /david/.m2/repository/org/apache/hbase/
$ sbt run
{code}
result:
{code}
[info] Resolving org.apache.hbase#hbase-testing-util;1.2.2rc2 ...
[warn] module not found: org.apache.hbase#hbase-testing-util;1.2.2rc2
[warn]
https://repository.apache.org/content/repositories/orgapachehbase-1142/org/apache/hbase/hbase-testing-util/1.2.2rc2/hbase-testing-util-1.2.2rc2.pom
{code}
it does not find 1.2.2rc2. am i doing sthy wrong?
> compat-module maven variable not evaluated
> --
>
> Key: HBASE-15925
> URL: https://issues.apache.org/jira/browse/HBASE-15925
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 1.0.0, 1.1.0, 1.2.0, 1.2.1, 1.0.3, 1.1.5
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-15925.1.patch
>
>
> Looks like we've regressed on HBASE-8488. Have a look at the dependency
> artifacts list on
> http://mvnrepository.com/artifact/org.apache.hbase/hbase-testing-util/1.2.1.
> Notice the direct dependency's artifactId is {{$\{compat.module\}}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16160) Get the UnsupportedOperationException when using delegation token with encryption
[ https://issues.apache.org/jira/browse/HBASE-16160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368322#comment-15368322 ] Gary Helmling commented on HBASE-16160: --- +1 on v2 of the patch. I'll apply to master and branch-1 shortly unless any objections. > Get the UnsupportedOperationException when using delegation token with > encryption > - > > Key: HBASE-16160 > URL: https://issues.apache.org/jira/browse/HBASE-16160 > Project: HBase > Issue Type: Bug >Reporter: Colin Ma >Assignee: Colin Ma > Attachments: HBASE-16160.001.patch, HBASE-16160.002.patch > > > Using delegation token with encryption, when do the Put operation, will get > the following exception: > [RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345] > ipc.CallRunner(161): > RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345: caught: > java.lang.UnsupportedOperationException > at java.nio.ByteBuffer.array(ByteBuffer.java:959) > at > org.apache.hadoop.hbase.ipc.BufferChain.getBytes(BufferChain.java:66) > at > org.apache.hadoop.hbase.ipc.RpcServer$Call.wrapWithSasl(RpcServer.java:547) > at > org.apache.hadoop.hbase.ipc.RpcServer$Call.setResponse(RpcServer.java:467) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:140) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:189) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:169) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16161) Remove a few unnecessary (uncontended) synchronizes
[ https://issues.apache.org/jira/browse/HBASE-16161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368294#comment-15368294 ] stack commented on HBASE-16161: --- Any chance of a +1 here? [~chenheng] or [~lhofhansl], what you think? > Remove a few unnecessary (uncontended) synchronizes > --- > > Key: HBASE-16161 > URL: https://issues.apache.org/jira/browse/HBASE-16161 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack >Priority: Minor > Attachments: HBASE-16161.branch-1.001.patch > > > This is followon from HBASE-15716. We have a few odd looking synchronizes. A > few are probably elided after analysis concludes single thread consumer but > lets remove to be clear. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16176) Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of contention when many threads reading a StoreFile
[
https://issues.apache.org/jira/browse/HBASE-16176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368292#comment-15368292
]
stack commented on HBASE-16176:
---
Two loop cycles with this patch passed. They used to fail. Am running with more
aggressive killing and larger sizes.
Any chance of a +1 here?
> Bug fixes/improvements on HBASE-15650 Remove TimeRangeTracker as point of
> contention when many threads reading a StoreFile
> --
>
> Key: HBASE-16176
> URL: https://issues.apache.org/jira/browse/HBASE-16176
> Project: HBase
> Issue Type: Sub-task
> Components: Performance
>Reporter: stack
>Assignee: stack
> Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.21
>
> Attachments: HBASE-16176.branch-1.3.001.patch,
> HBASE-16176.branch-1.3.002.patch, HBASE-16176.branch-1.3.002.patch,
> HBASE-16176.branch-1.3.003.patch
>
>
> Debugging the parent issue, came up with some improvements on old HBASE-15650
> "Remove TimeRangeTracker as point of contention when many threads reading a
> StoreFile". Lets get them in. Here are the changes:
> {code}
> 6 Change HFile Writer constructor so we pass in the TimeRangeTracker,
> if one,
> 7 on construction rather than set later (the flag and reference were
> not
> 8 volatile so could have made for issues in concurrent case) 2. Make
> sure the
> 9 construction of a TimeRange from a TimeRangeTracer on open of an
> HFile Reader
> 10 never makes a bad minimum value, one that would preclude us reading
> any
> 11 values from a file (add a log and set min to 0)
> 12 M hbase-common/src/main/java/org/apache/hadoop/hbase/io/TimeRange.java
> 13 Call through to next constructor (if minStamp was 0, we'd skip
> setting allTime=true)
> 14 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
> 15 Add constructor override that takes a TimeRangeTracker (set when
> flushing but
> 16 not when compacting)
> 17 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
> 18 Add override creating an HFile in tmp that takes a TimeRangeTracker
> 19 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
> 20 Add override for HFile Writer that takes a TimeRangeTracker
> 21 Take it on construction instead of having it passed by a setter
> later (flags
> 22 and reference set by the setter were not volatile... could have been
> prob
> 23 in concurrent case)
> 24 M
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/TimeRangeTracker.java
> 25 Log WARN if bad initial TimeRange value (and then 'fix' it)
> 26 M
> hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestTimeRangeTracker.java
> 27 A few tests to prove serialization works as expected and that we'll
> get a bad min if
> 28 not constructed properly.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15935) Have a separate Walker task running concurrently with Generator
[ https://issues.apache.org/jira/browse/HBASE-15935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368213#comment-15368213 ] Hudson commented on HBASE-15935: FAILURE: Integrated in HBase-Trunk_matrix #1192 (See [https://builds.apache.org/job/HBase-Trunk_matrix/1192/]) HBASE-15935 Set up a concurrent walker that walks flushed circular (eclark: rev cb118c8de6ddb783e90c07912a5fbdd629eabf06) * hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestBigLinkedListWithVisibility.java * hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestReplication.java * hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestBigLinkedList.java > Have a separate Walker task running concurrently with Generator > -- > > Key: HBASE-15935 > URL: https://issues.apache.org/jira/browse/HBASE-15935 > Project: HBase > Issue Type: Sub-task > Components: integration tests >Reporter: Joseph >Assignee: Joseph >Priority: Minor > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-15935.patch > > > Keep track of which linked lists have been flushed in an HBase table, so that > we can concurrently Walk these lists during the Generation phase. This will > allow us to test: > 1. HBase under concurrent read/writes > 2. Availability of data immediately after flushes (as opposed to waiting till > the Verification phase) > The review diff can be found at: > https://reviews.apache.org/r/48294/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16171) Fix the potential problems in TestHCM.testConnectionCloseAllowsInterrupt
[ https://issues.apache.org/jira/browse/HBASE-16171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368214#comment-15368214 ] Hudson commented on HBASE-16171: FAILURE: Integrated in HBase-Trunk_matrix #1192 (See [https://builds.apache.org/job/HBase-Trunk_matrix/1192/]) HBASE-16171 Fix the potential problems in (stack: rev 5e0d97e0e95434b03ba886279180a031ed997dfd) * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java > Fix the potential problems in TestHCM.testConnectionCloseAllowsInterrupt > > > Key: HBASE-16171 > URL: https://issues.apache.org/jira/browse/HBASE-16171 > Project: HBase > Issue Type: Bug >Reporter: Colin Ma >Assignee: Colin Ma > Fix For: 2.0.0 > > Attachments: HBASE-16171.001.patch, HBASE-16171.001.patch > > > TestHCM.testConnectionCloseAllowsInterrupt is not stable in QA runs, and > always failed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16160) Get the UnsupportedOperationException when using delegation token with encryption
[ https://issues.apache.org/jira/browse/HBASE-16160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368140#comment-15368140 ] Ted Yu commented on HBASE-16160: TestReplicationSmallTests is tracked in HBASE-16185. Its failure is not related to the patch. > Get the UnsupportedOperationException when using delegation token with > encryption > - > > Key: HBASE-16160 > URL: https://issues.apache.org/jira/browse/HBASE-16160 > Project: HBase > Issue Type: Bug >Reporter: Colin Ma >Assignee: Colin Ma > Attachments: HBASE-16160.001.patch, HBASE-16160.002.patch > > > Using delegation token with encryption, when do the Put operation, will get > the following exception: > [RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345] > ipc.CallRunner(161): > RpcServer.FifoWFPBQ.priority.handler=5,queue=1,port=48345: caught: > java.lang.UnsupportedOperationException > at java.nio.ByteBuffer.array(ByteBuffer.java:959) > at > org.apache.hadoop.hbase.ipc.BufferChain.getBytes(BufferChain.java:66) > at > org.apache.hadoop.hbase.ipc.RpcServer$Call.wrapWithSasl(RpcServer.java:547) > at > org.apache.hadoop.hbase.ipc.RpcServer$Call.setResponse(RpcServer.java:467) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:140) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:189) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:169) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15935) Have a separate Walker task running concurrently with Generator
[ https://issues.apache.org/jira/browse/HBASE-15935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368134#comment-15368134 ] Hudson commented on HBASE-15935: SUCCESS: Integrated in HBase-1.3 #773 (See [https://builds.apache.org/job/HBase-1.3/773/]) HBASE-15935 Set up a concurrent walker that walks flushed circular (eclark: rev 5ad4a824f1b39df7e6cdd22cac8146c84032496a) * hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestBigLinkedListWithVisibility.java * hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestBigLinkedList.java * hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestReplication.java > Have a separate Walker task running concurrently with Generator > -- > > Key: HBASE-15935 > URL: https://issues.apache.org/jira/browse/HBASE-15935 > Project: HBase > Issue Type: Sub-task > Components: integration tests >Reporter: Joseph >Assignee: Joseph >Priority: Minor > Fix For: 2.0.0, 1.3.0, 1.4.0 > > Attachments: HBASE-15935.patch > > > Keep track of which linked lists have been flushed in an HBase table, so that > we can concurrently Walk these lists during the Generation phase. This will > allow us to test: > 1. HBase under concurrent read/writes > 2. Availability of data immediately after flushes (as opposed to waiting till > the Verification phase) > The review diff can be found at: > https://reviews.apache.org/r/48294/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14422) Fix TestFastFailWithoutTestUtil
[ https://issues.apache.org/jira/browse/HBASE-14422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368116#comment-15368116 ] stack commented on HBASE-14422: --- [~ryakhovskiy] Yes. Nice work here Sir. Thank you for persisting. > Fix TestFastFailWithoutTestUtil > --- > > Key: HBASE-14422 > URL: https://issues.apache.org/jira/browse/HBASE-14422 > Project: HBase > Issue Type: Task > Components: test >Reporter: stack >Assignee: Konstantin Ryakhovskiy >Priority: Minor > Labels: beginner > Attachments: HBASE-14422.master.001.patch, > HBASE-14422.master.002.patch, HBASE-14422.master.003.patch, > HBASE-14422.master.004.patch, HBASE-14422.master.005.patch, > HBASE-14422.master.006.patch, HBASE-14422.master.007.patch, > HBASE-14422.master.008.patch, HBASE-14422.master.009.patch, > HBASE-14422.master.010.patch, HBASE-14422.master.011.patch, > HBASE-14422.master.012.patch, HBASE-14422.master.013.patch, > HBASE-14422.master.014.patch, HBASE-14422.master.015.patch, > HBASE-14422.master.016.patch, HBASE-14422.master.017.patch, log.txt, trace.log > > > TestFastFailWithoutTestUtil has a unit test that does > testInterceptorIntercept50Times Usually it passes but on occasion, the > latching between thread 1 and thread 2 goes awry and the test hangs and the > test hangs out. Depends on the hardware but it seems to happen about one in > four runs here on an internal rig. > HBASE-14421 changed the wait-on-latch to timeout and do a thread dump and > just let the test keep going. > This issue is about digging in on figuring why the hang up on latches and > then fixing it so the test doesn't have to have the latch timeout. Hopefully > the threaddump helps. > This one could be hard to fix since it not easy to reproduce. Marking it > beginner anyways. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16074) ITBLL fails, reports lost big or tiny families
[ https://issues.apache.org/jira/browse/HBASE-16074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-16074: Assignee: stack (was: Mikhail Antonov) > ITBLL fails, reports lost big or tiny families > -- > > Key: HBASE-16074 > URL: https://issues.apache.org/jira/browse/HBASE-16074 > Project: HBase > Issue Type: Bug > Components: integration tests >Affects Versions: 1.3.0, 0.98.20 >Reporter: Mikhail Antonov >Assignee: stack >Priority: Blocker > Fix For: 2.0.0, 1.3.0, 1.4.0, 0.98.21 > > Attachments: 16074.test.branch-1.3.patch, 16074.test.patch, > HBASE-16074.branch-1.3.001.patch, HBASE-16074.branch-1.3.002.patch, > HBASE-16074.branch-1.3.003.patch, HBASE-16074.branch-1.3.003.patch, > changes_to_stress_ITBLL.patch, changes_to_stress_ITBLL__a_bit_relaxed_.patch, > itbll log with failure, itbll log with success > > > Underlying MR jobs succeed but I'm seeing the following in the logs (mid-size > distributed test cluster): > ERROR test.IntegrationTestBigLinkedList$Verify: Found nodes which lost big or > tiny families, count=164 > I do not know exactly yet whether it's a bug, a test issue or env setup > issue, but need figure it out. Opening this to raise awareness and see if > someone saw that recently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16201) NPE in RpcServer causing intermittent UT failure of TestMasterReplication#testHFileCyclicReplication
[
https://issues.apache.org/jira/browse/HBASE-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368082#comment-15368082
]
Yu Li commented on HBASE-16201:
---
I see, thanks for the quick response sir [~stack], will follow the process and
make the change
> NPE in RpcServer causing intermittent UT failure of
> TestMasterReplication#testHFileCyclicReplication
>
>
> Key: HBASE-16201
> URL: https://issues.apache.org/jira/browse/HBASE-16201
> Project: HBase
> Issue Type: Bug
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-16201.patch
>
>
> Every several rounds of {{TestMasterReplication#testHFileCyclicReplication}},
> we could observe below NPE in UT log:
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2257)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:118)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:189)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:169)
> {noformat}
> And related codes at RpcServer line 2257 are:
> {code}
> if (e instanceof ServiceException) {
> e = e.getCause();
> }
> // increment the number of requests that were exceptions.
> metrics.exception(e);
> if (e instanceof LinkageError) throw new DoNotRetryIOException(e);
> if (e instanceof IOException) throw (IOException)e;
> {code}
> And after some debugging, we could find several places that constructing
> ServiceException with no cause, such as in
> {{RsRpcServices#replicateWALEntry}}:
> {code}
> if (regionServer.replicationSinkHandler != null) {
> ...
> } else {
> throw new ServiceException("Replication services are not initialized
> yet");
> }
> {code}
> So we should firstly check and only reset {{e=e.getCause()}} when the cause
> is not null
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16142) Trigger JFR session when under duress -- e.g. backed-up request queue count -- and dump the recording to log dir
[ https://issues.apache.org/jira/browse/HBASE-16142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368080#comment-15368080 ] stack commented on HBASE-16142: --- Yes. Excellent. Need a 'button' on the UI or in shell to enable the feature. Also need to add the flags enable commercial features and the jfr thing too Thanks for doing this [~ryakhovskiy] > Trigger JFR session when under duress -- e.g. backed-up request queue count > -- and dump the recording to log dir > > > Key: HBASE-16142 > URL: https://issues.apache.org/jira/browse/HBASE-16142 > Project: HBase > Issue Type: Task > Components: Operability >Reporter: stack >Priority: Minor > Labels: beginner > > Chatting today w/ a mighty hbase operator on how to figure what is happening > during transitory latency spike or any other transitory 'weirdness' in a > server, the idea came up that a java flight recording during a spike would > include a pretty good picture of what is going on during the time of duress > (more ideal would be a trace of the explicit slow queries showing call stack > with timings dumped to a sink for later review; i.e. trigger an htrace when a > query is slow...). > Taking a look, programmatically triggering a JFR recording seems doable, if > awkward (MBean invocations). There is even a means of specifying 'triggers' > based off any published mbean emission -- e.g. a query queue count threshold > -- which looks nice. See > https://community.oracle.com/thread/3676275?start=0=0 and > https://docs.oracle.com/javacomponents/jmc-5-4/jfr-runtime-guide/run.htm#JFRUH184 > This feature could start out as a blog post describing how to do it for one > server. A plugin on Canary that looks at mbean values and if over a > configured threshold, triggers a recording remotely could be next. Finally > could integrate a couple of triggers that fire when issue via the trigger > mechanism. > Marking as beginner feature. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16201) NPE in RpcServer causing intermittent UT failure of TestMasterReplication#testHFileCyclicReplication
[
https://issues.apache.org/jira/browse/HBASE-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368063#comment-15368063
]
stack commented on HBASE-16201:
---
+1 on patch.
> NPE in RpcServer causing intermittent UT failure of
> TestMasterReplication#testHFileCyclicReplication
>
>
> Key: HBASE-16201
> URL: https://issues.apache.org/jira/browse/HBASE-16201
> Project: HBase
> Issue Type: Bug
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-16201.patch
>
>
> Every several rounds of {{TestMasterReplication#testHFileCyclicReplication}},
> we could observe below NPE in UT log:
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2257)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:118)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:189)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:169)
> {noformat}
> And related codes at RpcServer line 2257 are:
> {code}
> if (e instanceof ServiceException) {
> e = e.getCause();
> }
> // increment the number of requests that were exceptions.
> metrics.exception(e);
> if (e instanceof LinkageError) throw new DoNotRetryIOException(e);
> if (e instanceof IOException) throw (IOException)e;
> {code}
> And after some debugging, we could find several places that constructing
> ServiceException with no cause, such as in
> {{RsRpcServices#replicateWALEntry}}:
> {code}
> if (regionServer.replicationSinkHandler != null) {
> ...
> } else {
> throw new ServiceException("Replication services are not initialized
> yet");
> }
> {code}
> So we should firstly check and only reset {{e=e.getCause()}} when the cause
> is not null
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16201) NPE in RpcServer causing intermittent UT failure of TestMasterReplication#testHFileCyclicReplication
[
https://issues.apache.org/jira/browse/HBASE-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368061#comment-15368061
]
stack commented on HBASE-16201:
---
No problem. Yes, revert and reapply. It is a pain but the only way to fix a
message that has been pushed to the public repo. [~carp84]
> NPE in RpcServer causing intermittent UT failure of
> TestMasterReplication#testHFileCyclicReplication
>
>
> Key: HBASE-16201
> URL: https://issues.apache.org/jira/browse/HBASE-16201
> Project: HBase
> Issue Type: Bug
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-16201.patch
>
>
> Every several rounds of {{TestMasterReplication#testHFileCyclicReplication}},
> we could observe below NPE in UT log:
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2257)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:118)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:189)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:169)
> {noformat}
> And related codes at RpcServer line 2257 are:
> {code}
> if (e instanceof ServiceException) {
> e = e.getCause();
> }
> // increment the number of requests that were exceptions.
> metrics.exception(e);
> if (e instanceof LinkageError) throw new DoNotRetryIOException(e);
> if (e instanceof IOException) throw (IOException)e;
> {code}
> And after some debugging, we could find several places that constructing
> ServiceException with no cause, such as in
> {{RsRpcServices#replicateWALEntry}}:
> {code}
> if (regionServer.replicationSinkHandler != null) {
> ...
> } else {
> throw new ServiceException("Replication services are not initialized
> yet");
> }
> {code}
> So we should firstly check and only reset {{e=e.getCause()}} when the cause
> is not null
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14479) Apply the Leader/Followers pattern to RpcServer's Reader
[
https://issues.apache.org/jira/browse/HBASE-14479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368055#comment-15368055
]
stack commented on HBASE-14479:
---
bq. I found that the method Reader.doRead(SelectionKey) just does one request
for each call, regardless of whether the next request is available...
How do you mean [~ikeda]? The doRunLoop will doRead for each key gotten on a
select.
bq. BTW, in order to resolve this, when we read as many requests from a
connection as possible, the queue will easily become full and it will be
difficult to handle requests fairly as to connections. I think it is better to
cap the count of the requests simultaneously executing for each connection,
according to the current requests queued (instead of using a fixed bounded
queue).
Sounds good. I can test any experiments you might want to try.
Thanks.
> Apply the Leader/Followers pattern to RpcServer's Reader
>
>
> Key: HBASE-14479
> URL: https://issues.apache.org/jira/browse/HBASE-14479
> Project: HBase
> Issue Type: Improvement
> Components: IPC/RPC, Performance
>Reporter: Hiroshi Ikeda
>Assignee: Hiroshi Ikeda
>Priority: Minor
> Attachments: HBASE-14479-V2 (1).patch, HBASE-14479-V2.patch,
> HBASE-14479-V2.patch, HBASE-14479.patch, flamegraph-19152.svg,
> flamegraph-32667.svg, gc.png, gets.png, io.png, median.png
>
>
> {{RpcServer}} uses multiple selectors to read data for load distribution, but
> the distribution is just done by round-robin. It is uncertain, especially for
> long run, whether load is equally divided and resources are used without
> being wasted.
> Moreover, multiple selectors may cause excessive context switches which give
> priority to low latency (while we just add the requests to queues), and it is
> possible to reduce throughput of the whole server.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16201) NPE in RpcServer causing intermittent UT failure of TestMasterReplication#testHFileCyclicReplication
[
https://issues.apache.org/jira/browse/HBASE-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368054#comment-15368054
]
Yu Li commented on HBASE-16201:
---
Pushed into master, branch-1, branch-1.2 and branch-1.3, and just noticed that
I missed adding JIRA number in the commit log... Do I need to revert the commit
and update the commit log? Sorry for the trouble... [~stack]
> NPE in RpcServer causing intermittent UT failure of
> TestMasterReplication#testHFileCyclicReplication
>
>
> Key: HBASE-16201
> URL: https://issues.apache.org/jira/browse/HBASE-16201
> Project: HBase
> Issue Type: Bug
>Reporter: Yu Li
>Assignee: Yu Li
> Attachments: HBASE-16201.patch
>
>
> Every several rounds of {{TestMasterReplication#testHFileCyclicReplication}},
> we could observe below NPE in UT log:
> {noformat}
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2257)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:118)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:189)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:169)
> {noformat}
> And related codes at RpcServer line 2257 are:
> {code}
> if (e instanceof ServiceException) {
> e = e.getCause();
> }
> // increment the number of requests that were exceptions.
> metrics.exception(e);
> if (e instanceof LinkageError) throw new DoNotRetryIOException(e);
> if (e instanceof IOException) throw (IOException)e;
> {code}
> And after some debugging, we could find several places that constructing
> ServiceException with no cause, such as in
> {{RsRpcServices#replicateWALEntry}}:
> {code}
> if (regionServer.replicationSinkHandler != null) {
> ...
> } else {
> throw new ServiceException("Replication services are not initialized
> yet");
> }
> {code}
> So we should firstly check and only reset {{e=e.getCause()}} when the cause
> is not null
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16194) Should count in MSLAB chunk allocation into heap size change when adding duplicate cells
[ https://issues.apache.org/jira/browse/HBASE-16194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-16194: --- Summary: Should count in MSLAB chunk allocation into heap size change when adding duplicate cells (was: Should count in MSLAB chunk allocation into heap size change when adding duplicated cells) > Should count in MSLAB chunk allocation into heap size change when adding > duplicate cells > > > Key: HBASE-16194 > URL: https://issues.apache.org/jira/browse/HBASE-16194 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.2.1, 1.1.5, 0.98.20 >Reporter: Yu Li >Assignee: Yu Li > Attachments: HBASE-16194.patch, HBASE-16194_v2.patch > > > See more details about problem description and analysis in HBASE-16193 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16134) Introduce InternalCell
[ https://issues.apache.org/jira/browse/HBASE-16134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368034#comment-15368034 ] stack commented on HBASE-16134: --- Can the Codec that we install at the RPC edge be one that filters out tags? So, a Codec that passes Tags and another version that filters them out? > Introduce InternalCell > -- > > Key: HBASE-16134 > URL: https://issues.apache.org/jira/browse/HBASE-16134 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0 > > Attachments: HBASE-16134.patch, HBASE-16134.patch, > HBASE-16134_V2.patch > > > Came after the discussion under HBASE-15721 and HBASE-15879. > InternalCell is a Cell extension. We do have Cell extensions across different > interfaces now. > SettableSeqId > SettableTimestamp > Streamable. > And demand for this keep growing. > So let us include everything into one Interface. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14743) Add metrics around HeapMemoryManager
[ https://issues.apache.org/jira/browse/HBASE-14743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368017#comment-15368017 ] stack commented on HBASE-14743: --- [~reidchan] It doesn't look like it is your code that is at issue here. A fix for TestHCM just went in too. @appy is out for a few days and will be back here on review of your latest when he is back. Just FYI. > Add metrics around HeapMemoryManager > > > Key: HBASE-14743 > URL: https://issues.apache.org/jira/browse/HBASE-14743 > Project: HBase > Issue Type: Improvement >Reporter: Elliott Clark >Assignee: Reid Chan >Priority: Minor > Attachments: HBASE-14743.009.patch, HBASE-14743.009.rw3.patch, > HBASE-14743.009.v2.patch, HBASE-14743.010.patch, HBASE-14743.010.v2.patch, > Metrics snapshot 2016-6-30.png, Screen Shot 2016-06-16 at 5.39.13 PM.png, > test2_1.png, test2_2.png, test2_3.png, test2_4.png > > > it would be good to know how many invocations there have been. > How many decided to expand memstore. > How many decided to expand block cache. > How many decided to do nothing. > etc. > When that's done use those metrics to clean up the tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16132) Scan does not return all the result when regionserver is busy
[
https://issues.apache.org/jira/browse/HBASE-16132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-16132:
Component/s: Client
> Scan does not return all the result when regionserver is busy
> -
>
> Key: HBASE-16132
> URL: https://issues.apache.org/jira/browse/HBASE-16132
> Project: HBase
> Issue Type: Bug
> Components: Client
>Reporter: binlijin
>Assignee: binlijin
> Fix For: 2.0.0, 1.3.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-16132.patch, HBASE-16132_v2.patch,
> HBASE-16132_v3.patch, HBASE-16132_v3.patch, TestScanMissingData.java
>
>
> We have find some corner case, when regionserver is busy and last a long
> time. Some scanner may return null even if they do not scan all data.
> We find in ScannerCallableWithReplicas there is a case do not handler
> correct, when cs.poll timeout and do not return any result , it is will
> return a null result, so scan get null result, and end the scan.
> {code}
> try {
> Future> f = cs.poll(timeout,
> TimeUnit.MILLISECONDS);
> if (f != null) {
> Pair r = f.get(timeout,
> TimeUnit.MILLISECONDS);
> if (r != null && r.getSecond() != null) {
> updateCurrentlyServingReplica(r.getSecond(), r.getFirst(), done,
> pool);
> }
> return r == null ? null : r.getFirst(); // great we got an answer
> }
> } catch (ExecutionException e) {
> RpcRetryingCallerWithReadReplicas.throwEnrichedException(e, retries);
> } catch (CancellationException e) {
> throw new InterruptedIOException(e.getMessage());
> } catch (InterruptedException e) {
> throw new InterruptedIOException(e.getMessage());
> } catch (TimeoutException e) {
> throw new InterruptedIOException(e.getMessage());
> } finally {
> // We get there because we were interrupted or because one or more of
> the
> // calls succeeded or failed. In all case, we stop all our tasks.
> cs.cancelAll();
> }
> return null; // unreachable
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16055) PutSortReducer loses any Visibility/acl attribute set on the Puts
[ https://issues.apache.org/jira/browse/HBASE-16055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368013#comment-15368013 ] Hudson commented on HBASE-16055: FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #1238 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/1238/]) Revert "HBASE-16055 PutSortReducer loses any Visibility/acl attribute (apurtell: rev 60ff6128ef297a379f75dfdca5fcac133eab9e55) * hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat2.java * hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/PutSortReducer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TextSortReducer.java > PutSortReducer loses any Visibility/acl attribute set on the Puts > -- > > Key: HBASE-16055 > URL: https://issues.apache.org/jira/browse/HBASE-16055 > Project: HBase > Issue Type: Bug > Components: security >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 2.0.0, 1.0.4, 1.4.0 > > Attachments: HBASE-16055_1.patch, HBASE-16055_2.patch > > > Based on a user discussion, I think as the user pointed out rightly, when a > PutSortReducer is used any visibility attribute or external attribute set on > the Put will be lost as we create KVs out of the cells in the puts whereas > the ACL and visibility are all set as Attributes. > In TextSortReducer we tend to read the information we tend to read the > information from the parsed line but here in PutSortReducer we don't do it. I > think this problem should be in all the existing versions where we support > Tags. Correct me if am wrong here. > [~anoop.hbase], [~andrew.purt...@gmail.com]? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16196) Update jruby to a newer version.
[ https://issues.apache.org/jira/browse/HBASE-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368012#comment-15368012 ] Elliott Clark commented on HBASE-16196: --- bq.It seems worth a user@ message to gauge impact. Yeah let me do it. > Update jruby to a newer version. > > > Key: HBASE-16196 > URL: https://issues.apache.org/jira/browse/HBASE-16196 > Project: HBase > Issue Type: Bug > Components: dependencies, shell >Reporter: Elliott Clark >Assignee: Matt Mullins > Attachments: 0001-Update-to-JRuby-9.1.2.0-and-JLine-2.12.patch, > hbase-16196.branch-1.patch > > > Ruby 1.8.7 is no longer maintained. > The TTY library in the old jruby is bad. The newer one is less bad. > Since this is only a dependency on the hbase-shell module and not on > hbase-client or hbase-server this should be a pretty simple thing that > doesn't have any backwards compat issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16171) Fix the potential problems in TestHCM.testConnectionCloseAllowsInterrupt
[ https://issues.apache.org/jira/browse/HBASE-16171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16171: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.0.0 Status: Resolved (was: Patch Available) Pushed to master. Failures can't be releated. Thanks for the fix [~colin_mjj] > Fix the potential problems in TestHCM.testConnectionCloseAllowsInterrupt > > > Key: HBASE-16171 > URL: https://issues.apache.org/jira/browse/HBASE-16171 > Project: HBase > Issue Type: Bug >Reporter: Colin Ma >Assignee: Colin Ma > Fix For: 2.0.0 > > Attachments: HBASE-16171.001.patch, HBASE-16171.001.patch > > > TestHCM.testConnectionCloseAllowsInterrupt is not stable in QA runs, and > always failed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16195) Should not add chunk into chunkQueue if not using chunk pool in HeapMemStoreLAB
[ https://issues.apache.org/jira/browse/HBASE-16195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15368006#comment-15368006 ] Yu Li commented on HBASE-16195: --- I think if user explicitly changes this setting rather than using the default one, he/she should have known exactly what it means. Just like user could specify the memstore flush size to any value and we don't have any upper-limit there, I think it's also ok to have no limit here. Agree? > Should not add chunk into chunkQueue if not using chunk pool in > HeapMemStoreLAB > --- > > Key: HBASE-16195 > URL: https://issues.apache.org/jira/browse/HBASE-16195 > Project: HBase > Issue Type: Sub-task >Reporter: Yu Li >Assignee: Yu Li > Attachments: HBASE-16195.patch > > > Problem description and analysis please refer to HBASE-16193 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16196) Update jruby to a newer version.
[
https://issues.apache.org/jira/browse/HBASE-16196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367988#comment-15367988
]
Sean Busbey commented on HBASE-16196:
-
{quote}
We've had times where output on the shell is changed, arguments accepted are
changed, all in minor releases.
{quote}
That's true. I guess I'm just worried that this is much more likely to break a
given script built on top of the shell than an output or argument change would
be. It seems worth a user@ message to gauge impact.
> Update jruby to a newer version.
>
>
> Key: HBASE-16196
> URL: https://issues.apache.org/jira/browse/HBASE-16196
> Project: HBase
> Issue Type: Bug
> Components: dependencies, shell
>Reporter: Elliott Clark
>Assignee: Matt Mullins
> Attachments: 0001-Update-to-JRuby-9.1.2.0-and-JLine-2.12.patch,
> hbase-16196.branch-1.patch
>
>
> Ruby 1.8.7 is no longer maintained.
> The TTY library in the old jruby is bad. The newer one is less bad.
> Since this is only a dependency on the hbase-shell module and not on
> hbase-client or hbase-server this should be a pretty simple thing that
> doesn't have any backwards compat issues.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15925) compat-module maven variable not evaluated
[
https://issues.apache.org/jira/browse/HBASE-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367983#comment-15367983
]
David Portabella commented on HBASE-15925:
--
oh, i understand now.
thanks for the explanation. :)
> compat-module maven variable not evaluated
> --
>
> Key: HBASE-15925
> URL: https://issues.apache.org/jira/browse/HBASE-15925
> Project: HBase
> Issue Type: Bug
> Components: build
>Affects Versions: 1.0.0, 1.1.0, 1.2.0, 1.2.1, 1.0.3, 1.1.5
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6
>
> Attachments: HBASE-15925.1.patch
>
>
> Looks like we've regressed on HBASE-8488. Have a look at the dependency
> artifacts list on
> http://mvnrepository.com/artifact/org.apache.hbase/hbase-testing-util/1.2.1.
> Notice the direct dependency's artifactId is {{$\{compat.module\}}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
