[jira] [Comment Edited] (HBASE-6104) Require EXEC permission to call coprocessor endpoints
[ https://issues.apache.org/jira/browse/HBASE-6104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855469#comment-13855469 ] Andrew Purtell edited comment on HBASE-6104 at 12/23/13 7:56 AM: - bq. So this breaks the compatibility for endpoints when Security is in place. See the release note on this issue. Is it sufficient? We are not changing APIs, but yes if the AccessController is active then endpoint invocations require granting EXEC privilege in addition to what was set up previously for the user. Now that I'm thinking about it, let me add a unit test that insures EXEC if granted to a user for an entire namespace works as expected. Edit: Clarity was (Author: apurtell): bq. So this breaks the compatibility for endpoints when Security is in place. See the release note on this issue. Is it sufficient? We are not changing APIs, but yes if the AccessController is active than, for example, granting a user the ability to do a secure bulk load will require granting EXEC privilege as well as the others, because that is a coprocessor endpoint based service. Now that I'm thinking about it, let me add a unit test that insures EXEC if granted to a user for an entire namespace works as expected. > Require EXEC permission to call coprocessor endpoints > - > > Key: HBASE-6104 > URL: https://issues.apache.org/jira/browse/HBASE-6104 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors, security >Reporter: Gary Helmling >Assignee: Andrew Purtell > Fix For: 0.98.0, 0.99.0 > > Attachments: 6104.patch, 6104.patch, 6104.patch, 6104.patch > > > The EXEC action currently exists as only a placeholder in access control. It > should really be used to enforce access to coprocessor endpoint RPC calls, > which are currently unrestricted. > How the ACLs to support this would be modeled deserves some discussion: > * Should access be scoped to a specific table and CoprocessorProtocol > extension? > * Should it be possible to grant access to a CoprocessorProtocol > implementation globally (regardless of table)? > * Are per-method restrictions necessary? > * Should we expose hooks available to endpoint implementors so that they > could additionally apply their own permission checks? Some CP endpoints may > want to require READ permissions, others may want to enforce WRITE, or READ + > WRITE. > To apply these kinds of checks we would also have to extend the > RegionObserver interface to provide hooks wrapping HRegion.exec(). -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-9346) HBCK should provide an option to check if regions boundaries are the same in META and in stores.
[ https://issues.apache.org/jira/browse/HBASE-9346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855480#comment-13855480 ] Hudson commented on HBASE-9346: --- SUCCESS: Integrated in HBase-0.94-security #366 (See [https://builds.apache.org/job/HBase-0.94-security/366/]) HBASE-9346 HBCK should provide an option to check if regions boundaries are the same in META and in stores. (Jean Marc) (anoopsamjohn: rev 1553078) * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java > HBCK should provide an option to check if regions boundaries are the same in > META and in stores. > > > Key: HBASE-9346 > URL: https://issues.apache.org/jira/browse/HBASE-9346 > Project: HBase > Issue Type: Bug > Components: hbck, Operability >Affects Versions: 0.94.14, 0.98.1, 0.99.0, 0.96.1.1 >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari > Fix For: 0.98.0, 0.94.16, 0.96.2, 0.99.0 > > Attachments: HBASE-9346-v0-0.94.patch, HBASE-9346-v1-trunk.patch, > HBASE-9346-v10-trunk.patch, HBASE-9346-v2-94.patch, > HBASE-9346-v2-trunk.patch, HBASE-9346-v3-trunk.patch, > HBASE-9346-v4-trunk.patch, HBASE-9346-v5-trunk.patch, > HBASE-9346-v6-trunk.patch, HBASE-9346-v7-trunk.patch, > HBASE-9346-v8-trunk.patch, HBASE-9346-v9-trunk.patch > > > If META don't have the same region boundaries as the stores files, writes and > read might go to the wrong place. We need to provide a way to check that > withing HBCK. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-6104) Require EXEC permission to call coprocessor endpoints
[ https://issues.apache.org/jira/browse/HBASE-6104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855479#comment-13855479 ] ramkrishna.s.vasudevan commented on HBASE-6104: --- +1 on patch. Looks good to me. The release note also is good . > Require EXEC permission to call coprocessor endpoints > - > > Key: HBASE-6104 > URL: https://issues.apache.org/jira/browse/HBASE-6104 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors, security >Reporter: Gary Helmling >Assignee: Andrew Purtell > Fix For: 0.98.0, 0.99.0 > > Attachments: 6104.patch, 6104.patch, 6104.patch, 6104.patch > > > The EXEC action currently exists as only a placeholder in access control. It > should really be used to enforce access to coprocessor endpoint RPC calls, > which are currently unrestricted. > How the ACLs to support this would be modeled deserves some discussion: > * Should access be scoped to a specific table and CoprocessorProtocol > extension? > * Should it be possible to grant access to a CoprocessorProtocol > implementation globally (regardless of table)? > * Are per-method restrictions necessary? > * Should we expose hooks available to endpoint implementors so that they > could additionally apply their own permission checks? Some CP endpoints may > want to require READ permissions, others may want to enforce WRITE, or READ + > WRITE. > To apply these kinds of checks we would also have to extend the > RegionObserver interface to provide hooks wrapping HRegion.exec(). -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HBASE-10140) enable lazy seek for reverse scan
[ https://issues.apache.org/jira/browse/HBASE-10140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chunhui shen updated HBASE-10140: - Attachment: hbase-10140-trunkv2.patch Make the patch to enable lazy seek in reversed scan > enable lazy seek for reverse scan > - > > Key: HBASE-10140 > URL: https://issues.apache.org/jira/browse/HBASE-10140 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 0.99.0 >Reporter: Liang Xie > Attachments: 10140-v1.txt, hbase-10140-trunkv2.patch > > > will pick it up once have chance, if nobody do it. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HBASE-6104) Require EXEC permission to call coprocessor endpoints
[ https://issues.apache.org/jira/browse/HBASE-6104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-6104: -- Status: Open (was: Patch Available) > Require EXEC permission to call coprocessor endpoints > - > > Key: HBASE-6104 > URL: https://issues.apache.org/jira/browse/HBASE-6104 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors, security >Reporter: Gary Helmling >Assignee: Andrew Purtell > Fix For: 0.98.0, 0.99.0 > > Attachments: 6104.patch, 6104.patch, 6104.patch, 6104.patch > > > The EXEC action currently exists as only a placeholder in access control. It > should really be used to enforce access to coprocessor endpoint RPC calls, > which are currently unrestricted. > How the ACLs to support this would be modeled deserves some discussion: > * Should access be scoped to a specific table and CoprocessorProtocol > extension? > * Should it be possible to grant access to a CoprocessorProtocol > implementation globally (regardless of table)? > * Are per-method restrictions necessary? > * Should we expose hooks available to endpoint implementors so that they > could additionally apply their own permission checks? Some CP endpoints may > want to require READ permissions, others may want to enforce WRITE, or READ + > WRITE. > To apply these kinds of checks we would also have to extend the > RegionObserver interface to provide hooks wrapping HRegion.exec(). -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Comment Edited] (HBASE-6104) Require EXEC permission to call coprocessor endpoints
[ https://issues.apache.org/jira/browse/HBASE-6104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855469#comment-13855469 ] Andrew Purtell edited comment on HBASE-6104 at 12/23/13 7:36 AM: - bq. So this breaks the compatibility for endpoints when Security is in place. See the release note on this issue. Is it sufficient? We are not changing APIs, but yes if the AccessController is active than, for example, granting a user the ability to do a secure bulk load will require granting EXEC privilege as well as the others, because that is a coprocessor endpoint based service. Now that I'm thinking about it, let me add a unit test that insures EXEC if granted to a user for an entire namespace works as expected. was (Author: apurtell): bq. So this breaks the compatibility for endpoints when Security is in place. See the release note on this issue. Is it sufficient? We are not changing APIs, but yes if the AccessController is active than, for example, granting a user the ability to do a secure bulk load will require granting EXEC privilege as well as the others, because that is a coprocessor endpoint based service. Now that I'm thinking about it, let me add a unit test that insures EXEC if granted to an entire namespace works as expected. > Require EXEC permission to call coprocessor endpoints > - > > Key: HBASE-6104 > URL: https://issues.apache.org/jira/browse/HBASE-6104 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors, security >Reporter: Gary Helmling >Assignee: Andrew Purtell > Fix For: 0.98.0, 0.99.0 > > Attachments: 6104.patch, 6104.patch, 6104.patch, 6104.patch > > > The EXEC action currently exists as only a placeholder in access control. It > should really be used to enforce access to coprocessor endpoint RPC calls, > which are currently unrestricted. > How the ACLs to support this would be modeled deserves some discussion: > * Should access be scoped to a specific table and CoprocessorProtocol > extension? > * Should it be possible to grant access to a CoprocessorProtocol > implementation globally (regardless of table)? > * Are per-method restrictions necessary? > * Should we expose hooks available to endpoint implementors so that they > could additionally apply their own permission checks? Some CP endpoints may > want to require READ permissions, others may want to enforce WRITE, or READ + > WRITE. > To apply these kinds of checks we would also have to extend the > RegionObserver interface to provide hooks wrapping HRegion.exec(). -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10212) New rpc metric: number of active handler
[ https://issues.apache.org/jira/browse/HBASE-10212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855468#comment-13855468 ] Liang Xie commented on HBASE-10212: --- The attached file could apply to 0.94 branch directly w/o any change:) +1 > New rpc metric: number of active handler > > > Key: HBASE-10212 > URL: https://issues.apache.org/jira/browse/HBASE-10212 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC >Reporter: Chao Shi >Assignee: Chao Shi > Attachments: hbase-10212.patch > > > The attached patch adds a new metric: number of active handler threads. We > found this is a good metric to measure how busy of a server. If this number > is too high (compared to the total number of handlers), the server has risks > in getting call queue full. > We used to monitor # reads or # writes. However we found this often produce > false alerts, because a read touching HDFS will produce much high workload > than a block-cached read. > The attached patch is based on our internal 0.94 branch, but I think it > pretty easy to port to rebase to other branches if you think it is useful. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-6104) Require EXEC permission to call coprocessor endpoints
[ https://issues.apache.org/jira/browse/HBASE-6104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855469#comment-13855469 ] Andrew Purtell commented on HBASE-6104: --- bq. So this breaks the compatibility for endpoints when Security is in place. See the release note on this issue. Is it sufficient? We are not changing APIs, but yes if the AccessController is active than, for example, granting a user the ability to do a secure bulk load will require granting EXEC privilege as well as the others, because that is a coprocessor endpoint based service. Now that I'm thinking about it, let me add a unit test that insures EXEC if granted to an entire namespace works as expected. > Require EXEC permission to call coprocessor endpoints > - > > Key: HBASE-6104 > URL: https://issues.apache.org/jira/browse/HBASE-6104 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors, security >Reporter: Gary Helmling >Assignee: Andrew Purtell > Fix For: 0.98.0, 0.99.0 > > Attachments: 6104.patch, 6104.patch, 6104.patch, 6104.patch > > > The EXEC action currently exists as only a placeholder in access control. It > should really be used to enforce access to coprocessor endpoint RPC calls, > which are currently unrestricted. > How the ACLs to support this would be modeled deserves some discussion: > * Should access be scoped to a specific table and CoprocessorProtocol > extension? > * Should it be possible to grant access to a CoprocessorProtocol > implementation globally (regardless of table)? > * Are per-method restrictions necessary? > * Should we expose hooks available to endpoint implementors so that they > could additionally apply their own permission checks? Some CP endpoints may > want to require READ permissions, others may want to enforce WRITE, or READ + > WRITE. > To apply these kinds of checks we would also have to extend the > RegionObserver interface to provide hooks wrapping HRegion.exec(). -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-6104) Require EXEC permission to call coprocessor endpoints
[ https://issues.apache.org/jira/browse/HBASE-6104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855461#comment-13855461 ] Anoop Sam John commented on HBASE-6104: --- +1 So this breaks the compatability for endpoints when Security is in place. Right Andy? We can highlight this. (You will make some changes in book?) > Require EXEC permission to call coprocessor endpoints > - > > Key: HBASE-6104 > URL: https://issues.apache.org/jira/browse/HBASE-6104 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors, security >Reporter: Gary Helmling >Assignee: Andrew Purtell > Fix For: 0.98.0, 0.99.0 > > Attachments: 6104.patch, 6104.patch, 6104.patch, 6104.patch > > > The EXEC action currently exists as only a placeholder in access control. It > should really be used to enforce access to coprocessor endpoint RPC calls, > which are currently unrestricted. > How the ACLs to support this would be modeled deserves some discussion: > * Should access be scoped to a specific table and CoprocessorProtocol > extension? > * Should it be possible to grant access to a CoprocessorProtocol > implementation globally (regardless of table)? > * Are per-method restrictions necessary? > * Should we expose hooks available to endpoint implementors so that they > could additionally apply their own permission checks? Some CP endpoints may > want to require READ permissions, others may want to enforce WRITE, or READ + > WRITE. > To apply these kinds of checks we would also have to extend the > RegionObserver interface to provide hooks wrapping HRegion.exec(). -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10225) Bug in calls to RegionObsever.postScannerFilterRow
[
https://issues.apache.org/jira/browse/HBASE-10225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855455#comment-13855455
]
Andrew Purtell commented on HBASE-10225:
+1 on 0.98.
> Bug in calls to RegionObsever.postScannerFilterRow
> --
>
> Key: HBASE-10225
> URL: https://issues.apache.org/jira/browse/HBASE-10225
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
> Fix For: 0.98.0, 0.94.16, 0.96.2, 0.99.0
>
> Attachments: 10225-0.94.txt, HBASE-10225.patch
>
>
> Just noticed that while looking at HBASE-10047.
> In 0.94 (and presumably in trunk, will check later) we have this:
> {code}
> protected boolean nextRow(byte [] currentRow, int offset, short length)
> throws IOException {
> ...
> if (this.region.getCoprocessorHost() != null) {
> return this.region.getCoprocessorHost().postScannerFilterRow(this,
> currentRow);
> }
> return true;
> }
> {code}
> Notice how we only pass currentRow into the coprocessor, but not offset and
> length. Anything using this hook currently is 100% broken. The hook was added
> in 0.94.5 (HBASE-5664), it never worked correctly.
> [~anoopsamjohn], you had added the hook. Do you still need it?
> We can either remove it (I'd prefer that in the light of the performance
> issued observed in HBASE-10047, we can leave the stub in BaseRegionObserver
> in 0.94, but document that it is no-op), or we'd have to have change its
> signature to be able to pass offset and length as well.
> Since nobody noticed nobody is using this hook currently, so both should be
> valid options.
> (Making a new standalone copy of the rowkey just to pass into this method
> absolutely out of the question for performance reasons).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10225) Bug in calls to RegionObsever.postScannerFilterRow
[
https://issues.apache.org/jira/browse/HBASE-10225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855456#comment-13855456
]
Andrew Purtell commented on HBASE-10225:
[~stack] - I think this is fine for 0.96 also, obviously nobody is using it
(yet).
> Bug in calls to RegionObsever.postScannerFilterRow
> --
>
> Key: HBASE-10225
> URL: https://issues.apache.org/jira/browse/HBASE-10225
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
> Fix For: 0.98.0, 0.94.16, 0.96.2, 0.99.0
>
> Attachments: 10225-0.94.txt, HBASE-10225.patch
>
>
> Just noticed that while looking at HBASE-10047.
> In 0.94 (and presumably in trunk, will check later) we have this:
> {code}
> protected boolean nextRow(byte [] currentRow, int offset, short length)
> throws IOException {
> ...
> if (this.region.getCoprocessorHost() != null) {
> return this.region.getCoprocessorHost().postScannerFilterRow(this,
> currentRow);
> }
> return true;
> }
> {code}
> Notice how we only pass currentRow into the coprocessor, but not offset and
> length. Anything using this hook currently is 100% broken. The hook was added
> in 0.94.5 (HBASE-5664), it never worked correctly.
> [~anoopsamjohn], you had added the hook. Do you still need it?
> We can either remove it (I'd prefer that in the light of the performance
> issued observed in HBASE-10047, we can leave the stub in BaseRegionObserver
> in 0.94, but document that it is no-op), or we'd have to have change its
> signature to be able to pass offset and length as well.
> Since nobody noticed nobody is using this hook currently, so both should be
> valid options.
> (Making a new standalone copy of the rowkey just to pass into this method
> absolutely out of the question for performance reasons).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10225) Bug in calls to RegionObsever.postScannerFilterRow
[
https://issues.apache.org/jira/browse/HBASE-10225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855454#comment-13855454
]
Anoop Sam John commented on HBASE-10225:
Committed to 0.98 and trunk.
Leaving it open for the decision on 96 and 94 branches.
> Bug in calls to RegionObsever.postScannerFilterRow
> --
>
> Key: HBASE-10225
> URL: https://issues.apache.org/jira/browse/HBASE-10225
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
> Fix For: 0.98.0, 0.94.16, 0.96.2, 0.99.0
>
> Attachments: 10225-0.94.txt, HBASE-10225.patch
>
>
> Just noticed that while looking at HBASE-10047.
> In 0.94 (and presumably in trunk, will check later) we have this:
> {code}
> protected boolean nextRow(byte [] currentRow, int offset, short length)
> throws IOException {
> ...
> if (this.region.getCoprocessorHost() != null) {
> return this.region.getCoprocessorHost().postScannerFilterRow(this,
> currentRow);
> }
> return true;
> }
> {code}
> Notice how we only pass currentRow into the coprocessor, but not offset and
> length. Anything using this hook currently is 100% broken. The hook was added
> in 0.94.5 (HBASE-5664), it never worked correctly.
> [~anoopsamjohn], you had added the hook. Do you still need it?
> We can either remove it (I'd prefer that in the light of the performance
> issued observed in HBASE-10047, we can leave the stub in BaseRegionObserver
> in 0.94, but document that it is no-op), or we'd have to have change its
> signature to be able to pass offset and length as well.
> Since nobody noticed nobody is using this hook currently, so both should be
> valid options.
> (Making a new standalone copy of the rowkey just to pass into this method
> absolutely out of the question for performance reasons).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-6104) Require EXEC permission to call coprocessor endpoints
[ https://issues.apache.org/jira/browse/HBASE-6104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855450#comment-13855450 ] Andrew Purtell commented on HBASE-6104: --- Significant functional change - can I get another review / +1: [~stack], [~anoop.hbase], [~ram_krish] ? > Require EXEC permission to call coprocessor endpoints > - > > Key: HBASE-6104 > URL: https://issues.apache.org/jira/browse/HBASE-6104 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors, security >Reporter: Gary Helmling >Assignee: Andrew Purtell > Fix For: 0.98.0, 0.99.0 > > Attachments: 6104.patch, 6104.patch, 6104.patch, 6104.patch > > > The EXEC action currently exists as only a placeholder in access control. It > should really be used to enforce access to coprocessor endpoint RPC calls, > which are currently unrestricted. > How the ACLs to support this would be modeled deserves some discussion: > * Should access be scoped to a specific table and CoprocessorProtocol > extension? > * Should it be possible to grant access to a CoprocessorProtocol > implementation globally (regardless of table)? > * Are per-method restrictions necessary? > * Should we expose hooks available to endpoint implementors so that they > could additionally apply their own permission checks? Some CP endpoints may > want to require READ permissions, others may want to enforce WRITE, or READ + > WRITE. > To apply these kinds of checks we would also have to extend the > RegionObserver interface to provide hooks wrapping HRegion.exec(). -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HBASE-10225) Bug in calls to RegionObsever.postScannerFilterRow
[
https://issues.apache.org/jira/browse/HBASE-10225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anoop Sam John updated HBASE-10225:
---
Attachment: HBASE-10225.patch
Reattaching trunk patch to get a QA result.
> Bug in calls to RegionObsever.postScannerFilterRow
> --
>
> Key: HBASE-10225
> URL: https://issues.apache.org/jira/browse/HBASE-10225
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
> Fix For: 0.98.0, 0.94.16, 0.96.2, 0.99.0
>
> Attachments: 10225-0.94.txt, HBASE-10225.patch
>
>
> Just noticed that while looking at HBASE-10047.
> In 0.94 (and presumably in trunk, will check later) we have this:
> {code}
> protected boolean nextRow(byte [] currentRow, int offset, short length)
> throws IOException {
> ...
> if (this.region.getCoprocessorHost() != null) {
> return this.region.getCoprocessorHost().postScannerFilterRow(this,
> currentRow);
> }
> return true;
> }
> {code}
> Notice how we only pass currentRow into the coprocessor, but not offset and
> length. Anything using this hook currently is 100% broken. The hook was added
> in 0.94.5 (HBASE-5664), it never worked correctly.
> [~anoopsamjohn], you had added the hook. Do you still need it?
> We can either remove it (I'd prefer that in the light of the performance
> issued observed in HBASE-10047, we can leave the stub in BaseRegionObserver
> in 0.94, but document that it is no-op), or we'd have to have change its
> signature to be able to pass offset and length as well.
> Since nobody noticed nobody is using this hook currently, so both should be
> valid options.
> (Making a new standalone copy of the rowkey just to pass into this method
> absolutely out of the question for performance reasons).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10225) Bug in calls to RegionObsever.postScannerFilterRow
[
https://issues.apache.org/jira/browse/HBASE-10225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855451#comment-13855451
]
Anoop Sam John commented on HBASE-10225:
Oh by that time we got a result :) Good.. I will commit to 98 and trunk.
> Bug in calls to RegionObsever.postScannerFilterRow
> --
>
> Key: HBASE-10225
> URL: https://issues.apache.org/jira/browse/HBASE-10225
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
> Fix For: 0.98.0, 0.94.16, 0.96.2, 0.99.0
>
> Attachments: 10225-0.94.txt, HBASE-10225.patch
>
>
> Just noticed that while looking at HBASE-10047.
> In 0.94 (and presumably in trunk, will check later) we have this:
> {code}
> protected boolean nextRow(byte [] currentRow, int offset, short length)
> throws IOException {
> ...
> if (this.region.getCoprocessorHost() != null) {
> return this.region.getCoprocessorHost().postScannerFilterRow(this,
> currentRow);
> }
> return true;
> }
> {code}
> Notice how we only pass currentRow into the coprocessor, but not offset and
> length. Anything using this hook currently is 100% broken. The hook was added
> in 0.94.5 (HBASE-5664), it never worked correctly.
> [~anoopsamjohn], you had added the hook. Do you still need it?
> We can either remove it (I'd prefer that in the light of the performance
> issued observed in HBASE-10047, we can leave the stub in BaseRegionObserver
> in 0.94, but document that it is no-op), or we'd have to have change its
> signature to be able to pass offset and length as well.
> Since nobody noticed nobody is using this hook currently, so both should be
> valid options.
> (Making a new standalone copy of the rowkey just to pass into this method
> absolutely out of the question for performance reasons).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-10225) Bug in calls to RegionObsever.postScannerFilterRow
[
https://issues.apache.org/jira/browse/HBASE-10225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anoop Sam John updated HBASE-10225:
---
Attachment: (was: HBASE-10225.patch)
> Bug in calls to RegionObsever.postScannerFilterRow
> --
>
> Key: HBASE-10225
> URL: https://issues.apache.org/jira/browse/HBASE-10225
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
> Fix For: 0.98.0, 0.94.16, 0.96.2, 0.99.0
>
> Attachments: 10225-0.94.txt
>
>
> Just noticed that while looking at HBASE-10047.
> In 0.94 (and presumably in trunk, will check later) we have this:
> {code}
> protected boolean nextRow(byte [] currentRow, int offset, short length)
> throws IOException {
> ...
> if (this.region.getCoprocessorHost() != null) {
> return this.region.getCoprocessorHost().postScannerFilterRow(this,
> currentRow);
> }
> return true;
> }
> {code}
> Notice how we only pass currentRow into the coprocessor, but not offset and
> length. Anything using this hook currently is 100% broken. The hook was added
> in 0.94.5 (HBASE-5664), it never worked correctly.
> [~anoopsamjohn], you had added the hook. Do you still need it?
> We can either remove it (I'd prefer that in the light of the performance
> issued observed in HBASE-10047, we can leave the stub in BaseRegionObserver
> in 0.94, but document that it is no-op), or we'd have to have change its
> signature to be able to pass offset and length as well.
> Since nobody noticed nobody is using this hook currently, so both should be
> valid options.
> (Making a new standalone copy of the rowkey just to pass into this method
> absolutely out of the question for performance reasons).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10225) Bug in calls to RegionObsever.postScannerFilterRow
[
https://issues.apache.org/jira/browse/HBASE-10225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855448#comment-13855448
]
Hadoop QA commented on HBASE-10225:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12620130/HBASE-10225.patch
against trunk revision .
ATTACHMENT ID: 12620130
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop
1.1 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8263//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8263//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8263//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8263//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8263//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8263//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8263//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8263//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8263//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8263//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8263//console
This message is automatically generated.
> Bug in calls to RegionObsever.postScannerFilterRow
> --
>
> Key: HBASE-10225
> URL: https://issues.apache.org/jira/browse/HBASE-10225
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
> Fix For: 0.98.0, 0.94.16, 0.96.2, 0.99.0
>
> Attachments: 10225-0.94.txt
>
>
> Just noticed that while looking at HBASE-10047.
> In 0.94 (and presumably in trunk, will check later) we have this:
> {code}
> protected boolean nextRow(byte [] currentRow, int offset, short length)
> throws IOException {
> ...
> if (this.region.getCoprocessorHost() != null) {
> return this.region.getCoprocessorHost().postScannerFilterRow(this,
> currentRow);
> }
> return true;
> }
> {code}
> Notice how we only pass currentRow into the coprocessor, but not offset and
> length. Anything using this hook currently is 100% broken. The hook was added
> in 0.94.5 (HBASE-5664), it never worked correctly.
> [~anoopsamjohn], you had added the hook. Do you still need it?
> We can either remove it (I'd prefer that in the light of the performance
> issued observed in HBASE-10047, we can leave the stub in BaseRegionObserver
> in 0.94, but document that it is no-op), or we'd have to have change its
> signature to be able to pass offset and length as well.
> Since nobody noticed nobody is using this hook currently, so both should be
> valid options.
> (Making a new standalone copy of the rowkey just to pass into this method
> absolutely out of the question for performance reasons).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10225) Bug in calls to RegionObsever.postScannerFilterRow
[
https://issues.apache.org/jira/browse/HBASE-10225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855445#comment-13855445
]
rajeshbabu commented on HBASE-10225:
I am +1 on correcting the signature. Patch lgtm.
> Bug in calls to RegionObsever.postScannerFilterRow
> --
>
> Key: HBASE-10225
> URL: https://issues.apache.org/jira/browse/HBASE-10225
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
> Fix For: 0.98.0, 0.94.16, 0.96.2, 0.99.0
>
> Attachments: 10225-0.94.txt, HBASE-10225.patch
>
>
> Just noticed that while looking at HBASE-10047.
> In 0.94 (and presumably in trunk, will check later) we have this:
> {code}
> protected boolean nextRow(byte [] currentRow, int offset, short length)
> throws IOException {
> ...
> if (this.region.getCoprocessorHost() != null) {
> return this.region.getCoprocessorHost().postScannerFilterRow(this,
> currentRow);
> }
> return true;
> }
> {code}
> Notice how we only pass currentRow into the coprocessor, but not offset and
> length. Anything using this hook currently is 100% broken. The hook was added
> in 0.94.5 (HBASE-5664), it never worked correctly.
> [~anoopsamjohn], you had added the hook. Do you still need it?
> We can either remove it (I'd prefer that in the light of the performance
> issued observed in HBASE-10047, we can leave the stub in BaseRegionObserver
> in 0.94, but document that it is no-op), or we'd have to have change its
> signature to be able to pass offset and length as well.
> Since nobody noticed nobody is using this hook currently, so both should be
> valid options.
> (Making a new standalone copy of the rowkey just to pass into this method
> absolutely out of the question for performance reasons).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-9346) HBCK should provide an option to check if regions boundaries are the same in META and in stores.
[ https://issues.apache.org/jira/browse/HBASE-9346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-9346: -- Resolution: Fixed Fix Version/s: 0.99.0 0.96.2 0.94.16 0.98.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to 94, 96, 98 and trunk. Thanks for the work JMS. Thanks all for the reviews. > HBCK should provide an option to check if regions boundaries are the same in > META and in stores. > > > Key: HBASE-9346 > URL: https://issues.apache.org/jira/browse/HBASE-9346 > Project: HBase > Issue Type: Bug > Components: hbck, Operability >Affects Versions: 0.94.14, 0.98.1, 0.99.0, 0.96.1.1 >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari > Fix For: 0.98.0, 0.94.16, 0.96.2, 0.99.0 > > Attachments: HBASE-9346-v0-0.94.patch, HBASE-9346-v1-trunk.patch, > HBASE-9346-v10-trunk.patch, HBASE-9346-v2-94.patch, > HBASE-9346-v2-trunk.patch, HBASE-9346-v3-trunk.patch, > HBASE-9346-v4-trunk.patch, HBASE-9346-v5-trunk.patch, > HBASE-9346-v6-trunk.patch, HBASE-9346-v7-trunk.patch, > HBASE-9346-v8-trunk.patch, HBASE-9346-v9-trunk.patch > > > If META don't have the same region boundaries as the stores files, writes and > read might go to the wrong place. We need to provide a way to check that > withing HBCK. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HBASE-9346) HBCK should provide an option to check if regions boundaries are the same in META and in stores.
[ https://issues.apache.org/jira/browse/HBASE-9346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-9346: -- Attachment: HBASE-9346-v2-94.patch HBASE-9346-v10-trunk.patch Versions I am going to commit. Addressed you comment also in this Liang > HBCK should provide an option to check if regions boundaries are the same in > META and in stores. > > > Key: HBASE-9346 > URL: https://issues.apache.org/jira/browse/HBASE-9346 > Project: HBase > Issue Type: Bug > Components: hbck, Operability >Affects Versions: 0.94.14, 0.98.1, 0.99.0, 0.96.1.1 >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari > Attachments: HBASE-9346-v0-0.94.patch, HBASE-9346-v1-trunk.patch, > HBASE-9346-v10-trunk.patch, HBASE-9346-v2-94.patch, > HBASE-9346-v2-trunk.patch, HBASE-9346-v3-trunk.patch, > HBASE-9346-v4-trunk.patch, HBASE-9346-v5-trunk.patch, > HBASE-9346-v6-trunk.patch, HBASE-9346-v7-trunk.patch, > HBASE-9346-v8-trunk.patch, HBASE-9346-v9-trunk.patch > > > If META don't have the same region boundaries as the stores files, writes and > read might go to the wrong place. We need to provide a way to check that > withing HBCK. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Comment Edited] (HBASE-9346) HBCK should provide an option to check if regions boundaries are the same in META and in stores.
[
https://issues.apache.org/jira/browse/HBASE-9346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855342#comment-13855342
]
Anoop Sam John edited comment on HBASE-9346 at 12/23/13 6:42 AM:
-
Oh... When u fixed the nested if the condition check become old one
again..(which was wrong)
{code}
if ((reader.getFirstKey() != null)
+ && ((storeFirstKey == null) ||
(comparator.compare(storeFirstKey,
+reader.getFirstKey()) < 0))) {
+storeFirstKey = reader.getFirstKey();
+ }
{code}
It should be >0
Can correct on commit
was (Author: anoop.hbase):
Oh... When u fixed the nested if the condition check become old one
again..(which was wrong)
{code}
if ((reader.getFirstKey() != null)
+ && ((storeFirstKey == null) ||
(comparator.compare(storeFirstKey,
+reader.getFirstKey()) < 0))) {
+storeFirstKey = reader.getFirstKey();
+ }
{code}
It should be <0
Can correct on commit
> HBCK should provide an option to check if regions boundaries are the same in
> META and in stores.
>
>
> Key: HBASE-9346
> URL: https://issues.apache.org/jira/browse/HBASE-9346
> Project: HBase
> Issue Type: Bug
> Components: hbck, Operability
>Affects Versions: 0.94.14, 0.98.1, 0.99.0, 0.96.1.1
>Reporter: Jean-Marc Spaggiari
>Assignee: Jean-Marc Spaggiari
> Attachments: HBASE-9346-v0-0.94.patch, HBASE-9346-v1-trunk.patch,
> HBASE-9346-v2-trunk.patch, HBASE-9346-v3-trunk.patch,
> HBASE-9346-v4-trunk.patch, HBASE-9346-v5-trunk.patch,
> HBASE-9346-v6-trunk.patch, HBASE-9346-v7-trunk.patch,
> HBASE-9346-v8-trunk.patch, HBASE-9346-v9-trunk.patch
>
>
> If META don't have the same region boundaries as the stores files, writes and
> read might go to the wrong place. We need to provide a way to check that
> withing HBCK.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10225) Bug in calls to RegionObsever.postScannerFilterRow
[
https://issues.apache.org/jira/browse/HBASE-10225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855439#comment-13855439
]
Hadoop QA commented on HBASE-10225:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12620138/10225-0.94.txt
against trunk revision .
ATTACHMENT ID: 12620138
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8264//console
This message is automatically generated.
> Bug in calls to RegionObsever.postScannerFilterRow
> --
>
> Key: HBASE-10225
> URL: https://issues.apache.org/jira/browse/HBASE-10225
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
> Fix For: 0.98.0, 0.94.16, 0.96.2, 0.99.0
>
> Attachments: 10225-0.94.txt, HBASE-10225.patch
>
>
> Just noticed that while looking at HBASE-10047.
> In 0.94 (and presumably in trunk, will check later) we have this:
> {code}
> protected boolean nextRow(byte [] currentRow, int offset, short length)
> throws IOException {
> ...
> if (this.region.getCoprocessorHost() != null) {
> return this.region.getCoprocessorHost().postScannerFilterRow(this,
> currentRow);
> }
> return true;
> }
> {code}
> Notice how we only pass currentRow into the coprocessor, but not offset and
> length. Anything using this hook currently is 100% broken. The hook was added
> in 0.94.5 (HBASE-5664), it never worked correctly.
> [~anoopsamjohn], you had added the hook. Do you still need it?
> We can either remove it (I'd prefer that in the light of the performance
> issued observed in HBASE-10047, we can leave the stub in BaseRegionObserver
> in 0.94, but document that it is no-op), or we'd have to have change its
> signature to be able to pass offset and length as well.
> Since nobody noticed nobody is using this hook currently, so both should be
> valid options.
> (Making a new standalone copy of the rowkey just to pass into this method
> absolutely out of the question for performance reasons).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10225) Bug in calls to RegionObsever.postScannerFilterRow
[
https://issues.apache.org/jira/browse/HBASE-10225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855434#comment-13855434
]
Lars Hofhansl commented on HBASE-10225:
---
I agree this should be fixed or removed. The risk of this causing issues is
low. A worry was coprocessor jars somehow shipping a copy of the
BaseRegionObserver, but since Phoenix still works fine - and there probably
isn't a more heavy user of coprocessors ATM, I'm +1 on 0.94 for this.
> Bug in calls to RegionObsever.postScannerFilterRow
> --
>
> Key: HBASE-10225
> URL: https://issues.apache.org/jira/browse/HBASE-10225
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
> Fix For: 0.98.0, 0.94.16, 0.96.2, 0.99.0
>
> Attachments: 10225-0.94.txt, HBASE-10225.patch
>
>
> Just noticed that while looking at HBASE-10047.
> In 0.94 (and presumably in trunk, will check later) we have this:
> {code}
> protected boolean nextRow(byte [] currentRow, int offset, short length)
> throws IOException {
> ...
> if (this.region.getCoprocessorHost() != null) {
> return this.region.getCoprocessorHost().postScannerFilterRow(this,
> currentRow);
> }
> return true;
> }
> {code}
> Notice how we only pass currentRow into the coprocessor, but not offset and
> length. Anything using this hook currently is 100% broken. The hook was added
> in 0.94.5 (HBASE-5664), it never worked correctly.
> [~anoopsamjohn], you had added the hook. Do you still need it?
> We can either remove it (I'd prefer that in the light of the performance
> issued observed in HBASE-10047, we can leave the stub in BaseRegionObserver
> in 0.94, but document that it is no-op), or we'd have to have change its
> signature to be able to pass offset and length as well.
> Since nobody noticed nobody is using this hook currently, so both should be
> valid options.
> (Making a new standalone copy of the rowkey just to pass into this method
> absolutely out of the question for performance reasons).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10225) Bug in calls to RegionObsever.postScannerFilterRow
[
https://issues.apache.org/jira/browse/HBASE-10225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855432#comment-13855432
]
Anoop Sam John commented on HBASE-10225:
I would vote for maintaining this with correcting the signature in 94 and 96.
As you said I firmly belive no one else using this. Else this could have come
up time back. Any way waiting for [~stack]'s call on this. :)
> Bug in calls to RegionObsever.postScannerFilterRow
> --
>
> Key: HBASE-10225
> URL: https://issues.apache.org/jira/browse/HBASE-10225
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
> Fix For: 0.98.0, 0.94.16, 0.96.2, 0.99.0
>
> Attachments: 10225-0.94.txt, HBASE-10225.patch
>
>
> Just noticed that while looking at HBASE-10047.
> In 0.94 (and presumably in trunk, will check later) we have this:
> {code}
> protected boolean nextRow(byte [] currentRow, int offset, short length)
> throws IOException {
> ...
> if (this.region.getCoprocessorHost() != null) {
> return this.region.getCoprocessorHost().postScannerFilterRow(this,
> currentRow);
> }
> return true;
> }
> {code}
> Notice how we only pass currentRow into the coprocessor, but not offset and
> length. Anything using this hook currently is 100% broken. The hook was added
> in 0.94.5 (HBASE-5664), it never worked correctly.
> [~anoopsamjohn], you had added the hook. Do you still need it?
> We can either remove it (I'd prefer that in the light of the performance
> issued observed in HBASE-10047, we can leave the stub in BaseRegionObserver
> in 0.94, but document that it is no-op), or we'd have to have change its
> signature to be able to pass offset and length as well.
> Since nobody noticed nobody is using this hook currently, so both should be
> valid options.
> (Making a new standalone copy of the rowkey just to pass into this method
> absolutely out of the question for performance reasons).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-9346) HBCK should provide an option to check if regions boundaries are the same in META and in stores.
[
https://issues.apache.org/jira/browse/HBASE-9346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855430#comment-13855430
]
Liang Xie commented on HBASE-9346:
--
Good stuff.
Just a nit:
{code}
+} catch (IOException e) {
+ e.printStackTrace();
+}
{code}
Mind changing into log statement? thanks:)
> HBCK should provide an option to check if regions boundaries are the same in
> META and in stores.
>
>
> Key: HBASE-9346
> URL: https://issues.apache.org/jira/browse/HBASE-9346
> Project: HBase
> Issue Type: Bug
> Components: hbck, Operability
>Affects Versions: 0.94.14, 0.98.1, 0.99.0, 0.96.1.1
>Reporter: Jean-Marc Spaggiari
>Assignee: Jean-Marc Spaggiari
> Attachments: HBASE-9346-v0-0.94.patch, HBASE-9346-v1-trunk.patch,
> HBASE-9346-v2-trunk.patch, HBASE-9346-v3-trunk.patch,
> HBASE-9346-v4-trunk.patch, HBASE-9346-v5-trunk.patch,
> HBASE-9346-v6-trunk.patch, HBASE-9346-v7-trunk.patch,
> HBASE-9346-v8-trunk.patch, HBASE-9346-v9-trunk.patch
>
>
> If META don't have the same region boundaries as the stores files, writes and
> read might go to the wrong place. We need to provide a way to check that
> withing HBCK.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-10225) Bug in calls to RegionObsever.postScannerFilterRow
[
https://issues.apache.org/jira/browse/HBASE-10225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-10225:
--
Attachment: 10225-0.94.txt
Here's a 0.94 patch.
(HadoopQA has already started).
I tried a patched 0.94 HBase with an old Phoenix jar and client, works fine.
> Bug in calls to RegionObsever.postScannerFilterRow
> --
>
> Key: HBASE-10225
> URL: https://issues.apache.org/jira/browse/HBASE-10225
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
> Fix For: 0.98.0, 0.94.16, 0.96.2, 0.99.0
>
> Attachments: 10225-0.94.txt, HBASE-10225.patch
>
>
> Just noticed that while looking at HBASE-10047.
> In 0.94 (and presumably in trunk, will check later) we have this:
> {code}
> protected boolean nextRow(byte [] currentRow, int offset, short length)
> throws IOException {
> ...
> if (this.region.getCoprocessorHost() != null) {
> return this.region.getCoprocessorHost().postScannerFilterRow(this,
> currentRow);
> }
> return true;
> }
> {code}
> Notice how we only pass currentRow into the coprocessor, but not offset and
> length. Anything using this hook currently is 100% broken. The hook was added
> in 0.94.5 (HBASE-5664), it never worked correctly.
> [~anoopsamjohn], you had added the hook. Do you still need it?
> We can either remove it (I'd prefer that in the light of the performance
> issued observed in HBASE-10047, we can leave the stub in BaseRegionObserver
> in 0.94, but document that it is no-op), or we'd have to have change its
> signature to be able to pass offset and length as well.
> Since nobody noticed nobody is using this hook currently, so both should be
> valid options.
> (Making a new standalone copy of the rowkey just to pass into this method
> absolutely out of the question for performance reasons).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10225) Bug in calls to RegionObsever.postScannerFilterRow
[
https://issues.apache.org/jira/browse/HBASE-10225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855423#comment-13855423
]
Lars Hofhansl commented on HBASE-10225:
---
+1 on patch for trunk and 0.98.
For 0.94, 0.96 should we keep the old method in BaseRegionObserver, and just
not call it, for (perhaps?) better binary compatibility? [~stack]?
> Bug in calls to RegionObsever.postScannerFilterRow
> --
>
> Key: HBASE-10225
> URL: https://issues.apache.org/jira/browse/HBASE-10225
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
> Fix For: 0.98.0, 0.94.16, 0.96.2, 0.99.0
>
> Attachments: HBASE-10225.patch
>
>
> Just noticed that while looking at HBASE-10047.
> In 0.94 (and presumably in trunk, will check later) we have this:
> {code}
> protected boolean nextRow(byte [] currentRow, int offset, short length)
> throws IOException {
> ...
> if (this.region.getCoprocessorHost() != null) {
> return this.region.getCoprocessorHost().postScannerFilterRow(this,
> currentRow);
> }
> return true;
> }
> {code}
> Notice how we only pass currentRow into the coprocessor, but not offset and
> length. Anything using this hook currently is 100% broken. The hook was added
> in 0.94.5 (HBASE-5664), it never worked correctly.
> [~anoopsamjohn], you had added the hook. Do you still need it?
> We can either remove it (I'd prefer that in the light of the performance
> issued observed in HBASE-10047, we can leave the stub in BaseRegionObserver
> in 0.94, but document that it is no-op), or we'd have to have change its
> signature to be able to pass offset and length as well.
> Since nobody noticed nobody is using this hook currently, so both should be
> valid options.
> (Making a new standalone copy of the rowkey just to pass into this method
> absolutely out of the question for performance reasons).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10215) TableNotFoundException should be thrown after removing stale znode in ETH
[
https://issues.apache.org/jira/browse/HBASE-10215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855422#comment-13855422
]
ramkrishna.s.vasudevan commented on HBASE-10215:
+1 on patch. Better to have this change.
> TableNotFoundException should be thrown after removing stale znode in ETH
> -
>
> Key: HBASE-10215
> URL: https://issues.apache.org/jira/browse/HBASE-10215
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.96.1, 0.94.14
>Reporter: rajeshbabu
>Assignee: rajeshbabu
>Priority: Minor
> Fix For: 0.98.0, 0.94.16, 0.96.2, 0.99.0
>
> Attachments: HBASE-10215.patch, HBASE-10215_v2.patch
>
>
> Lets suppose master went down while creating table then znode will be left in
> ENABLING state. Master to recover them on restart.
> If there are no meta entries for the table.
> While recovering the table we are checking whether table exists in meta or
> not, if not we are removing the znode. After removing znode we need to throw
> TableNotFoundException. Presently not throwing the exception so the znode
> will be recrated. It will be stale forever. Even on master restart we cannot
> delete. We cannot create the table with same name also.
> {code}
> // Check if table exists
> if (!MetaReader.tableExists(catalogTracker, tableName)) {
> // retainAssignment is true only during recovery. In normal case it
> is false
> if (!this.skipTableStateCheck) {
> throw new TableNotFoundException(tableName);
> }
> try {
> this.assignmentManager.getZKTable().removeEnablingTable(tableName,
> true);
> } catch (KeeperException e) {
> // TODO : Use HBCK to clear such nodes
> LOG.warn("Failed to delete the ENABLING node for the table " +
> tableName
> + ". The table will remain unusable. Run HBCK to manually fix
> the problem.");
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-7226) HRegion.checkAndMutate uses incorrect comparison result for <, <=, > and >=
[
https://issues.apache.org/jira/browse/HBASE-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855419#comment-13855419
]
Hadoop QA commented on HBASE-7226:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12620124/HBASE-7226-trunk-v2.patch
against trunk revision .
ATTACHMENT ID: 12620124
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop
1.1 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8262//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8262//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8262//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8262//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8262//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8262//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8262//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8262//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8262//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8262//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8262//console
This message is automatically generated.
> HRegion.checkAndMutate uses incorrect comparison result for <, <=, > and >=
> ---
>
> Key: HBASE-7226
> URL: https://issues.apache.org/jira/browse/HBASE-7226
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Reporter: Feng Honghua
>Assignee: Feng Honghua
> Fix For: 0.94.16
>
> Attachments: HBASE-7226-trunk-v2.patch, HBASE-7226-trunk.patch,
> HRegion_HBASE_7226_0.94.2.patch
>
>
> in HRegion.checkAndMutate, incorrect comparison results are used for <, <=, >
> and >=, as below:
> switch (compareOp) {
> case LESS:
> matches = compareResult <= 0; // should be '<' here
> break;
> case LESS_OR_EQUAL:
> matches = compareResult < 0; // should be '<=' here
> break;
> case EQUAL:
> matches = compareResult == 0;
> break;
> case NOT_EQUAL:
> matches = compareResult != 0;
> break;
> case GREATER_OR_EQUAL:
> matches = compareResult > 0; // should be '>=' here
> break;
> case GREATER:
> matches = compareResult >= 0; // should be '>' here
> break;
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10225) Bug in calls to RegionObsever.postScannerFilterRow
[
https://issues.apache.org/jira/browse/HBASE-10225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855420#comment-13855420
]
Lars Hofhansl commented on HBASE-10225:
---
Wow. You guys are fast, I was just about to post a patch for the same.
bq. This will be called when the filter filters out the row. So your test is
like filtering out most of the rows Lars?
Exactly. The scenario under which this is an issue is not common though (most
rows filters and very tall tables).
> Bug in calls to RegionObsever.postScannerFilterRow
> --
>
> Key: HBASE-10225
> URL: https://issues.apache.org/jira/browse/HBASE-10225
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
> Fix For: 0.98.0, 0.94.16, 0.96.2, 0.99.0
>
> Attachments: HBASE-10225.patch
>
>
> Just noticed that while looking at HBASE-10047.
> In 0.94 (and presumably in trunk, will check later) we have this:
> {code}
> protected boolean nextRow(byte [] currentRow, int offset, short length)
> throws IOException {
> ...
> if (this.region.getCoprocessorHost() != null) {
> return this.region.getCoprocessorHost().postScannerFilterRow(this,
> currentRow);
> }
> return true;
> }
> {code}
> Notice how we only pass currentRow into the coprocessor, but not offset and
> length. Anything using this hook currently is 100% broken. The hook was added
> in 0.94.5 (HBASE-5664), it never worked correctly.
> [~anoopsamjohn], you had added the hook. Do you still need it?
> We can either remove it (I'd prefer that in the light of the performance
> issued observed in HBASE-10047, we can leave the stub in BaseRegionObserver
> in 0.94, but document that it is no-op), or we'd have to have change its
> signature to be able to pass offset and length as well.
> Since nobody noticed nobody is using this hook currently, so both should be
> valid options.
> (Making a new standalone copy of the rowkey just to pass into this method
> absolutely out of the question for performance reasons).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-6104) Require EXEC permission to call coprocessor endpoints
[
https://issues.apache.org/jira/browse/HBASE-6104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855398#comment-13855398
]
Hadoop QA commented on HBASE-6104:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12620120/6104.patch
against trunk revision .
ATTACHMENT ID: 12620120
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop
1.1 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8261//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8261//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8261//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8261//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8261//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8261//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8261//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8261//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8261//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8261//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8261//console
This message is automatically generated.
> Require EXEC permission to call coprocessor endpoints
> -
>
> Key: HBASE-6104
> URL: https://issues.apache.org/jira/browse/HBASE-6104
> Project: HBase
> Issue Type: Sub-task
> Components: Coprocessors, security
>Reporter: Gary Helmling
>Assignee: Andrew Purtell
> Fix For: 0.98.0, 0.99.0
>
> Attachments: 6104.patch, 6104.patch, 6104.patch, 6104.patch
>
>
> The EXEC action currently exists as only a placeholder in access control. It
> should really be used to enforce access to coprocessor endpoint RPC calls,
> which are currently unrestricted.
> How the ACLs to support this would be modeled deserves some discussion:
> * Should access be scoped to a specific table and CoprocessorProtocol
> extension?
> * Should it be possible to grant access to a CoprocessorProtocol
> implementation globally (regardless of table)?
> * Are per-method restrictions necessary?
> * Should we expose hooks available to endpoint implementors so that they
> could additionally apply their own permission checks? Some CP endpoints may
> want to require READ permissions, others may want to enforce WRITE, or READ +
> WRITE.
> To apply these kinds of checks we would also have to extend the
> RegionObserver interface to provide hooks wrapping HRegion.exec().
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-10225) Bug in calls to RegionObsever.postScannerFilterRow
[
https://issues.apache.org/jira/browse/HBASE-10225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anoop Sam John updated HBASE-10225:
---
Status: Patch Available (was: Open)
> Bug in calls to RegionObsever.postScannerFilterRow
> --
>
> Key: HBASE-10225
> URL: https://issues.apache.org/jira/browse/HBASE-10225
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
> Fix For: 0.98.0, 0.94.16, 0.96.2, 0.99.0
>
> Attachments: HBASE-10225.patch
>
>
> Just noticed that while looking at HBASE-10047.
> In 0.94 (and presumably in trunk, will check later) we have this:
> {code}
> protected boolean nextRow(byte [] currentRow, int offset, short length)
> throws IOException {
> ...
> if (this.region.getCoprocessorHost() != null) {
> return this.region.getCoprocessorHost().postScannerFilterRow(this,
> currentRow);
> }
> return true;
> }
> {code}
> Notice how we only pass currentRow into the coprocessor, but not offset and
> length. Anything using this hook currently is 100% broken. The hook was added
> in 0.94.5 (HBASE-5664), it never worked correctly.
> [~anoopsamjohn], you had added the hook. Do you still need it?
> We can either remove it (I'd prefer that in the light of the performance
> issued observed in HBASE-10047, we can leave the stub in BaseRegionObserver
> in 0.94, but document that it is no-op), or we'd have to have change its
> signature to be able to pass offset and length as well.
> Since nobody noticed nobody is using this hook currently, so both should be
> valid options.
> (Making a new standalone copy of the rowkey just to pass into this method
> absolutely out of the question for performance reasons).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-9151) HBCK cannot fix when meta server znode deleted, this can happen if all region servers stopped and there are no logs to split.
[
https://issues.apache.org/jira/browse/HBASE-9151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855396#comment-13855396
]
Hudson commented on HBASE-9151:
---
SUCCESS: Integrated in hbase-0.96 #239 (See
[https://builds.apache.org/job/hbase-0.96/239/])
HBASE-9151 HBCK cannot fix when meta server znode deleted, this can happen if
all region servers stopped and there are no logs to split. (rajeshbabu: rev
1553058)
*
/hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
*
/hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
*
/hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRSKilledWhenInitializing.java
*
/hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java
> HBCK cannot fix when meta server znode deleted, this can happen if all region
> servers stopped and there are no logs to split.
> -
>
> Key: HBASE-9151
> URL: https://issues.apache.org/jira/browse/HBASE-9151
> Project: HBase
> Issue Type: Bug
> Components: hbck
>Reporter: rajeshbabu
>Assignee: rajeshbabu
> Fix For: 0.98.0, 0.96.2, 0.99.0
>
> Attachments: HBASE-9151.patch, HBASE-9151_v2.patch,
> HBASE-9151_v2.patch
>
>
> When meta server znode deleted and meta in FAILED_OPEN state, then hbck
> cannot fix it. This scenario can come when all region servers stopped by stop
> command and didnt start any RS within 10 secs(with default configurations).
> {code}
> public void assignMeta() throws KeeperException {
> MetaRegionTracker.deleteMetaLocation(this.watcher);
> assign(HRegionInfo.FIRST_META_REGIONINFO, true);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10225) Bug in calls to RegionObsever.postScannerFilterRow
[
https://issues.apache.org/jira/browse/HBASE-10225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855394#comment-13855394
]
Anoop Sam John commented on HBASE-10225:
[~lhofhansl] If you okey with current patch, I can give patch for 94 branch
also now.
> Bug in calls to RegionObsever.postScannerFilterRow
> --
>
> Key: HBASE-10225
> URL: https://issues.apache.org/jira/browse/HBASE-10225
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
> Fix For: 0.98.0, 0.94.16, 0.96.2, 0.99.0
>
> Attachments: HBASE-10225.patch
>
>
> Just noticed that while looking at HBASE-10047.
> In 0.94 (and presumably in trunk, will check later) we have this:
> {code}
> protected boolean nextRow(byte [] currentRow, int offset, short length)
> throws IOException {
> ...
> if (this.region.getCoprocessorHost() != null) {
> return this.region.getCoprocessorHost().postScannerFilterRow(this,
> currentRow);
> }
> return true;
> }
> {code}
> Notice how we only pass currentRow into the coprocessor, but not offset and
> length. Anything using this hook currently is 100% broken. The hook was added
> in 0.94.5 (HBASE-5664), it never worked correctly.
> [~anoopsamjohn], you had added the hook. Do you still need it?
> We can either remove it (I'd prefer that in the light of the performance
> issued observed in HBASE-10047, we can leave the stub in BaseRegionObserver
> in 0.94, but document that it is no-op), or we'd have to have change its
> signature to be able to pass offset and length as well.
> Since nobody noticed nobody is using this hook currently, so both should be
> valid options.
> (Making a new standalone copy of the rowkey just to pass into this method
> absolutely out of the question for performance reasons).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-10225) Bug in calls to RegionObsever.postScannerFilterRow
[
https://issues.apache.org/jira/browse/HBASE-10225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anoop Sam John updated HBASE-10225:
---
Attachment: HBASE-10225.patch
Fix with changing the CP hook signature.
> Bug in calls to RegionObsever.postScannerFilterRow
> --
>
> Key: HBASE-10225
> URL: https://issues.apache.org/jira/browse/HBASE-10225
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
> Fix For: 0.98.0, 0.94.16, 0.96.2, 0.99.0
>
> Attachments: HBASE-10225.patch
>
>
> Just noticed that while looking at HBASE-10047.
> In 0.94 (and presumably in trunk, will check later) we have this:
> {code}
> protected boolean nextRow(byte [] currentRow, int offset, short length)
> throws IOException {
> ...
> if (this.region.getCoprocessorHost() != null) {
> return this.region.getCoprocessorHost().postScannerFilterRow(this,
> currentRow);
> }
> return true;
> }
> {code}
> Notice how we only pass currentRow into the coprocessor, but not offset and
> length. Anything using this hook currently is 100% broken. The hook was added
> in 0.94.5 (HBASE-5664), it never worked correctly.
> [~anoopsamjohn], you had added the hook. Do you still need it?
> We can either remove it (I'd prefer that in the light of the performance
> issued observed in HBASE-10047, we can leave the stub in BaseRegionObserver
> in 0.94, but document that it is no-op), or we'd have to have change its
> signature to be able to pass offset and length as well.
> Since nobody noticed nobody is using this hook currently, so both should be
> valid options.
> (Making a new standalone copy of the rowkey just to pass into this method
> absolutely out of the question for performance reasons).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-9151) HBCK cannot fix when meta server znode deleted, this can happen if all region servers stopped and there are no logs to split.
[
https://issues.apache.org/jira/browse/HBASE-9151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855393#comment-13855393
]
Hudson commented on HBASE-9151:
---
SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-1.1 #19 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/19/])
HBASE-9151 HBCK cannot fix when meta server znode deleted, this can happen if
all region servers stopped and there are no logs to split. (rajeshbabu: rev
1553056)
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRSKilledWhenInitializing.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java
> HBCK cannot fix when meta server znode deleted, this can happen if all region
> servers stopped and there are no logs to split.
> -
>
> Key: HBASE-9151
> URL: https://issues.apache.org/jira/browse/HBASE-9151
> Project: HBase
> Issue Type: Bug
> Components: hbck
>Reporter: rajeshbabu
>Assignee: rajeshbabu
> Fix For: 0.98.0, 0.96.2, 0.99.0
>
> Attachments: HBASE-9151.patch, HBASE-9151_v2.patch,
> HBASE-9151_v2.patch
>
>
> When meta server znode deleted and meta in FAILED_OPEN state, then hbck
> cannot fix it. This scenario can come when all region servers stopped by stop
> command and didnt start any RS within 10 secs(with default configurations).
> {code}
> public void assignMeta() throws KeeperException {
> MetaRegionTracker.deleteMetaLocation(this.watcher);
> assign(HRegionInfo.FIRST_META_REGIONINFO, true);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10214) Regionserver shutdown improperly and leaves the dir in .old not deleted
[
https://issues.apache.org/jira/browse/HBASE-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855389#comment-13855389
]
chunhui shen commented on HBASE-10214:
--
+1 for patch v3
Will commit it if no further comment
> Regionserver shutdown improperly and leaves the dir in .old not deleted
> ---
>
> Key: HBASE-10214
> URL: https://issues.apache.org/jira/browse/HBASE-10214
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.94.14
>Reporter: binlijin
> Attachments: HBASE-10214-94-V2.patch, HBASE-10214-94-V3.patch,
> HBASE-10214-94.patch
>
>
> RegionServer log
> {code}
> 2013-12-18 15:17:45,771 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegionServer: Waiting on
> 51b27391410efdca841db264df46085f
> 2013-12-18 15:17:45,776 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Connected to master at
> null
> 2013-12-18 15:17:48,776 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Exiting; cluster
> shutdown set and not carrying any regions
> 2013-12-18 15:17:48,776 FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server
> node,60020,1384410974572: Unhandled exception: null
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:880)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:753)
> at java.lang.Thread.run(Thread.java:662)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10225) Bug in calls to RegionObsever.postScannerFilterRow
[
https://issues.apache.org/jira/browse/HBASE-10225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855388#comment-13855388
]
rajeshbabu commented on HBASE-10225:
Yes Anoop, the hook has used in HIndex and the current row is not used.
Its better to correct.
> Bug in calls to RegionObsever.postScannerFilterRow
> --
>
> Key: HBASE-10225
> URL: https://issues.apache.org/jira/browse/HBASE-10225
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
> Fix For: 0.98.0, 0.94.16, 0.96.2, 0.99.0
>
>
> Just noticed that while looking at HBASE-10047.
> In 0.94 (and presumably in trunk, will check later) we have this:
> {code}
> protected boolean nextRow(byte [] currentRow, int offset, short length)
> throws IOException {
> ...
> if (this.region.getCoprocessorHost() != null) {
> return this.region.getCoprocessorHost().postScannerFilterRow(this,
> currentRow);
> }
> return true;
> }
> {code}
> Notice how we only pass currentRow into the coprocessor, but not offset and
> length. Anything using this hook currently is 100% broken. The hook was added
> in 0.94.5 (HBASE-5664), it never worked correctly.
> [~anoopsamjohn], you had added the hook. Do you still need it?
> We can either remove it (I'd prefer that in the light of the performance
> issued observed in HBASE-10047, we can leave the stub in BaseRegionObserver
> in 0.94, but document that it is no-op), or we'd have to have change its
> signature to be able to pass offset and length as well.
> Since nobody noticed nobody is using this hook currently, so both should be
> valid options.
> (Making a new standalone copy of the rowkey just to pass into this method
> absolutely out of the question for performance reasons).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10225) Bug in calls to RegionObsever.postScannerFilterRow
[
https://issues.apache.org/jira/browse/HBASE-10225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855381#comment-13855381
]
Anoop Sam John commented on HBASE-10225:
I can give a simple quick patch..
> Bug in calls to RegionObsever.postScannerFilterRow
> --
>
> Key: HBASE-10225
> URL: https://issues.apache.org/jira/browse/HBASE-10225
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
> Fix For: 0.98.0, 0.94.16, 0.96.2, 0.99.0
>
>
> Just noticed that while looking at HBASE-10047.
> In 0.94 (and presumably in trunk, will check later) we have this:
> {code}
> protected boolean nextRow(byte [] currentRow, int offset, short length)
> throws IOException {
> ...
> if (this.region.getCoprocessorHost() != null) {
> return this.region.getCoprocessorHost().postScannerFilterRow(this,
> currentRow);
> }
> return true;
> }
> {code}
> Notice how we only pass currentRow into the coprocessor, but not offset and
> length. Anything using this hook currently is 100% broken. The hook was added
> in 0.94.5 (HBASE-5664), it never worked correctly.
> [~anoopsamjohn], you had added the hook. Do you still need it?
> We can either remove it (I'd prefer that in the light of the performance
> issued observed in HBASE-10047, we can leave the stub in BaseRegionObserver
> in 0.94, but document that it is no-op), or we'd have to have change its
> signature to be able to pass offset and length as well.
> Since nobody noticed nobody is using this hook currently, so both should be
> valid options.
> (Making a new standalone copy of the rowkey just to pass into this method
> absolutely out of the question for performance reasons).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10214) Regionserver shutdown improperly and leaves the dir in .old not deleted
[
https://issues.apache.org/jira/browse/HBASE-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855384#comment-13855384
]
binlijin commented on HBASE-10214:
--
@chunhui shen, test result is:
Results :
Tests run: 1405, Failures: 0, Errors: 0, Skipped: 13
[INFO]
[INFO] BUILD SUCCESS
[INFO]
[INFO] Total time: 50:11.332s
[INFO] Finished at: Mon Dec 23 12:09:17 CST 2013
[INFO] Final Memory: 38M/751M
[INFO]
All test passed.
> Regionserver shutdown improperly and leaves the dir in .old not deleted
> ---
>
> Key: HBASE-10214
> URL: https://issues.apache.org/jira/browse/HBASE-10214
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.94.14
>Reporter: binlijin
> Attachments: HBASE-10214-94-V2.patch, HBASE-10214-94-V3.patch,
> HBASE-10214-94.patch
>
>
> RegionServer log
> {code}
> 2013-12-18 15:17:45,771 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegionServer: Waiting on
> 51b27391410efdca841db264df46085f
> 2013-12-18 15:17:45,776 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Connected to master at
> null
> 2013-12-18 15:17:48,776 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Exiting; cluster
> shutdown set and not carrying any regions
> 2013-12-18 15:17:48,776 FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server
> node,60020,1384410974572: Unhandled exception: null
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:880)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:753)
> at java.lang.Thread.run(Thread.java:662)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10225) Bug in calls to RegionObsever.postScannerFilterRow
[
https://issues.apache.org/jira/browse/HBASE-10225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855380#comment-13855380
]
Anoop Sam John commented on HBASE-10225:
It is still used by HIndex. Correct [~rajesh23]?
We never needed the current row and was not using that at all. So not noticed
this issue. :(
+1 for correcting this in all versions.
This will be called when the filter filters out the row. So your test is like
filtering out most of the rows Lars?
> Bug in calls to RegionObsever.postScannerFilterRow
> --
>
> Key: HBASE-10225
> URL: https://issues.apache.org/jira/browse/HBASE-10225
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
> Fix For: 0.98.0, 0.94.16, 0.96.2, 0.99.0
>
>
> Just noticed that while looking at HBASE-10047.
> In 0.94 (and presumably in trunk, will check later) we have this:
> {code}
> protected boolean nextRow(byte [] currentRow, int offset, short length)
> throws IOException {
> ...
> if (this.region.getCoprocessorHost() != null) {
> return this.region.getCoprocessorHost().postScannerFilterRow(this,
> currentRow);
> }
> return true;
> }
> {code}
> Notice how we only pass currentRow into the coprocessor, but not offset and
> length. Anything using this hook currently is 100% broken. The hook was added
> in 0.94.5 (HBASE-5664), it never worked correctly.
> [~anoopsamjohn], you had added the hook. Do you still need it?
> We can either remove it (I'd prefer that in the light of the performance
> issued observed in HBASE-10047, we can leave the stub in BaseRegionObserver
> in 0.94, but document that it is no-op), or we'd have to have change its
> signature to be able to pass offset and length as well.
> Since nobody noticed nobody is using this hook currently, so both should be
> valid options.
> (Making a new standalone copy of the rowkey just to pass into this method
> absolutely out of the question for performance reasons).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-9151) HBCK cannot fix when meta server znode deleted, this can happen if all region servers stopped and there are no logs to split.
[
https://issues.apache.org/jira/browse/HBASE-9151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855377#comment-13855377
]
Hudson commented on HBASE-9151:
---
SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #26 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/26/])
HBASE-9151 HBCK cannot fix when meta server znode deleted, this can happen if
all region servers stopped and there are no logs to split. (rajeshbabu: rev
1553057)
*
/hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
*
/hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
*
/hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRSKilledWhenInitializing.java
*
/hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java
> HBCK cannot fix when meta server znode deleted, this can happen if all region
> servers stopped and there are no logs to split.
> -
>
> Key: HBASE-9151
> URL: https://issues.apache.org/jira/browse/HBASE-9151
> Project: HBase
> Issue Type: Bug
> Components: hbck
>Reporter: rajeshbabu
>Assignee: rajeshbabu
> Fix For: 0.98.0, 0.96.2, 0.99.0
>
> Attachments: HBASE-9151.patch, HBASE-9151_v2.patch,
> HBASE-9151_v2.patch
>
>
> When meta server znode deleted and meta in FAILED_OPEN state, then hbck
> cannot fix it. This scenario can come when all region servers stopped by stop
> command and didnt start any RS within 10 secs(with default configurations).
> {code}
> public void assignMeta() throws KeeperException {
> MetaRegionTracker.deleteMetaLocation(this.watcher);
> assign(HRegionInfo.FIRST_META_REGIONINFO, true);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10225) Bug is calls to RegionObsever.postScannerFilterRow
[
https://issues.apache.org/jira/browse/HBASE-10225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855376#comment-13855376
]
Lars Hofhansl commented on HBASE-10225:
---
[~stack], [~apurtell], any comments for 0.96 and 0.98?
> Bug is calls to RegionObsever.postScannerFilterRow
> --
>
> Key: HBASE-10225
> URL: https://issues.apache.org/jira/browse/HBASE-10225
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
> Fix For: 0.98.0, 0.94.16, 0.96.2, 0.99.0
>
>
> Just noticed that while looking at HBASE-10047.
> In 0.94 (and presumably in trunk, will check later) we have this:
> {code}
> protected boolean nextRow(byte [] currentRow, int offset, short length)
> throws IOException {
> ...
> if (this.region.getCoprocessorHost() != null) {
> return this.region.getCoprocessorHost().postScannerFilterRow(this,
> currentRow);
> }
> return true;
> }
> {code}
> Notice how we only pass currentRow into the coprocessor, but not offset and
> length. Anything using this hook currently is 100% broken. The hook was added
> in 0.94.5 (HBASE-5664), it never worked correctly.
> [~anoopsamjohn], you had added the hook. Do you still need it?
> We can either remove it (I'd prefer that in the light of the performance
> issued observed in HBASE-10047, we can leave the stub in BaseRegionObserver
> in 0.94, but document that it is no-op), or we'd have to have change its
> signature to be able to pass offset and length as well.
> Since nobody noticed nobody is using this hook currently, so both should be
> valid options.
> (Making a new standalone copy of the rowkey just to pass into this method
> absolutely out of the question for performance reasons).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-10225) Bug in calls to RegionObsever.postScannerFilterRow
[
https://issues.apache.org/jira/browse/HBASE-10225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-10225:
--
Summary: Bug in calls to RegionObsever.postScannerFilterRow (was: Bug is
calls to RegionObsever.postScannerFilterRow)
> Bug in calls to RegionObsever.postScannerFilterRow
> --
>
> Key: HBASE-10225
> URL: https://issues.apache.org/jira/browse/HBASE-10225
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
> Fix For: 0.98.0, 0.94.16, 0.96.2, 0.99.0
>
>
> Just noticed that while looking at HBASE-10047.
> In 0.94 (and presumably in trunk, will check later) we have this:
> {code}
> protected boolean nextRow(byte [] currentRow, int offset, short length)
> throws IOException {
> ...
> if (this.region.getCoprocessorHost() != null) {
> return this.region.getCoprocessorHost().postScannerFilterRow(this,
> currentRow);
> }
> return true;
> }
> {code}
> Notice how we only pass currentRow into the coprocessor, but not offset and
> length. Anything using this hook currently is 100% broken. The hook was added
> in 0.94.5 (HBASE-5664), it never worked correctly.
> [~anoopsamjohn], you had added the hook. Do you still need it?
> We can either remove it (I'd prefer that in the light of the performance
> issued observed in HBASE-10047, we can leave the stub in BaseRegionObserver
> in 0.94, but document that it is no-op), or we'd have to have change its
> signature to be able to pass offset and length as well.
> Since nobody noticed nobody is using this hook currently, so both should be
> valid options.
> (Making a new standalone copy of the rowkey just to pass into this method
> absolutely out of the question for performance reasons).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-10225) Bug is calls to RegionObsever.postScannerFilterRow
[
https://issues.apache.org/jira/browse/HBASE-10225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-10225:
--
Fix Version/s: 0.99.0
0.96.2
0.94.16
0.98.0
> Bug is calls to RegionObsever.postScannerFilterRow
> --
>
> Key: HBASE-10225
> URL: https://issues.apache.org/jira/browse/HBASE-10225
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
> Fix For: 0.98.0, 0.94.16, 0.96.2, 0.99.0
>
>
> Just noticed that while looking at HBASE-10047.
> In 0.94 (and presumably in trunk, will check later) we have this:
> {code}
> protected boolean nextRow(byte [] currentRow, int offset, short length)
> throws IOException {
> ...
> if (this.region.getCoprocessorHost() != null) {
> return this.region.getCoprocessorHost().postScannerFilterRow(this,
> currentRow);
> }
> return true;
> }
> {code}
> Notice how we only pass currentRow into the coprocessor, but not offset and
> length. Anything using this hook currently is 100% broken. The hook was added
> in 0.94.5 (HBASE-5664), it never worked correctly.
> [~anoopsamjohn], you had added the hook. Do you still need it?
> We can either remove it (I'd prefer that in the light of the performance
> issued observed in HBASE-10047, we can leave the stub in BaseRegionObserver
> in 0.94, but document that it is no-op), or we'd have to have change its
> signature to be able to pass offset and length as well.
> Since nobody noticed nobody is using this hook currently, so both should be
> valid options.
> (Making a new standalone copy of the rowkey just to pass into this method
> absolutely out of the question for performance reasons).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Created] (HBASE-10225) Bug is calls to RegionObsever.postScannerFilterRow
Lars Hofhansl created HBASE-10225:
-
Summary: Bug is calls to RegionObsever.postScannerFilterRow
Key: HBASE-10225
URL: https://issues.apache.org/jira/browse/HBASE-10225
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Just noticed that while looking at HBASE-10047.
In 0.94 (and presumably in trunk, will check later) we have this:
{code}
protected boolean nextRow(byte [] currentRow, int offset, short length)
throws IOException {
...
if (this.region.getCoprocessorHost() != null) {
return this.region.getCoprocessorHost().postScannerFilterRow(this,
currentRow);
}
return true;
}
{code}
Notice how we only pass currentRow into the coprocessor, but not offset and
length. Anything using this hook currently is 100% broken. The hook was added
in 0.94.5 (HBASE-5664), it never worked correctly.
[~anoopsamjohn], you had added the hook. Do you still need it?
We can either remove it (I'd prefer that in the light of the performance issued
observed in HBASE-10047, we can leave the stub in BaseRegionObserver in 0.94,
but document that it is no-op), or we'd have to have change its signature to be
able to pass offset and length as well.
Since nobody noticed nobody is using this hook currently, so both should be
valid options.
(Making a new standalone copy of the rowkey just to pass into this method
absolutely out of the question for performance reasons).
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Created] (HBASE-10224) Line length check in test-patch.sh is broken
Ted Yu created HBASE-10224:
--
Summary: Line length check in test-patch.sh is broken
Key: HBASE-10224
URL: https://issues.apache.org/jira/browse/HBASE-10224
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Here is related code in the script:
{code}
MAX_LINE_LENGTH_PATCH=`expr $MAX_LINE_LENGTH + 1`
...
ll=`echo "$lines" | wc -l`
if [[ "$ll" -gt "$MAX_LINE_LENGTH_PATCH" ]]; then
{code}
Here was the result from dry run:
{code}
TYus-MacBook-Pro:m7 tyu$ lines=`cat ~/trunk/7226-trunk.patch | grep "^+" | grep
-v "^@@" | grep -v "^+++" | grep -v "import" | grep -v
"hbase.protobuf.generated" | awk -v len="101" 'length ($0) > len' | head -n 10`
TYus-MacBook-Pro:m7 tyu$ ll=`echo "$lines" | wc -l`
TYus-MacBook-Pro:m7 tyu$ echo $ll
3
TYus-MacBook-Pro:m7 tyu$ echo $lines
+ // Test CompareOp.LESS_OR_EQUAL: original = val2, compare with val2, succeed
(value still = val2) + // Test CompareOp.LESS_OR_EQUAL: original = val2,
compare with val1, succeed (now value = val3) + // Test
CompareOp.GREATER_OR_EQUAL: original = val2, compare with val2, succeed (value
still = val2)
{code}
The value of $ll should be compared with 0, not the value of
$MAX_LINE_LENGTH_PATCH
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-9151) HBCK cannot fix when meta server znode deleted, this can happen if all region servers stopped and there are no logs to split.
[
https://issues.apache.org/jira/browse/HBASE-9151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855372#comment-13855372
]
Hudson commented on HBASE-9151:
---
SUCCESS: Integrated in HBase-0.98 #29 (See
[https://builds.apache.org/job/HBase-0.98/29/])
HBASE-9151 HBCK cannot fix when meta server znode deleted, this can happen if
all region servers stopped and there are no logs to split. (rajeshbabu: rev
1553057)
*
/hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
*
/hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
*
/hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRSKilledWhenInitializing.java
*
/hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java
> HBCK cannot fix when meta server znode deleted, this can happen if all region
> servers stopped and there are no logs to split.
> -
>
> Key: HBASE-9151
> URL: https://issues.apache.org/jira/browse/HBASE-9151
> Project: HBase
> Issue Type: Bug
> Components: hbck
>Reporter: rajeshbabu
>Assignee: rajeshbabu
> Fix For: 0.98.0, 0.96.2, 0.99.0
>
> Attachments: HBASE-9151.patch, HBASE-9151_v2.patch,
> HBASE-9151_v2.patch
>
>
> When meta server znode deleted and meta in FAILED_OPEN state, then hbck
> cannot fix it. This scenario can come when all region servers stopped by stop
> command and didnt start any RS within 10 secs(with default configurations).
> {code}
> public void assignMeta() throws KeeperException {
> MetaRegionTracker.deleteMetaLocation(this.watcher);
> assign(HRegionInfo.FIRST_META_REGIONINFO, true);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-7226) HRegion.checkAndMutate uses incorrect comparison result for <, <=, > and >=
[
https://issues.apache.org/jira/browse/HBASE-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-7226:
--
Hadoop Flags: Reviewed
Status: Patch Available (was: Reopened)
> HRegion.checkAndMutate uses incorrect comparison result for <, <=, > and >=
> ---
>
> Key: HBASE-7226
> URL: https://issues.apache.org/jira/browse/HBASE-7226
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Reporter: Feng Honghua
>Assignee: Feng Honghua
> Fix For: 0.94.16
>
> Attachments: HBASE-7226-trunk-v2.patch, HBASE-7226-trunk.patch,
> HRegion_HBASE_7226_0.94.2.patch
>
>
> in HRegion.checkAndMutate, incorrect comparison results are used for <, <=, >
> and >=, as below:
> switch (compareOp) {
> case LESS:
> matches = compareResult <= 0; // should be '<' here
> break;
> case LESS_OR_EQUAL:
> matches = compareResult < 0; // should be '<=' here
> break;
> case EQUAL:
> matches = compareResult == 0;
> break;
> case NOT_EQUAL:
> matches = compareResult != 0;
> break;
> case GREATER_OR_EQUAL:
> matches = compareResult > 0; // should be '>=' here
> break;
> case GREATER:
> matches = compareResult >= 0; // should be '>' here
> break;
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-9151) HBCK cannot fix when meta server znode deleted, this can happen if all region servers stopped and there are no logs to split.
[
https://issues.apache.org/jira/browse/HBASE-9151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855368#comment-13855368
]
Hudson commented on HBASE-9151:
---
FAILURE: Integrated in HBase-TRUNK #4745 (See
[https://builds.apache.org/job/HBase-TRUNK/4745/])
HBASE-9151 HBCK cannot fix when meta server znode deleted, this can happen if
all region servers stopped and there are no logs to split. (rajeshbabu: rev
1553056)
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRSKilledWhenInitializing.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java
> HBCK cannot fix when meta server znode deleted, this can happen if all region
> servers stopped and there are no logs to split.
> -
>
> Key: HBASE-9151
> URL: https://issues.apache.org/jira/browse/HBASE-9151
> Project: HBase
> Issue Type: Bug
> Components: hbck
>Reporter: rajeshbabu
>Assignee: rajeshbabu
> Fix For: 0.98.0, 0.96.2, 0.99.0
>
> Attachments: HBASE-9151.patch, HBASE-9151_v2.patch,
> HBASE-9151_v2.patch
>
>
> When meta server znode deleted and meta in FAILED_OPEN state, then hbck
> cannot fix it. This scenario can come when all region servers stopped by stop
> command and didnt start any RS within 10 secs(with default configurations).
> {code}
> public void assignMeta() throws KeeperException {
> MetaRegionTracker.deleteMetaLocation(this.watcher);
> assign(HRegionInfo.FIRST_META_REGIONINFO, true);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-7226) HRegion.checkAndMutate uses incorrect comparison result for <, <=, > and >=
[
https://issues.apache.org/jira/browse/HBASE-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855365#comment-13855365
]
Feng Honghua commented on HBASE-7226:
-
thanks [~lhofhansl] and [~yuzhih...@gmail.com], attach a new patch per
[~yuzhih...@gmail.com]'s feedback
> HRegion.checkAndMutate uses incorrect comparison result for <, <=, > and >=
> ---
>
> Key: HBASE-7226
> URL: https://issues.apache.org/jira/browse/HBASE-7226
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Reporter: Feng Honghua
>Assignee: Feng Honghua
> Fix For: 0.94.16
>
> Attachments: HBASE-7226-trunk-v2.patch, HBASE-7226-trunk.patch,
> HRegion_HBASE_7226_0.94.2.patch
>
>
> in HRegion.checkAndMutate, incorrect comparison results are used for <, <=, >
> and >=, as below:
> switch (compareOp) {
> case LESS:
> matches = compareResult <= 0; // should be '<' here
> break;
> case LESS_OR_EQUAL:
> matches = compareResult < 0; // should be '<=' here
> break;
> case EQUAL:
> matches = compareResult == 0;
> break;
> case NOT_EQUAL:
> matches = compareResult != 0;
> break;
> case GREATER_OR_EQUAL:
> matches = compareResult > 0; // should be '>=' here
> break;
> case GREATER:
> matches = compareResult >= 0; // should be '>' here
> break;
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-7226) HRegion.checkAndMutate uses incorrect comparison result for <, <=, > and >=
[
https://issues.apache.org/jira/browse/HBASE-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Honghua updated HBASE-7226:
Attachment: HBASE-7226-trunk-v2.patch
> HRegion.checkAndMutate uses incorrect comparison result for <, <=, > and >=
> ---
>
> Key: HBASE-7226
> URL: https://issues.apache.org/jira/browse/HBASE-7226
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Reporter: Feng Honghua
>Assignee: Feng Honghua
> Fix For: 0.94.16
>
> Attachments: HBASE-7226-trunk-v2.patch, HBASE-7226-trunk.patch,
> HRegion_HBASE_7226_0.94.2.patch
>
>
> in HRegion.checkAndMutate, incorrect comparison results are used for <, <=, >
> and >=, as below:
> switch (compareOp) {
> case LESS:
> matches = compareResult <= 0; // should be '<' here
> break;
> case LESS_OR_EQUAL:
> matches = compareResult < 0; // should be '<=' here
> break;
> case EQUAL:
> matches = compareResult == 0;
> break;
> case NOT_EQUAL:
> matches = compareResult != 0;
> break;
> case GREATER_OR_EQUAL:
> matches = compareResult > 0; // should be '>=' here
> break;
> case GREATER:
> matches = compareResult >= 0; // should be '>' here
> break;
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-6104) Require EXEC permission to call coprocessor endpoints
[ https://issues.apache.org/jira/browse/HBASE-6104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-6104: -- Status: Patch Available (was: Open) > Require EXEC permission to call coprocessor endpoints > - > > Key: HBASE-6104 > URL: https://issues.apache.org/jira/browse/HBASE-6104 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors, security >Reporter: Gary Helmling >Assignee: Andrew Purtell > Fix For: 0.98.0, 0.99.0 > > Attachments: 6104.patch, 6104.patch, 6104.patch, 6104.patch > > > The EXEC action currently exists as only a placeholder in access control. It > should really be used to enforce access to coprocessor endpoint RPC calls, > which are currently unrestricted. > How the ACLs to support this would be modeled deserves some discussion: > * Should access be scoped to a specific table and CoprocessorProtocol > extension? > * Should it be possible to grant access to a CoprocessorProtocol > implementation globally (regardless of table)? > * Are per-method restrictions necessary? > * Should we expose hooks available to endpoint implementors so that they > could additionally apply their own permission checks? Some CP endpoints may > want to require READ permissions, others may want to enforce WRITE, or READ + > WRITE. > To apply these kinds of checks we would also have to extend the > RegionObserver interface to provide hooks wrapping HRegion.exec(). -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HBASE-6104) Require EXEC permission to call coprocessor endpoints
[ https://issues.apache.org/jira/browse/HBASE-6104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-6104: -- Attachment: 6104.patch New patch including test fix. > Require EXEC permission to call coprocessor endpoints > - > > Key: HBASE-6104 > URL: https://issues.apache.org/jira/browse/HBASE-6104 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors, security >Reporter: Gary Helmling >Assignee: Andrew Purtell > Fix For: 0.98.0, 0.99.0 > > Attachments: 6104.patch, 6104.patch, 6104.patch, 6104.patch > > > The EXEC action currently exists as only a placeholder in access control. It > should really be used to enforce access to coprocessor endpoint RPC calls, > which are currently unrestricted. > How the ACLs to support this would be modeled deserves some discussion: > * Should access be scoped to a specific table and CoprocessorProtocol > extension? > * Should it be possible to grant access to a CoprocessorProtocol > implementation globally (regardless of table)? > * Are per-method restrictions necessary? > * Should we expose hooks available to endpoint implementors so that they > could additionally apply their own permission checks? Some CP endpoints may > want to require READ permissions, others may want to enforce WRITE, or READ + > WRITE. > To apply these kinds of checks we would also have to extend the > RegionObserver interface to provide hooks wrapping HRegion.exec(). -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Resolved] (HBASE-10047) postScannerFilterRow consumes a lot of CPU in tall table scans
[ https://issues.apache.org/jira/browse/HBASE-10047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl resolved HBASE-10047. --- Resolution: Later Can't measure any meaningful perf enhancement in a real world setup. I might to come back to this one later. Might need to look at calling nextRow in RegionScannerImpl anyway. > postScannerFilterRow consumes a lot of CPU in tall table scans > -- > > Key: HBASE-10047 > URL: https://issues.apache.org/jira/browse/HBASE-10047 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl > Attachments: 10047-0.94-poc-v3.txt, 10047-0.94-poc-v4.txt, > 10047-0.94-sample-v2.txt, 10047-0.94-sample.txt, postScannerFilterRow.png > > > Continuing my profiling quest, I find that in scanning tall table (and > filtering everything on the server) a quarter of the time is now spent in the > postScannerFilterRow coprocessor hook. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HBASE-10214) Regionserver shutdown improperly and leaves the dir in .old not deleted
[
https://issues.apache.org/jira/browse/HBASE-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
binlijin updated HBASE-10214:
-
Attachment: HBASE-10214-94-V3.patch
> Regionserver shutdown improperly and leaves the dir in .old not deleted
> ---
>
> Key: HBASE-10214
> URL: https://issues.apache.org/jira/browse/HBASE-10214
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.94.14
>Reporter: binlijin
> Attachments: HBASE-10214-94-V2.patch, HBASE-10214-94-V3.patch,
> HBASE-10214-94.patch
>
>
> RegionServer log
> {code}
> 2013-12-18 15:17:45,771 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegionServer: Waiting on
> 51b27391410efdca841db264df46085f
> 2013-12-18 15:17:45,776 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Connected to master at
> null
> 2013-12-18 15:17:48,776 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Exiting; cluster
> shutdown set and not carrying any regions
> 2013-12-18 15:17:48,776 FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server
> node,60020,1384410974572: Unhandled exception: null
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:880)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:753)
> at java.lang.Thread.run(Thread.java:662)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-10214) Regionserver shutdown improperly and leaves the dir in .old not deleted
[
https://issues.apache.org/jira/browse/HBASE-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
binlijin updated HBASE-10214:
-
Attachment: (was: HBASE-10214-94-3.patch)
> Regionserver shutdown improperly and leaves the dir in .old not deleted
> ---
>
> Key: HBASE-10214
> URL: https://issues.apache.org/jira/browse/HBASE-10214
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.94.14
>Reporter: binlijin
> Attachments: HBASE-10214-94-V2.patch, HBASE-10214-94-V3.patch,
> HBASE-10214-94.patch
>
>
> RegionServer log
> {code}
> 2013-12-18 15:17:45,771 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegionServer: Waiting on
> 51b27391410efdca841db264df46085f
> 2013-12-18 15:17:45,776 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Connected to master at
> null
> 2013-12-18 15:17:48,776 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Exiting; cluster
> shutdown set and not carrying any regions
> 2013-12-18 15:17:48,776 FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server
> node,60020,1384410974572: Unhandled exception: null
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:880)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:753)
> at java.lang.Thread.run(Thread.java:662)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-10214) Regionserver shutdown improperly and leaves the dir in .old not deleted
[
https://issues.apache.org/jira/browse/HBASE-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
binlijin updated HBASE-10214:
-
Attachment: HBASE-10214-94-3.patch
> Regionserver shutdown improperly and leaves the dir in .old not deleted
> ---
>
> Key: HBASE-10214
> URL: https://issues.apache.org/jira/browse/HBASE-10214
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.94.14
>Reporter: binlijin
> Attachments: HBASE-10214-94-V2.patch, HBASE-10214-94-V3.patch,
> HBASE-10214-94.patch
>
>
> RegionServer log
> {code}
> 2013-12-18 15:17:45,771 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegionServer: Waiting on
> 51b27391410efdca841db264df46085f
> 2013-12-18 15:17:45,776 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Connected to master at
> null
> 2013-12-18 15:17:48,776 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Exiting; cluster
> shutdown set and not carrying any regions
> 2013-12-18 15:17:48,776 FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server
> node,60020,1384410974572: Unhandled exception: null
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:880)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:753)
> at java.lang.Thread.run(Thread.java:662)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-6104) Require EXEC permission to call coprocessor endpoints
[ https://issues.apache.org/jira/browse/HBASE-6104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-6104: -- Status: Open (was: Patch Available) > Require EXEC permission to call coprocessor endpoints > - > > Key: HBASE-6104 > URL: https://issues.apache.org/jira/browse/HBASE-6104 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors, security >Reporter: Gary Helmling >Assignee: Andrew Purtell > Fix For: 0.98.0, 0.99.0 > > Attachments: 6104.patch, 6104.patch, 6104.patch > > > The EXEC action currently exists as only a placeholder in access control. It > should really be used to enforce access to coprocessor endpoint RPC calls, > which are currently unrestricted. > How the ACLs to support this would be modeled deserves some discussion: > * Should access be scoped to a specific table and CoprocessorProtocol > extension? > * Should it be possible to grant access to a CoprocessorProtocol > implementation globally (regardless of table)? > * Are per-method restrictions necessary? > * Should we expose hooks available to endpoint implementors so that they > could additionally apply their own permission checks? Some CP endpoints may > want to require READ permissions, others may want to enforce WRITE, or READ + > WRITE. > To apply these kinds of checks we would also have to extend the > RegionObserver interface to provide hooks wrapping HRegion.exec(). -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-6104) Require EXEC permission to call coprocessor endpoints
[ https://issues.apache.org/jira/browse/HBASE-6104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855356#comment-13855356 ] Andrew Purtell commented on HBASE-6104: --- The TestVisibilityLabelsWithACL.testLabelsTableOpsWithDifferentUsers failure is a real issue with the test. > Require EXEC permission to call coprocessor endpoints > - > > Key: HBASE-6104 > URL: https://issues.apache.org/jira/browse/HBASE-6104 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors, security >Reporter: Gary Helmling >Assignee: Andrew Purtell > Fix For: 0.98.0, 0.99.0 > > Attachments: 6104.patch, 6104.patch, 6104.patch > > > The EXEC action currently exists as only a placeholder in access control. It > should really be used to enforce access to coprocessor endpoint RPC calls, > which are currently unrestricted. > How the ACLs to support this would be modeled deserves some discussion: > * Should access be scoped to a specific table and CoprocessorProtocol > extension? > * Should it be possible to grant access to a CoprocessorProtocol > implementation globally (regardless of table)? > * Are per-method restrictions necessary? > * Should we expose hooks available to endpoint implementors so that they > could additionally apply their own permission checks? Some CP endpoints may > want to require READ permissions, others may want to enforce WRITE, or READ + > WRITE. > To apply these kinds of checks we would also have to extend the > RegionObserver interface to provide hooks wrapping HRegion.exec(). -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10214) Regionserver shutdown improperly and leaves the dir in .old not deleted
[
https://issues.apache.org/jira/browse/HBASE-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855355#comment-13855355
]
Ted Yu commented on HBASE-10214:
nit:
{code}
+ // the current server is stoping
{code}
typo: stoping -> stopping
> Regionserver shutdown improperly and leaves the dir in .old not deleted
> ---
>
> Key: HBASE-10214
> URL: https://issues.apache.org/jira/browse/HBASE-10214
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.94.14
>Reporter: binlijin
> Attachments: HBASE-10214-94-V2.patch, HBASE-10214-94.patch
>
>
> RegionServer log
> {code}
> 2013-12-18 15:17:45,771 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegionServer: Waiting on
> 51b27391410efdca841db264df46085f
> 2013-12-18 15:17:45,776 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Connected to master at
> null
> 2013-12-18 15:17:48,776 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Exiting; cluster
> shutdown set and not carrying any regions
> 2013-12-18 15:17:48,776 FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server
> node,60020,1384410974572: Unhandled exception: null
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:880)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:753)
> at java.lang.Thread.run(Thread.java:662)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-6104) Require EXEC permission to call coprocessor endpoints
[ https://issues.apache.org/jira/browse/HBASE-6104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-6104: -- Release Note: If access control is active (the AccessController coprocessor is installed either as a system coprocessor or on a table as a table coprocessor) then you must now grant all relevant users EXEC privilege if they require the ability to execute coprocessor endpoint calls. EXEC privilege, like any other permission, can be granted globally to a user, or on a per table basis. For more information on coprocessor endpoints, see the coprocessor section of the HBase online manual. For more information on granting or revoking permissions using the AccessController, see the security section of the HBase online manual. > Require EXEC permission to call coprocessor endpoints > - > > Key: HBASE-6104 > URL: https://issues.apache.org/jira/browse/HBASE-6104 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors, security >Reporter: Gary Helmling >Assignee: Andrew Purtell > Fix For: 0.98.0, 0.99.0 > > Attachments: 6104.patch, 6104.patch, 6104.patch > > > The EXEC action currently exists as only a placeholder in access control. It > should really be used to enforce access to coprocessor endpoint RPC calls, > which are currently unrestricted. > How the ACLs to support this would be modeled deserves some discussion: > * Should access be scoped to a specific table and CoprocessorProtocol > extension? > * Should it be possible to grant access to a CoprocessorProtocol > implementation globally (regardless of table)? > * Are per-method restrictions necessary? > * Should we expose hooks available to endpoint implementors so that they > could additionally apply their own permission checks? Some CP endpoints may > want to require READ permissions, others may want to enforce WRITE, or READ + > WRITE. > To apply these kinds of checks we would also have to extend the > RegionObserver interface to provide hooks wrapping HRegion.exec(). -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HBASE-10214) Regionserver shutdown improperly and leaves the dir in .old not deleted
[
https://issues.apache.org/jira/browse/HBASE-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-10214:
---
Summary: Regionserver shutdown improperly and leaves the dir in .old not
deleted (was: Regionserver shutdown impropery and leave the dir in .old not
delete.)
> Regionserver shutdown improperly and leaves the dir in .old not deleted
> ---
>
> Key: HBASE-10214
> URL: https://issues.apache.org/jira/browse/HBASE-10214
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.94.14
>Reporter: binlijin
> Attachments: HBASE-10214-94-V2.patch, HBASE-10214-94.patch
>
>
> RegionServer log
> {code}
> 2013-12-18 15:17:45,771 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegionServer: Waiting on
> 51b27391410efdca841db264df46085f
> 2013-12-18 15:17:45,776 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Connected to master at
> null
> 2013-12-18 15:17:48,776 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Exiting; cluster
> shutdown set and not carrying any regions
> 2013-12-18 15:17:48,776 FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server
> node,60020,1384410974572: Unhandled exception: null
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:880)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:753)
> at java.lang.Thread.run(Thread.java:662)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-6104) Require EXEC permission to call coprocessor endpoints
[
https://issues.apache.org/jira/browse/HBASE-6104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855354#comment-13855354
]
Hadoop QA commented on HBASE-6104:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12620110/6104.patch
against trunk revision .
ATTACHMENT ID: 12620110
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop
1.1 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithACL
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8260//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8260//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8260//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8260//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8260//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8260//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8260//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8260//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8260//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8260//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8260//console
This message is automatically generated.
> Require EXEC permission to call coprocessor endpoints
> -
>
> Key: HBASE-6104
> URL: https://issues.apache.org/jira/browse/HBASE-6104
> Project: HBase
> Issue Type: Sub-task
> Components: Coprocessors, security
>Reporter: Gary Helmling
>Assignee: Andrew Purtell
> Fix For: 0.98.0, 0.99.0
>
> Attachments: 6104.patch, 6104.patch, 6104.patch
>
>
> The EXEC action currently exists as only a placeholder in access control. It
> should really be used to enforce access to coprocessor endpoint RPC calls,
> which are currently unrestricted.
> How the ACLs to support this would be modeled deserves some discussion:
> * Should access be scoped to a specific table and CoprocessorProtocol
> extension?
> * Should it be possible to grant access to a CoprocessorProtocol
> implementation globally (regardless of table)?
> * Are per-method restrictions necessary?
> * Should we expose hooks available to endpoint implementors so that they
> could additionally apply their own permission checks? Some CP endpoints may
> want to require READ permissions, others may want to enforce WRITE, or READ +
> WRITE.
> To apply these kinds of checks we would also have to extend the
> RegionObserver interface to provide hooks wrapping HRegion.exec().
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10214) Regionserver shutdown impropery and leave the dir in .old not delete.
[
https://issues.apache.org/jira/browse/HBASE-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855352#comment-13855352
]
chunhui shen commented on HBASE-10214:
--
I see the above comment now that trunk don't need this patch,
Could you run the tests for 0.94 branch and upload the test result? Thanks
> Regionserver shutdown impropery and leave the dir in .old not delete.
> -
>
> Key: HBASE-10214
> URL: https://issues.apache.org/jira/browse/HBASE-10214
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.94.14
>Reporter: binlijin
> Attachments: HBASE-10214-94-V2.patch, HBASE-10214-94.patch
>
>
> RegionServer log
> {code}
> 2013-12-18 15:17:45,771 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegionServer: Waiting on
> 51b27391410efdca841db264df46085f
> 2013-12-18 15:17:45,776 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Connected to master at
> null
> 2013-12-18 15:17:48,776 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Exiting; cluster
> shutdown set and not carrying any regions
> 2013-12-18 15:17:48,776 FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server
> node,60020,1384410974572: Unhandled exception: null
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:880)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:753)
> at java.lang.Thread.run(Thread.java:662)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10209) Speed region assign in failover
[ https://issues.apache.org/jira/browse/HBASE-10209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855350#comment-13855350 ] chunhui shen commented on HBASE-10209: -- The improvement makes sense, but is there any risk on concurrent assignment, e.g. Server Shutdown event happen in the same time when assigning. Wait more body to take a see since I can't think the scenarios about assignment clearly. > Speed region assign in failover > --- > > Key: HBASE-10209 > URL: https://issues.apache.org/jira/browse/HBASE-10209 > Project: HBase > Issue Type: Improvement > Components: Region Assignment >Affects Versions: 0.94.14 >Reporter: binlijin > Attachments: HBASE-10209-94-2.patch, HBASE-10209-94.patch > > > During master initializate, if there are regionservers shutdown improperly > previous or start up failed, master will think this is a failover and assign > regions one by one. > We have a cluster with 200+ nodes and 32000+regions, it takes 50mins to start > up all. > The patch is based on 0.94 branch, and trunk is different now. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-7226) HRegion.checkAndMutate uses incorrect comparison result for <, <=, > and >=
[
https://issues.apache.org/jira/browse/HBASE-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855349#comment-13855349
]
Ted Yu commented on HBASE-7226:
---
lgtm
nit: long line:
{code}
+ // Test CompareOp.LESS_OR_EQUAL: original = val2, compare with val2,
succeed (value still = val2)
{code}
> HRegion.checkAndMutate uses incorrect comparison result for <, <=, > and >=
> ---
>
> Key: HBASE-7226
> URL: https://issues.apache.org/jira/browse/HBASE-7226
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Reporter: Feng Honghua
>Assignee: Feng Honghua
> Fix For: 0.94.16
>
> Attachments: HBASE-7226-trunk.patch, HRegion_HBASE_7226_0.94.2.patch
>
>
> in HRegion.checkAndMutate, incorrect comparison results are used for <, <=, >
> and >=, as below:
> switch (compareOp) {
> case LESS:
> matches = compareResult <= 0; // should be '<' here
> break;
> case LESS_OR_EQUAL:
> matches = compareResult < 0; // should be '<=' here
> break;
> case EQUAL:
> matches = compareResult == 0;
> break;
> case NOT_EQUAL:
> matches = compareResult != 0;
> break;
> case GREATER_OR_EQUAL:
> matches = compareResult > 0; // should be '>=' here
> break;
> case GREATER:
> matches = compareResult >= 0; // should be '>' here
> break;
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10214) Regionserver shutdown impropery and leave the dir in .old not delete.
[
https://issues.apache.org/jira/browse/HBASE-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855347#comment-13855347
]
chunhui shen commented on HBASE-10214:
--
Looks good to me, make a trunk patch and run HadoopQA?[~aoxiang]
> Regionserver shutdown impropery and leave the dir in .old not delete.
> -
>
> Key: HBASE-10214
> URL: https://issues.apache.org/jira/browse/HBASE-10214
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.94.14
>Reporter: binlijin
> Attachments: HBASE-10214-94-V2.patch, HBASE-10214-94.patch
>
>
> RegionServer log
> {code}
> 2013-12-18 15:17:45,771 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegionServer: Waiting on
> 51b27391410efdca841db264df46085f
> 2013-12-18 15:17:45,776 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Connected to master at
> null
> 2013-12-18 15:17:48,776 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Exiting; cluster
> shutdown set and not carrying any regions
> 2013-12-18 15:17:48,776 FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server
> node,60020,1384410974572: Unhandled exception: null
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:880)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:753)
> at java.lang.Thread.run(Thread.java:662)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-10214) Regionserver shutdown impropery and leave the dir in .old not delete.
[
https://issues.apache.org/jira/browse/HBASE-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
binlijin updated HBASE-10214:
-
Affects Version/s: 0.94.14
> Regionserver shutdown impropery and leave the dir in .old not delete.
> -
>
> Key: HBASE-10214
> URL: https://issues.apache.org/jira/browse/HBASE-10214
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.94.14
>Reporter: binlijin
> Attachments: HBASE-10214-94-V2.patch, HBASE-10214-94.patch
>
>
> RegionServer log
> {code}
> 2013-12-18 15:17:45,771 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegionServer: Waiting on
> 51b27391410efdca841db264df46085f
> 2013-12-18 15:17:45,776 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Connected to master at
> null
> 2013-12-18 15:17:48,776 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Exiting; cluster
> shutdown set and not carrying any regions
> 2013-12-18 15:17:48,776 FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server
> node,60020,1384410974572: Unhandled exception: null
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:880)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:753)
> at java.lang.Thread.run(Thread.java:662)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-10209) Speed region assign in failover
[ https://issues.apache.org/jira/browse/HBASE-10209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] binlijin updated HBASE-10209: - Affects Version/s: 0.94.14 > Speed region assign in failover > --- > > Key: HBASE-10209 > URL: https://issues.apache.org/jira/browse/HBASE-10209 > Project: HBase > Issue Type: Improvement > Components: Region Assignment >Affects Versions: 0.94.14 >Reporter: binlijin > Attachments: HBASE-10209-94-2.patch, HBASE-10209-94.patch > > > During master initializate, if there are regionservers shutdown improperly > previous or start up failed, master will think this is a failover and assign > regions one by one. > We have a cluster with 200+ nodes and 32000+regions, it takes 50mins to start > up all. > The patch is based on 0.94 branch, and trunk is different now. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HBASE-10209) Speed region assign in failover
[ https://issues.apache.org/jira/browse/HBASE-10209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] binlijin updated HBASE-10209: - Component/s: Region Assignment > Speed region assign in failover > --- > > Key: HBASE-10209 > URL: https://issues.apache.org/jira/browse/HBASE-10209 > Project: HBase > Issue Type: Improvement > Components: Region Assignment >Affects Versions: 0.94.14 >Reporter: binlijin > Attachments: HBASE-10209-94-2.patch, HBASE-10209-94.patch > > > During master initializate, if there are regionservers shutdown improperly > previous or start up failed, master will think this is a failover and assign > regions one by one. > We have a cluster with 200+ nodes and 32000+regions, it takes 50mins to start > up all. > The patch is based on 0.94 branch, and trunk is different now. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HBASE-10214) Regionserver shutdown impropery and leave the dir in .old not delete.
[
https://issues.apache.org/jira/browse/HBASE-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
binlijin updated HBASE-10214:
-
Component/s: regionserver
> Regionserver shutdown impropery and leave the dir in .old not delete.
> -
>
> Key: HBASE-10214
> URL: https://issues.apache.org/jira/browse/HBASE-10214
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.94.14
>Reporter: binlijin
> Attachments: HBASE-10214-94-V2.patch, HBASE-10214-94.patch
>
>
> RegionServer log
> {code}
> 2013-12-18 15:17:45,771 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegionServer: Waiting on
> 51b27391410efdca841db264df46085f
> 2013-12-18 15:17:45,776 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Connected to master at
> null
> 2013-12-18 15:17:48,776 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Exiting; cluster
> shutdown set and not carrying any regions
> 2013-12-18 15:17:48,776 FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server
> node,60020,1384410974572: Unhandled exception: null
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:880)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:753)
> at java.lang.Thread.run(Thread.java:662)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Comment Edited] (HBASE-9346) HBCK should provide an option to check if regions boundaries are the same in META and in stores.
[
https://issues.apache.org/jira/browse/HBASE-9346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855342#comment-13855342
]
Anoop Sam John edited comment on HBASE-9346 at 12/23/13 2:36 AM:
-
Oh... When u fixed the nested if the condition check become old one
again..(which was wrong)
{code}
if ((reader.getFirstKey() != null)
+ && ((storeFirstKey == null) ||
(comparator.compare(storeFirstKey,
+reader.getFirstKey()) < 0))) {
+storeFirstKey = reader.getFirstKey();
+ }
{code}
It should be <0
Can correct on commit
was (Author: anoop.hbase):
Oh... When u fixed the nested if the condition check become old one
again..(which was wrong)
{code}
if ((reader.getFirstKey() != null)
+ && ((storeFirstKey == null) ||
(comparator.compare(storeFirstKey,
+reader.getFirstKey()) < 0))) {
+storeFirstKey = reader.getFirstKey();
+ }
{code}
It should be <0
> HBCK should provide an option to check if regions boundaries are the same in
> META and in stores.
>
>
> Key: HBASE-9346
> URL: https://issues.apache.org/jira/browse/HBASE-9346
> Project: HBase
> Issue Type: Bug
> Components: hbck, Operability
>Affects Versions: 0.94.14, 0.98.1, 0.99.0, 0.96.1.1
>Reporter: Jean-Marc Spaggiari
>Assignee: Jean-Marc Spaggiari
> Attachments: HBASE-9346-v0-0.94.patch, HBASE-9346-v1-trunk.patch,
> HBASE-9346-v2-trunk.patch, HBASE-9346-v3-trunk.patch,
> HBASE-9346-v4-trunk.patch, HBASE-9346-v5-trunk.patch,
> HBASE-9346-v6-trunk.patch, HBASE-9346-v7-trunk.patch,
> HBASE-9346-v8-trunk.patch, HBASE-9346-v9-trunk.patch
>
>
> If META don't have the same region boundaries as the stores files, writes and
> read might go to the wrong place. We need to provide a way to check that
> withing HBCK.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-7226) HRegion.checkAndMutate uses incorrect comparison result for <, <=, > and >=
[
https://issues.apache.org/jira/browse/HBASE-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-7226:
-
Fix Version/s: 0.94.16
> HRegion.checkAndMutate uses incorrect comparison result for <, <=, > and >=
> ---
>
> Key: HBASE-7226
> URL: https://issues.apache.org/jira/browse/HBASE-7226
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Reporter: Feng Honghua
>Assignee: Feng Honghua
> Fix For: 0.94.16
>
> Attachments: HBASE-7226-trunk.patch, HRegion_HBASE_7226_0.94.2.patch
>
>
> in HRegion.checkAndMutate, incorrect comparison results are used for <, <=, >
> and >=, as below:
> switch (compareOp) {
> case LESS:
> matches = compareResult <= 0; // should be '<' here
> break;
> case LESS_OR_EQUAL:
> matches = compareResult < 0; // should be '<=' here
> break;
> case EQUAL:
> matches = compareResult == 0;
> break;
> case NOT_EQUAL:
> matches = compareResult != 0;
> break;
> case GREATER_OR_EQUAL:
> matches = compareResult > 0; // should be '>=' here
> break;
> case GREATER:
> matches = compareResult >= 0; // should be '>' here
> break;
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-10047) postScannerFilterRow consumes a lot of CPU in tall table scans
[ https://issues.apache.org/jira/browse/HBASE-10047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-10047: -- Attachment: 10047-0.94-poc-v4.txt Based on your hint [~apurtell]. Simply removes the endpoints (everything that does not implement RegionObserver). Looked into refactoring loadSystemCoprocessors into RegionCoprocessorHost, but would have led to a lot of cut'n'paste. What do you think. Will also performance test this, but this is generally cleaner I think. > postScannerFilterRow consumes a lot of CPU in tall table scans > -- > > Key: HBASE-10047 > URL: https://issues.apache.org/jira/browse/HBASE-10047 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl > Attachments: 10047-0.94-poc-v3.txt, 10047-0.94-poc-v4.txt, > 10047-0.94-sample-v2.txt, 10047-0.94-sample.txt, postScannerFilterRow.png > > > Continuing my profiling quest, I find that in scanning tall table (and > filtering everything on the server) a quarter of the time is now spent in the > postScannerFilterRow coprocessor hook. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-9346) HBCK should provide an option to check if regions boundaries are the same in META and in stores.
[
https://issues.apache.org/jira/browse/HBASE-9346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855342#comment-13855342
]
Anoop Sam John commented on HBASE-9346:
---
Oh... When u fixed the nested if the condition check become old one
again..(which was wrong)
{code}
if ((reader.getFirstKey() != null)
+ && ((storeFirstKey == null) ||
(comparator.compare(storeFirstKey,
+reader.getFirstKey()) < 0))) {
+storeFirstKey = reader.getFirstKey();
+ }
{code}
It should be <0
> HBCK should provide an option to check if regions boundaries are the same in
> META and in stores.
>
>
> Key: HBASE-9346
> URL: https://issues.apache.org/jira/browse/HBASE-9346
> Project: HBase
> Issue Type: Bug
> Components: hbck, Operability
>Affects Versions: 0.94.14, 0.98.1, 0.99.0, 0.96.1.1
>Reporter: Jean-Marc Spaggiari
>Assignee: Jean-Marc Spaggiari
> Attachments: HBASE-9346-v0-0.94.patch, HBASE-9346-v1-trunk.patch,
> HBASE-9346-v2-trunk.patch, HBASE-9346-v3-trunk.patch,
> HBASE-9346-v4-trunk.patch, HBASE-9346-v5-trunk.patch,
> HBASE-9346-v6-trunk.patch, HBASE-9346-v7-trunk.patch,
> HBASE-9346-v8-trunk.patch, HBASE-9346-v9-trunk.patch
>
>
> If META don't have the same region boundaries as the stores files, writes and
> read might go to the wrong place. We need to provide a way to check that
> withing HBCK.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-7226) HRegion.checkAndMutate uses incorrect comparison result for <, <=, > and >=
[
https://issues.apache.org/jira/browse/HBASE-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855344#comment-13855344
]
Lars Hofhansl commented on HBASE-7226:
--
I'll check the test, seems you were right all along. Thanks for persisting :)
> HRegion.checkAndMutate uses incorrect comparison result for <, <=, > and >=
> ---
>
> Key: HBASE-7226
> URL: https://issues.apache.org/jira/browse/HBASE-7226
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Reporter: Feng Honghua
>Assignee: Feng Honghua
> Fix For: 0.94.16
>
> Attachments: HBASE-7226-trunk.patch, HRegion_HBASE_7226_0.94.2.patch
>
>
> in HRegion.checkAndMutate, incorrect comparison results are used for <, <=, >
> and >=, as below:
> switch (compareOp) {
> case LESS:
> matches = compareResult <= 0; // should be '<' here
> break;
> case LESS_OR_EQUAL:
> matches = compareResult < 0; // should be '<=' here
> break;
> case EQUAL:
> matches = compareResult == 0;
> break;
> case NOT_EQUAL:
> matches = compareResult != 0;
> break;
> case GREATER_OR_EQUAL:
> matches = compareResult > 0; // should be '>=' here
> break;
> case GREATER:
> matches = compareResult >= 0; // should be '>' here
> break;
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-7226) HRegion.checkAndMutate uses incorrect comparison result for <, <=, > and >=
[
https://issues.apache.org/jira/browse/HBASE-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855340#comment-13855340
]
Feng Honghua commented on HBASE-7226:
-
provide fix and unit-test to prove it for trunk. but this bug should exist in
all branches(I think), please branch owners check and apply accordingly
> HRegion.checkAndMutate uses incorrect comparison result for <, <=, > and >=
> ---
>
> Key: HBASE-7226
> URL: https://issues.apache.org/jira/browse/HBASE-7226
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Reporter: Feng Honghua
>Assignee: Feng Honghua
> Attachments: HBASE-7226-trunk.patch, HRegion_HBASE_7226_0.94.2.patch
>
>
> in HRegion.checkAndMutate, incorrect comparison results are used for <, <=, >
> and >=, as below:
> switch (compareOp) {
> case LESS:
> matches = compareResult <= 0; // should be '<' here
> break;
> case LESS_OR_EQUAL:
> matches = compareResult < 0; // should be '<=' here
> break;
> case EQUAL:
> matches = compareResult == 0;
> break;
> case NOT_EQUAL:
> matches = compareResult != 0;
> break;
> case GREATER_OR_EQUAL:
> matches = compareResult > 0; // should be '>=' here
> break;
> case GREATER:
> matches = compareResult >= 0; // should be '>' here
> break;
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-7226) HRegion.checkAndMutate uses incorrect comparison result for <, <=, > and >=
[
https://issues.apache.org/jira/browse/HBASE-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Honghua updated HBASE-7226:
Attachment: HBASE-7226-trunk.patch
patch and unit-test for trunk
> HRegion.checkAndMutate uses incorrect comparison result for <, <=, > and >=
> ---
>
> Key: HBASE-7226
> URL: https://issues.apache.org/jira/browse/HBASE-7226
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Reporter: Feng Honghua
>Assignee: Feng Honghua
> Attachments: HBASE-7226-trunk.patch, HRegion_HBASE_7226_0.94.2.patch
>
>
> in HRegion.checkAndMutate, incorrect comparison results are used for <, <=, >
> and >=, as below:
> switch (compareOp) {
> case LESS:
> matches = compareResult <= 0; // should be '<' here
> break;
> case LESS_OR_EQUAL:
> matches = compareResult < 0; // should be '<=' here
> break;
> case EQUAL:
> matches = compareResult == 0;
> break;
> case NOT_EQUAL:
> matches = compareResult != 0;
> break;
> case GREATER_OR_EQUAL:
> matches = compareResult > 0; // should be '>=' here
> break;
> case GREATER:
> matches = compareResult >= 0; // should be '>' here
> break;
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-7226) HRegion.checkAndMutate uses incorrect comparison result for <, <=, > and >=
[
https://issues.apache.org/jira/browse/HBASE-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Honghua updated HBASE-7226:
Priority: Major (was: Minor)
Environment: (was: 0.94.2)
Affects Version/s: (was: 0.94.2)
Assignee: Feng Honghua
Remaining Estimate: (was: 10m)
Original Estimate: (was: 10m)
> HRegion.checkAndMutate uses incorrect comparison result for <, <=, > and >=
> ---
>
> Key: HBASE-7226
> URL: https://issues.apache.org/jira/browse/HBASE-7226
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Reporter: Feng Honghua
>Assignee: Feng Honghua
> Attachments: HRegion_HBASE_7226_0.94.2.patch
>
>
> in HRegion.checkAndMutate, incorrect comparison results are used for <, <=, >
> and >=, as below:
> switch (compareOp) {
> case LESS:
> matches = compareResult <= 0; // should be '<' here
> break;
> case LESS_OR_EQUAL:
> matches = compareResult < 0; // should be '<=' here
> break;
> case EQUAL:
> matches = compareResult == 0;
> break;
> case NOT_EQUAL:
> matches = compareResult != 0;
> break;
> case GREATER_OR_EQUAL:
> matches = compareResult > 0; // should be '>=' here
> break;
> case GREATER:
> matches = compareResult >= 0; // should be '>' here
> break;
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Reopened] (HBASE-7226) HRegion.checkAndMutate uses incorrect comparison result for <, <=, > and >=
[
https://issues.apache.org/jira/browse/HBASE-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Honghua reopened HBASE-7226:
-
provide fix and unit-test
> HRegion.checkAndMutate uses incorrect comparison result for <, <=, > and >=
> ---
>
> Key: HBASE-7226
> URL: https://issues.apache.org/jira/browse/HBASE-7226
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Reporter: Feng Honghua
>Assignee: Feng Honghua
> Attachments: HRegion_HBASE_7226_0.94.2.patch
>
>
> in HRegion.checkAndMutate, incorrect comparison results are used for <, <=, >
> and >=, as below:
> switch (compareOp) {
> case LESS:
> matches = compareResult <= 0; // should be '<' here
> break;
> case LESS_OR_EQUAL:
> matches = compareResult < 0; // should be '<=' here
> break;
> case EQUAL:
> matches = compareResult == 0;
> break;
> case NOT_EQUAL:
> matches = compareResult != 0;
> break;
> case GREATER_OR_EQUAL:
> matches = compareResult > 0; // should be '>=' here
> break;
> case GREATER:
> matches = compareResult >= 0; // should be '>' here
> break;
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-9151) HBCK cannot fix when meta server znode deleted, this can happen if all region servers stopped and there are no logs to split.
[
https://issues.apache.org/jira/browse/HBASE-9151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
rajeshbabu updated HBASE-9151:
--
Resolution: Fixed
Fix Version/s: 0.96.2
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed to trunk, 0.98 and 0.96.
Thanks for review Ram.
> HBCK cannot fix when meta server znode deleted, this can happen if all region
> servers stopped and there are no logs to split.
> -
>
> Key: HBASE-9151
> URL: https://issues.apache.org/jira/browse/HBASE-9151
> Project: HBase
> Issue Type: Bug
> Components: hbck
>Reporter: rajeshbabu
>Assignee: rajeshbabu
> Fix For: 0.98.0, 0.96.2, 0.99.0
>
> Attachments: HBASE-9151.patch, HBASE-9151_v2.patch,
> HBASE-9151_v2.patch
>
>
> When meta server znode deleted and meta in FAILED_OPEN state, then hbck
> cannot fix it. This scenario can come when all region servers stopped by stop
> command and didnt start any RS within 10 secs(with default configurations).
> {code}
> public void assignMeta() throws KeeperException {
> MetaRegionTracker.deleteMetaLocation(this.watcher);
> assign(HRegionInfo.FIRST_META_REGIONINFO, true);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-6104) Require EXEC permission to call coprocessor endpoints
[ https://issues.apache.org/jira/browse/HBASE-6104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-6104: -- Fix Version/s: 0.99.0 Affects Version/s: (was: 0.95.2) Status: Patch Available (was: Open) > Require EXEC permission to call coprocessor endpoints > - > > Key: HBASE-6104 > URL: https://issues.apache.org/jira/browse/HBASE-6104 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors, security >Reporter: Gary Helmling >Assignee: Andrew Purtell > Fix For: 0.98.0, 0.99.0 > > Attachments: 6104.patch, 6104.patch, 6104.patch > > > The EXEC action currently exists as only a placeholder in access control. It > should really be used to enforce access to coprocessor endpoint RPC calls, > which are currently unrestricted. > How the ACLs to support this would be modeled deserves some discussion: > * Should access be scoped to a specific table and CoprocessorProtocol > extension? > * Should it be possible to grant access to a CoprocessorProtocol > implementation globally (regardless of table)? > * Are per-method restrictions necessary? > * Should we expose hooks available to endpoint implementors so that they > could additionally apply their own permission checks? Some CP endpoints may > want to require READ permissions, others may want to enforce WRITE, or READ + > WRITE. > To apply these kinds of checks we would also have to extend the > RegionObserver interface to provide hooks wrapping HRegion.exec(). -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HBASE-6104) Require EXEC permission to call coprocessor endpoints
[ https://issues.apache.org/jira/browse/HBASE-6104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-6104: -- Attachment: 6104.patch Latest patch includes new unit test in TestAccessController for checking EXEC privilege. > Require EXEC permission to call coprocessor endpoints > - > > Key: HBASE-6104 > URL: https://issues.apache.org/jira/browse/HBASE-6104 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors, security >Affects Versions: 0.95.2 >Reporter: Gary Helmling >Assignee: Andrew Purtell > Fix For: 0.98.0 > > Attachments: 6104.patch, 6104.patch, 6104.patch > > > The EXEC action currently exists as only a placeholder in access control. It > should really be used to enforce access to coprocessor endpoint RPC calls, > which are currently unrestricted. > How the ACLs to support this would be modeled deserves some discussion: > * Should access be scoped to a specific table and CoprocessorProtocol > extension? > * Should it be possible to grant access to a CoprocessorProtocol > implementation globally (regardless of table)? > * Are per-method restrictions necessary? > * Should we expose hooks available to endpoint implementors so that they > could additionally apply their own permission checks? Some CP endpoints may > want to require READ permissions, others may want to enforce WRITE, or READ + > WRITE. > To apply these kinds of checks we would also have to extend the > RegionObserver interface to provide hooks wrapping HRegion.exec(). -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10142) TestLogRolling#testLogRollOnDatanodeDeath test failure
[
https://issues.apache.org/jira/browse/HBASE-10142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855328#comment-13855328
]
Hudson commented on HBASE-10142:
SUCCESS: Integrated in hbase-0.96 #238 (See
[https://builds.apache.org/job/hbase-0.96/238/])
HBASE-10218 Port HBASE-10142 'TestLogRolling#testLogRollOnDatanodeDeath test
failure' to 0.96 branch (tedyu: rev 1553044)
*
/hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java
*
/hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java
> TestLogRolling#testLogRollOnDatanodeDeath test failure
> --
>
> Key: HBASE-10142
> URL: https://issues.apache.org/jira/browse/HBASE-10142
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.0, 0.99.0
>Reporter: Andrew Purtell
>Assignee: Ted Yu
> Fix For: 0.98.0, 0.99.0
>
> Attachments: 10142-v1.txt
>
>
> This is a demanding unit test, which fails fairly often as software versions
> (JVM, Hadoop) and system load change. Currently when testing 0.98 branch I
> see this failure:
> {noformat}
> Failed tests:
> testLogRollOnDatanodeDeath(org.apache.hadoop.hbase.regionserver.wal.TestLogRolling):
> LowReplication Roller should've been disabled, current replication=1
> {noformat}
> Could be a timing issue after the recent switch to Hadoop 2 as default
> build/test profile. Let's see if more leniency makes sense and if it can
> stabilize the test before disabling it.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10218) Port HBASE-10142 'TestLogRolling#testLogRollOnDatanodeDeath test failure' to 0.96 branch
[ https://issues.apache.org/jira/browse/HBASE-10218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855329#comment-13855329 ] Hudson commented on HBASE-10218: SUCCESS: Integrated in hbase-0.96 #238 (See [https://builds.apache.org/job/hbase-0.96/238/]) HBASE-10218 Port HBASE-10142 'TestLogRolling#testLogRollOnDatanodeDeath test failure' to 0.96 branch (tedyu: rev 1553044) * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java > Port HBASE-10142 'TestLogRolling#testLogRollOnDatanodeDeath test failure' to > 0.96 branch > > > Key: HBASE-10218 > URL: https://issues.apache.org/jira/browse/HBASE-10218 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 0.96.2 > > Attachments: 10218-v1.txt, 10218-v2.txt > > > TestLogRolling#testLogRollOnDatanodeDeath used to fail quite often. > This is fixed by HBASE-10142 in 0.98 and above. > Toward the end of HBASE-10142, Jonathan and Enis proposed porting the fix to > 0.96 and Stack ack'ed. > This issue ports the fix to 0.96 branch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10047) postScannerFilterRow consumes a lot of CPU in tall table scans
[ https://issues.apache.org/jira/browse/HBASE-10047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855327#comment-13855327 ] Lars Hofhansl commented on HBASE-10047: --- You're saying we catch ClassCastException and then remove the coprocessor? Looking at the code a bit more; in the RegionCoprocessorHost case we add Endpoint to the list of coprocessors, which does not seem to be necessary. Unfortunately only the tableCoprocessors are loaded in RegionCoprocessorHost, the systemCoprocessors are loaded in CoprocessorHost. Could refactor that, or simply just remove the Endpoint from the set of coprocessors, it seems they need (and should?) not to in there. > postScannerFilterRow consumes a lot of CPU in tall table scans > -- > > Key: HBASE-10047 > URL: https://issues.apache.org/jira/browse/HBASE-10047 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl > Attachments: 10047-0.94-poc-v3.txt, 10047-0.94-sample-v2.txt, > 10047-0.94-sample.txt, postScannerFilterRow.png > > > Continuing my profiling quest, I find that in scanning tall table (and > filtering everything on the server) a quarter of the time is now spent in the > postScannerFilterRow coprocessor hook. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10223) [VisibilityController] cellVisibility presence check on Delete mutation is wrong
[ https://issues.apache.org/jira/browse/HBASE-10223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855325#comment-13855325 ] Hudson commented on HBASE-10223: FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #18 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/18/]) HBASE-10223 [VisibilityController] cellVisibility presence check on Delete mutation is wrong (anoopsamjohn: rev 1553014) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java > [VisibilityController] cellVisibility presence check on Delete mutation is > wrong > > > Key: HBASE-10223 > URL: https://issues.apache.org/jira/browse/HBASE-10223 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.0 >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 0.98.0, 0.99.0 > > Attachments: HBASE-10223.patch > > > cellVisibility presence check should be performed against null. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10161) [AccessController] Tolerate regions in recovery
[ https://issues.apache.org/jira/browse/HBASE-10161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855316#comment-13855316 ] Hudson commented on HBASE-10161: SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #25 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/25/]) HBASE-10161 [AccessController] Tolerate regions in recovery (anoopsamjohn: rev 1552749) * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java > [AccessController] Tolerate regions in recovery > --- > > Key: HBASE-10161 > URL: https://issues.apache.org/jira/browse/HBASE-10161 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.0 >Reporter: Andrew Purtell >Assignee: Anoop Sam John >Priority: Blocker > Fix For: 0.98.0, 0.96.2, 0.99.0 > > Attachments: HBASE-10161.patch, HBASE-10161_V2.patch, > HBASE-10161_V3.patch > > > AccessController fixes for the issue also affecting VisibilityController > described on HBASE-10148. Coprocessors that initialize in postOpen upcalls > must check if the region is still in recovery and defer initialization until > recovery is complete. We need to add a new CP hook for post recovery upcalls > and modify existing CPs to defer initialization until this new hook as needed. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10173) Need HFile version check in security coprocessors
[ https://issues.apache.org/jira/browse/HBASE-10173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855313#comment-13855313 ] Hudson commented on HBASE-10173: SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #25 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/25/]) HBASE-10173 Need HFile version check in security coprocessors (anoopsamjohn: rev 1552727) * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java > Need HFile version check in security coprocessors > - > > Key: HBASE-10173 > URL: https://issues.apache.org/jira/browse/HBASE-10173 > Project: HBase > Issue Type: Improvement > Components: security >Affects Versions: 0.98.0, 0.99.0 >Reporter: Anoop Sam John >Assignee: Andrew Purtell >Priority: Critical > Fix For: 0.98.0, 0.99.0 > > Attachments: 10173.patch, 10173.patch, HBASE-10173.patch, > HBASE-10173_Addendum.patch, HBASE-10173_partial.patch > > > Cell level visibility labels are stored as cell tags. So HFile V3 is the > minimum version which can support this feature. Better to have a version > check in VisibilityController. Some one using this CP but with any HFile > version as V2, we can better throw error. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10195) "mvn site" build failed with OutOfMemoryError
[ https://issues.apache.org/jira/browse/HBASE-10195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855311#comment-13855311 ] Hudson commented on HBASE-10195: SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #25 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/25/]) HBASE-10195: mvn site build failed with OutOfMemoryError (jeffreyz: rev 1552813) * /hbase/branches/0.98/pom.xml > "mvn site" build failed with OutOfMemoryError > - > > Key: HBASE-10195 > URL: https://issues.apache.org/jira/browse/HBASE-10195 > Project: HBase > Issue Type: Bug >Affects Versions: 0.96.1 >Reporter: Jeffrey Zhong >Assignee: Jeffrey Zhong > Fix For: 0.98.0, 0.96.2 > > Attachments: hbase-10195.patch > > > "mvn site" build command failed with OutOfMemoryError. It lasts for a while > and our book reference guid seems out of date as well. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10219) HTTPS support for HBase in RegionServerListTmpl.jamon
[
https://issues.apache.org/jira/browse/HBASE-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855317#comment-13855317
]
Hudson commented on HBASE-10219:
SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #25 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/25/])
HBASE-10219 HTTPS support for HBase in RegionServerListTmpl.jamon (tedyu: rev
1552881)
*
/hbase/branches/0.98/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/RegionServerListTmpl.jamon
> HTTPS support for HBase in RegionServerListTmpl.jamon
> -
>
> Key: HBASE-10219
> URL: https://issues.apache.org/jira/browse/HBASE-10219
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 0.98.0, 0.96.2, 0.99.0
>
> Attachments: 10219.txt
>
>
> HBASE-9954 added https support but missed the following occurrence in
> RegionServerListTmpl.jamon:
> {code}
> String url = "http://"; + serverName.getHostname() + ":" + infoPort +
> "/";
> {code}
> Thanks to Benoy Antony who discovered this issue.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-8529) checkOpen is missing from multi, mutate, get and multiGet etc.
[ https://issues.apache.org/jira/browse/HBASE-8529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855318#comment-13855318 ] Hudson commented on HBASE-8529: --- SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #25 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/25/]) hbase-8529: checkOpen is missing from multi, mutate, get and multiGet etc. (jeffreyz: rev 1552791) * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java > checkOpen is missing from multi, mutate, get and multiGet etc. > -- > > Key: HBASE-8529 > URL: https://issues.apache.org/jira/browse/HBASE-8529 > Project: HBase > Issue Type: Bug >Reporter: Jeffrey Zhong >Assignee: Jeffrey Zhong >Priority: Minor > Fix For: 0.98.0, 0.99.0 > > Attachments: hbase-8529.patch > > > I saw we have checkOpen in all those functions in 0.94 while they're missing > from trunk. Does anyone know why? > For multi and mutate, if we don't call checkOpen we could flood our logs with > bunch of "DFSOutputStream is closed" errors when we sync WAL. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10193) Cleanup HRegion if one of the store fails to open at region initialization
[ https://issues.apache.org/jira/browse/HBASE-10193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855310#comment-13855310 ] Hudson commented on HBASE-10193: SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #25 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/25/]) HBASE-10193 Cleanup HRegion if one of the store fails to open at region initialization (Aditya) (anoopsamjohn: rev 1552736) * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java > Cleanup HRegion if one of the store fails to open at region initialization > -- > > Key: HBASE-10193 > URL: https://issues.apache.org/jira/browse/HBASE-10193 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.96.1, 0.94.14 >Reporter: Aditya Kishore >Assignee: Aditya Kishore >Priority: Critical > Fix For: 0.98.0, 0.94.16, 0.96.2, 0.99.0 > > Attachments: HBASE-10193.patch, HBASE-10193_0.94.patch, > HBASE-10193_0.94_v2.patch, HBASE-10193_0.94_v3.patch, > HBASE-10193_0.94_v4.patch, HBASE-10193_v2.patch, HBASE-10193_v3.patch, > HBASE-10193_v4.patch > > > While investigating a different issue, I realized that the fix for HBASE-9737 > is not sufficient to prevent resource leak if a region fails to open for some > reason, say a corrupt HFile. > The region may have, by then, opened other good HFiles in that store or other > stores if it has more than one column family and their streams may leak if > not closed. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10221) Region from coprocessor invocations can be null on failure
[
https://issues.apache.org/jira/browse/HBASE-10221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855314#comment-13855314
]
Hudson commented on HBASE-10221:
SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #25 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/25/])
HBASE-10221. Region from coprocessor invocations can be null on failure
(apurtell: rev 1552968)
*
/hbase/branches/0.98/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
> Region from coprocessor invocations can be null on failure
> --
>
> Key: HBASE-10221
> URL: https://issues.apache.org/jira/browse/HBASE-10221
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.0, 0.96.2, 0.99.0
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Fix For: 0.98.0, 0.96.2, 0.99.0
>
> Attachments: 10221-v2.patch, 10221.patch
>
>
> In a visibility controller unit test, we fail a VisibilityClient.getAuths
> call with an AccessDeniedException, leading to a NPE in the client:
> {noformat}
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hbase.util.Bytes.compareTo(Bytes.java:1017)
> at
> org.apache.hadoop.hbase.util.Bytes$ByteArrayComparator.compare(Bytes.java:151)
> at
> org.apache.hadoop.hbase.util.Bytes$ByteArrayComparator.compare(Bytes.java:140)
> at java.util.TreeMap.compare(TreeMap.java:1188)
> at java.util.TreeMap.put(TreeMap.java:531)
> at java.util.Collections$SynchronizedMap.put(Collections.java:2041)
> at org.apache.hadoop.hbase.client.HTable$15.update(HTable.java:1486)
> at org.apache.hadoop.hbase.client.HTable$16.call(HTable.java:1515)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10223) [VisibilityController] cellVisibility presence check on Delete mutation is wrong
[ https://issues.apache.org/jira/browse/HBASE-10223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855319#comment-13855319 ] Hudson commented on HBASE-10223: SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #25 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/25/]) HBASE-10223 [VisibilityController] cellVisibility presence check on Delete mutation is wrong (anoopsamjohn: rev 1553013) * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java > [VisibilityController] cellVisibility presence check on Delete mutation is > wrong > > > Key: HBASE-10223 > URL: https://issues.apache.org/jira/browse/HBASE-10223 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.0 >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Critical > Fix For: 0.98.0, 0.99.0 > > Attachments: HBASE-10223.patch > > > cellVisibility presence check should be performed against null. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-8701) distributedLogReplay need to apply wal edits in the receiving order of those edits
[ https://issues.apache.org/jira/browse/HBASE-8701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855315#comment-13855315 ] Hudson commented on HBASE-8701: --- SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #25 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/25/]) HBASE-8701: distributedLogReplay need to apply wal edits in the receiving order of those edits (jeffreyz: rev 1552831) * /hbase/branches/0.98/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java * /hbase/branches/0.98/hbase-common/src/main/java/org/apache/hadoop/hbase/Tag.java * /hbase/branches/0.98/hbase-common/src/main/java/org/apache/hadoop/hbase/TagType.java * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/master/RegionStates.java * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/ServerShutdownHandler.java * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SplitLogWorker.java * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityUtils.java * /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java * /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java > distributedLogReplay need to apply wal edits in the receiving order of those > edits > -- > > Key: HBASE-8701 > URL: https://issues.apache.org/jira/browse/HBASE-8701 > Project: HBase > Issue Type: Bug > Components: MTTR >Reporter: Jeffrey Zhong >Assignee: Jeffrey Zhong > Fix For: 0.98.0, 0.99.0 > > Attachments: 8701-v3.txt, hbase-8701-tag-v1.patch, > hbase-8701-tag-v2-update.patch, hbase-8701-tag-v2.patch, > hbase-8701-tag.patch, hbase-8701-v4.patch, hbase-8701-v5.patch, > hbase-8701-v6.patch, hbase-8701-v7.patch, hbase-8701-v8.patch > > > This issue happens in distributedLogReplay mode when recovering multiple puts > of the same key + version(timestamp). After replay, the value is > nondeterministic of the key > h5. The original concern situation raised from [~eclark]: > For all edits the rowkey is the same. > There's a log with: [ A (ts = 0), B (ts = 0) ] > Replay the first half of the log. > A user puts in C (ts = 0) > Memstore has to flush > A new Hfile will be created with [ C, A ] and MaxSequenceId = C's seqid. > Replay the rest of the Log. > Flush > The issue will happen in similar situation like Put(key, t=T) in WAL1 and > Put(key,t=T) in WAL2 > h5. Below is the option(proposed by Ted) I'd like to use: > a) During replay, we pass original wal sequence number of each edit to the > receiving RS > b) In receiving RS, we store negative original sequence number of wal edits > into mvcc field of KVs of wal edits > c) Add handling of negative MVCC in KVScannerComparator and KVComparator > d) In receiving RS, write original sequence number into an optional field of > wal file for chained RS failure situation > e) When opening a region, we add a safety bumper(a large number) in order for > the new sequence number of a newly opened region not to collide with old > sequence numbers. > In the future, when we stores sequence number along with KVs, we can adjust > the above solution a little bit by avoiding to overload MVCC field. > h5. The other alternative options are listed below for references: > Option one > a) disallow writes during recovery > b) during replay, we pass original wal sequence ids > c) hold flush till all wals of a recovering region are replayed. Memstore > should hold because we only recover unflushed wal edits. For edits with same > key + version, whichever with larger sequence Id wins. > Option two > a) During replay, we pass original wal sequence ids > b) for each wal edit, we store each edit's original sequence id along with > its key. > c) during scanning, we use the original sequenc
[jira] [Commented] (HBASE-10220) Put all test service principals into the superusers list
[
https://issues.apache.org/jira/browse/HBASE-10220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855312#comment-13855312
]
Hudson commented on HBASE-10220:
SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #25 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/25/])
HBASE-10220. Put all test service principals into the superusers list
(apurtell: rev 1552948)
*
/hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java
*
/hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabelsWithACL.java
> Put all test service principals into the superusers list
>
>
> Key: HBASE-10220
> URL: https://issues.apache.org/jira/browse/HBASE-10220
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.0, 0.96.2, 0.99.0
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Fix For: 0.98.0, 0.96.2, 0.99.0
>
> Attachments: 10220-0.96.patch, 10220.patch
>
>
> TestVisibilityLabelsWithACL is not setting up the secure minicluster
> correctly.
> You can see the issue here:
> {noformat}
> 2013-12-20 18:25:44,979 WARN [htable-pool42-t1] client.AsyncProcess(664):
> #57, table=hbase:meta, attempt=1/350 failed 1 ops, last exception:
> org.apache.hadoop.hbase.security.AccessDeniedException:
> org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient
> permissions (user=apurtell.hfs.1, scope=hbase:meta,
> family=info:server|info:serverstartcode|info:seqnumDuringOpen, action=WRITE)
> at
> org.apache.hadoop.hbase.security.access.AccessController.requireCoveringPermission(AccessController.java:567)
> at
> org.apache.hadoop.hbase.security.access.AccessController.prePut(AccessController.java:1139)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.prePut(RegionCoprocessorHost.java:964)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doPreMutationHook(HRegion.java:2131)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2106)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2062)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2066)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.doBatchOp(HRegionServer.java:4206)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.doNonAtomicRegionMutation(HRegionServer.java:3448)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3352)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:28460)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2008)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:92)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:160)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
> at java.lang.Thread.run(Thread.java:744)
> on acer,47702,1387592735459, tracking started Fri Dec 20 18:25:44 PST 2013 -
> FAILED, NOT RETRYING ANYMORE
> 2013-12-20 18:25:44,980 DEBUG
> [PostOpenDeployTasks:bdd47ebb21d762590edf81127c8cd30f] client.HTable(949):
> hbase:meta: One or more of the operations have failed - waiting for all
> operation in progress to finish (successfully or not)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10142) TestLogRolling#testLogRollOnDatanodeDeath test failure
[
https://issues.apache.org/jira/browse/HBASE-10142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855308#comment-13855308
]
Hudson commented on HBASE-10142:
FAILURE: Integrated in hbase-0.96-hadoop2 #160 (See
[https://builds.apache.org/job/hbase-0.96-hadoop2/160/])
HBASE-10218 Port HBASE-10142 'TestLogRolling#testLogRollOnDatanodeDeath test
failure' to 0.96 branch (tedyu: rev 1553044)
*
/hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java
*
/hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java
> TestLogRolling#testLogRollOnDatanodeDeath test failure
> --
>
> Key: HBASE-10142
> URL: https://issues.apache.org/jira/browse/HBASE-10142
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.0, 0.99.0
>Reporter: Andrew Purtell
>Assignee: Ted Yu
> Fix For: 0.98.0, 0.99.0
>
> Attachments: 10142-v1.txt
>
>
> This is a demanding unit test, which fails fairly often as software versions
> (JVM, Hadoop) and system load change. Currently when testing 0.98 branch I
> see this failure:
> {noformat}
> Failed tests:
> testLogRollOnDatanodeDeath(org.apache.hadoop.hbase.regionserver.wal.TestLogRolling):
> LowReplication Roller should've been disabled, current replication=1
> {noformat}
> Could be a timing issue after the recent switch to Hadoop 2 as default
> build/test profile. Let's see if more leniency makes sense and if it can
> stabilize the test before disabling it.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10218) Port HBASE-10142 'TestLogRolling#testLogRollOnDatanodeDeath test failure' to 0.96 branch
[ https://issues.apache.org/jira/browse/HBASE-10218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855309#comment-13855309 ] Hudson commented on HBASE-10218: FAILURE: Integrated in hbase-0.96-hadoop2 #160 (See [https://builds.apache.org/job/hbase-0.96-hadoop2/160/]) HBASE-10218 Port HBASE-10142 'TestLogRolling#testLogRollOnDatanodeDeath test failure' to 0.96 branch (tedyu: rev 1553044) * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java * /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java > Port HBASE-10142 'TestLogRolling#testLogRollOnDatanodeDeath test failure' to > 0.96 branch > > > Key: HBASE-10218 > URL: https://issues.apache.org/jira/browse/HBASE-10218 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Fix For: 0.96.2 > > Attachments: 10218-v1.txt, 10218-v2.txt > > > TestLogRolling#testLogRollOnDatanodeDeath used to fail quite often. > This is fixed by HBASE-10142 in 0.98 and above. > Toward the end of HBASE-10142, Jonathan and Enis proposed porting the fix to > 0.96 and Stack ack'ed. > This issue ports the fix to 0.96 branch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
