[jira] [Commented] (HBASE-9900) Fix unintended byte[].toString in AccessController
[ https://issues.apache.org/jira/browse/HBASE-9900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13817029#comment-13817029 ] Hudson commented on HBASE-9900: --- SUCCESS: Integrated in HBase-TRUNK #4673 (See [https://builds.apache.org/job/HBase-TRUNK/4673/]) HBASE-9900. Fix unintended byte[].toString in AccessController (apurtell: rev 1539882) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/TableAuthManager.java > Fix unintended byte[].toString in AccessController > -- > > Key: HBASE-9900 > URL: https://issues.apache.org/jira/browse/HBASE-9900 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.0, 0.96.1 >Reporter: Andrew Purtell >Assignee: Andrew Purtell > Fix For: 0.98.0, 0.96.1 > > Attachments: 9900.patch > > > Found while running FindBugs for another change. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9915) Severe performance bug: isSeeked() in EncodedScannerV2 is always false
[ https://issues.apache.org/jira/browse/HBASE-9915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13817032#comment-13817032 ] stack commented on HBASE-9915: -- It is 9 lines two of which are empty lines and a third is '@Override'. Nice numbers. > Severe performance bug: isSeeked() in EncodedScannerV2 is always false > -- > > Key: HBASE-9915 > URL: https://issues.apache.org/jira/browse/HBASE-9915 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.98.0, 0.96.1, 0.94.14 > > Attachments: 9915-0.94.txt, 9915-trunk-v2.txt, 9915-trunk-v2.txt, > 9915-trunk.txt, profile.png > > > While debugging why reseek is so slow I found that it is quite broken for > encoded scanners. > The problem is this: > AbstractScannerV2.reseekTo(...) calls isSeeked() to check whether scanner was > seeked or not. If it was it checks whether the KV we want to seek to is in > the current block, if not it always consults the index blocks again. > isSeeked checks the blockBuffer member, which is not used by EncodedScannerV2 > and thus always returns false, which in turns causes an index lookup for each > reseek. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9117) Remove HTablePool and all HConnection pooling related APIs
[ https://issues.apache.org/jira/browse/HBASE-9117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-9117: Attachment: HBASE-9117.00.patch Taking a crack at deleting some code. [~lhofhansl], please pick apart my decisions regarding what to delete. > Remove HTablePool and all HConnection pooling related APIs > -- > > Key: HBASE-9117 > URL: https://issues.apache.org/jira/browse/HBASE-9117 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.98.0 > > Attachments: HBASE-9117.00.patch > > > The recommended way is now: > # Create an HConnection: HConnectionManager.createConnection(...) > # Create a light HTable: HConnection.getTable(...) > # table.close() > # connection.close() > All other API and pooling will be removed. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9117) Remove HTablePool and all HConnection pooling related APIs
[ https://issues.apache.org/jira/browse/HBASE-9117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-9117: Status: Patch Available (was: Open) > Remove HTablePool and all HConnection pooling related APIs > -- > > Key: HBASE-9117 > URL: https://issues.apache.org/jira/browse/HBASE-9117 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.98.0 > > Attachments: HBASE-9117.00.patch > > > The recommended way is now: > # Create an HConnection: HConnectionManager.createConnection(...) > # Create a light HTable: HConnection.getTable(...) > # table.close() > # connection.close() > All other API and pooling will be removed. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9906) Restore snapshot fails to restore the meta edits sporadically
[
https://issues.apache.org/jira/browse/HBASE-9906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13817042#comment-13817042
]
Hudson commented on HBASE-9906:
---
SUCCESS: Integrated in HBase-0.94-security #330 (See
[https://builds.apache.org/job/HBase-0.94-security/330/])
HBASE-9906 Restore snapshot fails to restore the meta edits sporadically (enis:
rev 1539910)
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/catalog/MetaEditor.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/RestoreSnapshotHandler.java
> Restore snapshot fails to restore the meta edits sporadically
> ---
>
> Key: HBASE-9906
> URL: https://issues.apache.org/jira/browse/HBASE-9906
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.98.0, 0.96.1, 0.94.14
>
> Attachments: hbase-9906-0.94_v1.patch, hbase-9906_v1.patch
>
>
> After snaphot restore, we see failures to find the table in meta:
> {code}
> > disable 'tablefour'
> > restore_snapshot 'snapshot_tablefour'
> > enable 'tablefour'
> ERROR: Table tablefour does not exist.'
> {code}
> This is quite subtle. From the looks of it, we successfully restore the
> snapshot, do the meta updates, return to the client about the status. The
> client then tries to do an operation for the table (like enable table, or
> scan in the test outputs) which fails because the meta entry for the region
> seems to be gone (in case of single region, the table will be reported
> missing). Subsequent attempts for creating the table will also fail because
> the table directories will be there, but not the meta entries.
> For restoring meta entries, we are doing a delete then a put to the same
> region:
> {code}
> 2013-11-04 10:39:51,582 INFO
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper: region to restore:
> 76d0e2b7ec3291afcaa82e18a56ccc30
> 2013-11-04 10:39:51,582 INFO
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper: region to remove:
> fa41edf43fe3ee131db4a34b848ff432
> ...
> 2013-11-04 10:39:52,102 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
> Deleted [{ENCODED => fa41edf43fe3ee131db4a34b848ff432, NAME =>
> 'tablethree_mod,,1383559723345.fa41edf43fe3ee131db4a34b848ff432.', STARTKEY
> => '', ENDKEY => ''}, {ENCODED => 76d0e2b7ec3291afcaa82e18a56ccc30, NAME =>
> 'tablethree_mod,,1383561123097.76d0e2b7ec3291afcaa82e18a56ccc30.', STARTKE
> 2013-11-04 10:39:52,111 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
> Added 1
> {code}
> The root cause for this sporadic failure is that, the delete and subsequent
> put will have the same timestamp if they execute in the same ms. The delete
> will override the put in the same ts, even though the put have a larger ts.
> See: HBASE-9905, HBASE-8770
> Credit goes to [~huned] for reporting this bug.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-9923) In HBASE we need to delete a row based on the column name&value (not using rowkey)...
[ https://issues.apache.org/jira/browse/HBASE-9923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13817045#comment-13817045 ] ranjini commented on HBASE-9923: Please review the above scenario. pl give the reply as soon as possible > In HBASE we need to delete a row based on the column name&value (not using > rowkey)... > - > > Key: HBASE-9923 > URL: https://issues.apache.org/jira/browse/HBASE-9923 > Project: HBase > Issue Type: Task >Reporter: ranjini > -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9923) In HBASE we need to delete a row based on the column name&value (not using rowkey)...
[ https://issues.apache.org/jira/browse/HBASE-9923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ranjini updated HBASE-9923: --- Description: Need to Delete a row based on the column name and value, not using rowkey in hbase. Please provide query. Need to Delete a row based on partial rowkey in hbase.Please provide query. Thanks in advance. Issue Type: Bug (was: Task) Please help to do the above task.Thanks > In HBASE we need to delete a row based on the column name&value (not using > rowkey)... > - > > Key: HBASE-9923 > URL: https://issues.apache.org/jira/browse/HBASE-9923 > Project: HBase > Issue Type: Bug >Reporter: ranjini > > Need to Delete a row based on the column name and value, not using rowkey in > hbase. Please provide query. > Need to Delete a row based on partial rowkey in hbase.Please provide query. > Thanks in advance. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9924) avoid filename conflict in region_mover.rb
[ https://issues.apache.org/jira/browse/HBASE-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang Xie updated HBASE-9924: - Attachment: HBase-9924.txt > avoid filename conflict in region_mover.rb > -- > > Key: HBASE-9924 > URL: https://issues.apache.org/jira/browse/HBASE-9924 > Project: HBase > Issue Type: Improvement > Components: shell >Affects Versions: 0.96.0, 0.94.13 >Reporter: Liang Xie >Assignee: Liang Xie > Attachments: HBase-9924.txt > > > when i worked at a shared/common box with my colleague, found this error > while moving region: > NativeException: java.io.FileNotFoundException: /tmp/hh-hadoop-srv-st01.bj > (Permission denied) > writeFile at > /home/xieliang/infra/hbase/target/hbase-0.94.3-mdh1.0.0-SNAPSHOT/hbase-0.94.3-mdh1.0.0-SNAPSHOT/bin/region_mover.rb:283 > unloadRegions at > /home/xieliang/infra/hbase/target/hbase-0.94.3-mdh1.0.0-SNAPSHOT/hbase-0.94.3-mdh1.0.0-SNAPSHOT/bin/region_mover.rb:354 > (root) at > /home/xieliang/infra/hbase/target/hbase-0.94.3-mdh1.0.0-SNAPSHOT/hbase-0.94.3-mdh1.0.0-SNAPSHOT/bin/region_mover.rb:480 > 2013-11-07 15:08:12 Unload host hh-hadoop-srv-st01.bj failed. > The root cause is currently getFilename in region move script will get the > same output with diff users. One possible quick fix is just add the username > to the filename. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9924) avoid filename conflict in region_mover.rb
[ https://issues.apache.org/jira/browse/HBASE-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang Xie updated HBASE-9924: - Status: Patch Available (was: Open) > avoid filename conflict in region_mover.rb > -- > > Key: HBASE-9924 > URL: https://issues.apache.org/jira/browse/HBASE-9924 > Project: HBase > Issue Type: Improvement > Components: shell >Affects Versions: 0.94.13, 0.96.0 >Reporter: Liang Xie >Assignee: Liang Xie > Attachments: HBase-9924.txt > > > when i worked at a shared/common box with my colleague, found this error > while moving region: > NativeException: java.io.FileNotFoundException: /tmp/hh-hadoop-srv-st01.bj > (Permission denied) > writeFile at > /home/xieliang/infra/hbase/target/hbase-0.94.3-mdh1.0.0-SNAPSHOT/hbase-0.94.3-mdh1.0.0-SNAPSHOT/bin/region_mover.rb:283 > unloadRegions at > /home/xieliang/infra/hbase/target/hbase-0.94.3-mdh1.0.0-SNAPSHOT/hbase-0.94.3-mdh1.0.0-SNAPSHOT/bin/region_mover.rb:354 > (root) at > /home/xieliang/infra/hbase/target/hbase-0.94.3-mdh1.0.0-SNAPSHOT/hbase-0.94.3-mdh1.0.0-SNAPSHOT/bin/region_mover.rb:480 > 2013-11-07 15:08:12 Unload host hh-hadoop-srv-st01.bj failed. > The root cause is currently getFilename in region move script will get the > same output with diff users. One possible quick fix is just add the username > to the filename. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (HBASE-9924) avoid filename conflict in region_mover.rb
Liang Xie created HBASE-9924: Summary: avoid filename conflict in region_mover.rb Key: HBASE-9924 URL: https://issues.apache.org/jira/browse/HBASE-9924 Project: HBase Issue Type: Improvement Components: shell Affects Versions: 0.94.13, 0.96.0 Reporter: Liang Xie Assignee: Liang Xie Attachments: HBase-9924.txt when i worked at a shared/common box with my colleague, found this error while moving region: NativeException: java.io.FileNotFoundException: /tmp/hh-hadoop-srv-st01.bj (Permission denied) writeFile at /home/xieliang/infra/hbase/target/hbase-0.94.3-mdh1.0.0-SNAPSHOT/hbase-0.94.3-mdh1.0.0-SNAPSHOT/bin/region_mover.rb:283 unloadRegions at /home/xieliang/infra/hbase/target/hbase-0.94.3-mdh1.0.0-SNAPSHOT/hbase-0.94.3-mdh1.0.0-SNAPSHOT/bin/region_mover.rb:354 (root) at /home/xieliang/infra/hbase/target/hbase-0.94.3-mdh1.0.0-SNAPSHOT/hbase-0.94.3-mdh1.0.0-SNAPSHOT/bin/region_mover.rb:480 2013-11-07 15:08:12 Unload host hh-hadoop-srv-st01.bj failed. The root cause is currently getFilename in region move script will get the same output with diff users. One possible quick fix is just add the username to the filename. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9924) avoid filename conflict in region_mover.rb
[ https://issues.apache.org/jira/browse/HBASE-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13817053#comment-13817053 ] Liang Xie commented on HBASE-9924: -- Before this fix, i need to xcu to root and remove the exsiting tmp file manually to go on. > avoid filename conflict in region_mover.rb > -- > > Key: HBASE-9924 > URL: https://issues.apache.org/jira/browse/HBASE-9924 > Project: HBase > Issue Type: Improvement > Components: shell >Affects Versions: 0.96.0, 0.94.13 >Reporter: Liang Xie >Assignee: Liang Xie > Attachments: HBase-9924.txt > > > when i worked at a shared/common box with my colleague, found this error > while moving region: > NativeException: java.io.FileNotFoundException: /tmp/hh-hadoop-srv-st01.bj > (Permission denied) > writeFile at > /home/xieliang/infra/hbase/target/hbase-0.94.3-mdh1.0.0-SNAPSHOT/hbase-0.94.3-mdh1.0.0-SNAPSHOT/bin/region_mover.rb:283 > unloadRegions at > /home/xieliang/infra/hbase/target/hbase-0.94.3-mdh1.0.0-SNAPSHOT/hbase-0.94.3-mdh1.0.0-SNAPSHOT/bin/region_mover.rb:354 > (root) at > /home/xieliang/infra/hbase/target/hbase-0.94.3-mdh1.0.0-SNAPSHOT/hbase-0.94.3-mdh1.0.0-SNAPSHOT/bin/region_mover.rb:480 > 2013-11-07 15:08:12 Unload host hh-hadoop-srv-st01.bj failed. > The root cause is currently getFilename in region move script will get the > same output with diff users. One possible quick fix is just add the username > to the filename. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9890) MR jobs are not working if started by a delegated user
[
https://issues.apache.org/jira/browse/HBASE-9890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13817059#comment-13817059
]
Hadoop QA commented on HBASE-9890:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12612749/HBASE-9890-94-v4.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7787//console
This message is automatically generated.
> MR jobs are not working if started by a delegated user
> --
>
> Key: HBASE-9890
> URL: https://issues.apache.org/jira/browse/HBASE-9890
> Project: HBase
> Issue Type: Bug
> Components: mapreduce, security
>Affects Versions: 0.98.0, 0.94.12, 0.96.0
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
> Fix For: 0.98.0, 0.96.1, 0.94.14
>
> Attachments: HBASE-9890-94-v0.patch, HBASE-9890-94-v1.patch,
> HBASE-9890-94-v4.patch, HBASE-9890-v0.patch, HBASE-9890-v1.patch,
> HBASE-9890-v2.patch, HBASE-9890-v3.patch, HBASE-9890-v4.patch
>
>
> If Map-Reduce jobs are started with by a proxy user that has already the
> delegation tokens, we get an exception on "obtain token" since the proxy user
> doesn't have the kerberos auth.
> For example:
> * If we use oozie to execute RowCounter - oozie will get the tokens required
> (HBASE_AUTH_TOKEN) and it will start the RowCounter. Once the RowCounter
> tries to obtain the token, it will get an exception.
> * If we use oozie to execute LoadIncrementalHFiles - oozie will get the
> tokens required (HDFS_DELEGATION_TOKEN) and it will start the
> LoadIncrementalHFiles. Once the LoadIncrementalHFiles tries to obtain the
> token, it will get an exception.
> {code}
> org.apache.hadoop.hbase.security.AccessDeniedException: Token generation
> only allowed for Kerberos authenticated clients
> at
> org.apache.hadoop.hbase.security.token.TokenProvider.getAuthenticationToken(TokenProvider.java:87)
> {code}
> {code}
> org.apache.hadoop.ipc.RemoteException(java.io.IOException): Delegation Token
> can be issued only with kerberos or web authentication
> at
> org.apache.hadoop.hdfs.DFSClient.getDelegationToken(DFSClient.java:783)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getDelegationToken(DistributedFileSystem.java:868)
> at
> org.apache.hadoop.fs.FileSystem.collectDelegationTokens(FileSystem.java:509)
> at
> org.apache.hadoop.fs.FileSystem.addDelegationTokens(FileSystem.java:487)
> at
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:130)
> at
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:111)
> at
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:85)
> at
> org.apache.hadoop.filecache.TrackerDistributedCacheManager.getDelegationTokens(TrackerDistributedCacheManager.java:949)
> at
> org.apache.hadoop.mapred.JobClient.copyAndConfigureFiles(JobClient.java:854)
> at
> org.apache.hadoop.mapred.JobClient.copyAndConfigureFiles(JobClient.java:743)
> at
> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:945)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:566)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:596)
> at
> org.apache.hadoop.hbase.mapreduce.RowCounter.main(RowCounter.java:173)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Resolved] (HBASE-9923) In HBASE we need to delete a row based on the column name&value (not using rowkey)...
[ https://issues.apache.org/jira/browse/HBASE-9923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl resolved HBASE-9923. -- Resolution: Invalid Release Note: Hi @ranjini, you'll probably get better answers if you ask this on the mailing lists. > In HBASE we need to delete a row based on the column name&value (not using > rowkey)... > - > > Key: HBASE-9923 > URL: https://issues.apache.org/jira/browse/HBASE-9923 > Project: HBase > Issue Type: Bug >Reporter: ranjini > > Need to Delete a row based on the column name and value, not using rowkey in > hbase. Please provide query. > Need to Delete a row based on partial rowkey in hbase.Please provide query. > Thanks in advance. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Resolved] (HBASE-9922) Need to delete a row based on the column name&value (not the row key)... please provide the delete query for the same...
[ https://issues.apache.org/jira/browse/HBASE-9922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl resolved HBASE-9922. -- Resolution: Invalid Please ask on the mailing list > Need to delete a row based on the column name&value (not the row key)... > please provide the delete query for the same... > > > Key: HBASE-9922 > URL: https://issues.apache.org/jira/browse/HBASE-9922 > Project: HBase > Issue Type: Bug >Reporter: ranjini > -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9923) In HBASE we need to delete a row based on the column name&value (not using rowkey)...
[ https://issues.apache.org/jira/browse/HBASE-9923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-9923: - Release Note: (was: Hi @ranjini, you'll probably get better answers if you ask this on the mailing lists.) Hi @ranjini, you'll probably get better answers if you ask this on the mailing lists. > In HBASE we need to delete a row based on the column name&value (not using > rowkey)... > - > > Key: HBASE-9923 > URL: https://issues.apache.org/jira/browse/HBASE-9923 > Project: HBase > Issue Type: Bug >Reporter: ranjini > > Need to Delete a row based on the column name and value, not using rowkey in > hbase. Please provide query. > Need to Delete a row based on partial rowkey in hbase.Please provide query. > Thanks in advance. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9915) Performance: isSeeked() in EncodedScannerV2 always returns false
[ https://issues.apache.org/jira/browse/HBASE-9915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-9915: - Summary: Performance: isSeeked() in EncodedScannerV2 always returns false (was: Severe performance bug: isSeeked() in EncodedScannerV2 is always false) > Performance: isSeeked() in EncodedScannerV2 always returns false > > > Key: HBASE-9915 > URL: https://issues.apache.org/jira/browse/HBASE-9915 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Fix For: 0.98.0, 0.96.1, 0.94.14 > > Attachments: 9915-0.94.txt, 9915-trunk-v2.txt, 9915-trunk-v2.txt, > 9915-trunk.txt, profile.png > > > While debugging why reseek is so slow I found that it is quite broken for > encoded scanners. > The problem is this: > AbstractScannerV2.reseekTo(...) calls isSeeked() to check whether scanner was > seeked or not. If it was it checks whether the KV we want to seek to is in > the current block, if not it always consults the index blocks again. > isSeeked checks the blockBuffer member, which is not used by EncodedScannerV2 > and thus always returns false, which in turns causes an index lookup for each > reseek. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9915) Performance: isSeeked() in EncodedScannerV2 always returns false
[ https://issues.apache.org/jira/browse/HBASE-9915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-9915: - Component/s: Scanners > Performance: isSeeked() in EncodedScannerV2 always returns false > > > Key: HBASE-9915 > URL: https://issues.apache.org/jira/browse/HBASE-9915 > Project: HBase > Issue Type: Bug > Components: Scanners >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Labels: performance > Fix For: 0.98.0, 0.96.1, 0.94.14 > > Attachments: 9915-0.94.txt, 9915-trunk-v2.txt, 9915-trunk-v2.txt, > 9915-trunk.txt, profile.png > > > While debugging why reseek is so slow I found that it is quite broken for > encoded scanners. > The problem is this: > AbstractScannerV2.reseekTo(...) calls isSeeked() to check whether scanner was > seeked or not. If it was it checks whether the KV we want to seek to is in > the current block, if not it always consults the index blocks again. > isSeeked checks the blockBuffer member, which is not used by EncodedScannerV2 > and thus always returns false, which in turns causes an index lookup for each > reseek. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-9915) Performance: isSeeked() in EncodedScannerV2 always returns false
[ https://issues.apache.org/jira/browse/HBASE-9915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-9915: - Labels: performance (was: ) > Performance: isSeeked() in EncodedScannerV2 always returns false > > > Key: HBASE-9915 > URL: https://issues.apache.org/jira/browse/HBASE-9915 > Project: HBase > Issue Type: Bug > Components: Scanners >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Labels: performance > Fix For: 0.98.0, 0.96.1, 0.94.14 > > Attachments: 9915-0.94.txt, 9915-trunk-v2.txt, 9915-trunk-v2.txt, > 9915-trunk.txt, profile.png > > > While debugging why reseek is so slow I found that it is quite broken for > encoded scanners. > The problem is this: > AbstractScannerV2.reseekTo(...) calls isSeeked() to check whether scanner was > seeked or not. If it was it checks whether the KV we want to seek to is in > the current block, if not it always consults the index blocks again. > isSeeked checks the blockBuffer member, which is not used by EncodedScannerV2 > and thus always returns false, which in turns causes an index lookup for each > reseek. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Resolved] (HBASE-9915) Performance: isSeeked() in EncodedScannerV2 always returns false
[ https://issues.apache.org/jira/browse/HBASE-9915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl resolved HBASE-9915. -- Resolution: Fixed Hadoop Flags: Reviewed Committed to all branches. Thanks for the reviews. > Performance: isSeeked() in EncodedScannerV2 always returns false > > > Key: HBASE-9915 > URL: https://issues.apache.org/jira/browse/HBASE-9915 > Project: HBase > Issue Type: Bug > Components: Scanners >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Labels: performance > Fix For: 0.98.0, 0.96.1, 0.94.14 > > Attachments: 9915-0.94.txt, 9915-trunk-v2.txt, 9915-trunk-v2.txt, > 9915-trunk.txt, profile.png > > > While debugging why reseek is so slow I found that it is quite broken for > encoded scanners. > The problem is this: > AbstractScannerV2.reseekTo(...) calls isSeeked() to check whether scanner was > seeked or not. If it was it checks whether the KV we want to seek to is in > the current block, if not it always consults the index blocks again. > isSeeked checks the blockBuffer member, which is not used by EncodedScannerV2 > and thus always returns false, which in turns causes an index lookup for each > reseek. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9915) Performance: isSeeked() in EncodedScannerV2 always returns false
[ https://issues.apache.org/jira/browse/HBASE-9915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13817090#comment-13817090 ] James Taylor commented on HBASE-9915: - Did I hear someone call that it's time to wrap-up for the 0.94.14 release :-). Awesome work, Lars! > Performance: isSeeked() in EncodedScannerV2 always returns false > > > Key: HBASE-9915 > URL: https://issues.apache.org/jira/browse/HBASE-9915 > Project: HBase > Issue Type: Bug > Components: Scanners >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Labels: performance > Fix For: 0.98.0, 0.96.1, 0.94.14 > > Attachments: 9915-0.94.txt, 9915-trunk-v2.txt, 9915-trunk-v2.txt, > 9915-trunk.txt, profile.png > > > While debugging why reseek is so slow I found that it is quite broken for > encoded scanners. > The problem is this: > AbstractScannerV2.reseekTo(...) calls isSeeked() to check whether scanner was > seeked or not. If it was it checks whether the KV we want to seek to is in > the current block, if not it always consults the index blocks again. > isSeeked checks the blockBuffer member, which is not used by EncodedScannerV2 > and thus always returns false, which in turns causes an index lookup for each > reseek. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HBASE-9915) Performance: isSeeked() in EncodedScannerV2 always returns false
[ https://issues.apache.org/jira/browse/HBASE-9915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13817092#comment-13817092 ] Lars Hofhansl commented on HBASE-9915: -- Since 0.94.13 was somewhat late, I'm planning to do 0.94.14 a bit sooner; before the end of the month. :) > Performance: isSeeked() in EncodedScannerV2 always returns false > > > Key: HBASE-9915 > URL: https://issues.apache.org/jira/browse/HBASE-9915 > Project: HBase > Issue Type: Bug > Components: Scanners >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl > Labels: performance > Fix For: 0.98.0, 0.96.1, 0.94.14 > > Attachments: 9915-0.94.txt, 9915-trunk-v2.txt, 9915-trunk-v2.txt, > 9915-trunk.txt, profile.png > > > While debugging why reseek is so slow I found that it is quite broken for > encoded scanners. > The problem is this: > AbstractScannerV2.reseekTo(...) calls isSeeked() to check whether scanner was > seeked or not. If it was it checks whether the KV we want to seek to is in > the current block, if not it always consults the index blocks again. > isSeeked checks the blockBuffer member, which is not used by EncodedScannerV2 > and thus always returns false, which in turns causes an index lookup for each > reseek. -- This message was sent by Atlassian JIRA (v6.1#6144)
