[jira] [Reopened] (HBASE-4433) avoid extra next (potentially a seek) if done with column/row
[ https://issues.apache.org/jira/browse/HBASE-4433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reopened HBASE-4433: -- Reopening until we apply to 0.92 and 0.90 too (as per Andrew). I'll do it (stop me Jon if you are already at this). > avoid extra next (potentially a seek) if done with column/row > - > > Key: HBASE-4433 > URL: https://issues.apache.org/jira/browse/HBASE-4433 > Project: HBase > Issue Type: Improvement >Reporter: Kannan Muthukkaruppan >Assignee: Kannan Muthukkaruppan > Fix For: 0.94.0 > > > [Noticed this in 89, but quite likely true of trunk as well.] > When we are done with the requested column(s) the code still does an extra > next() call before it realizes that it is actually done. This extra next() > call could potentially result in an unnecessary extra block load. This is > likely to be especially bad for CFs where the KVs are large blobs where each > KV may be occupying a block of its own. So the next() can often load a new > unrelated block unnecessarily. > -- > For the simple case of reading say the top-most column in a row in a single > file, where each column (KV) was say a block of its own-- it seems that we > are reading 3 blocks, instead of 1 block! > I am working on a simple patch and with that the number of seeks is down to > 2. > [There is still an extra seek left. I think there were two levels of > extra/unnecessary next() we were doing without actually confirming that the > next was needed. One at the StoreScanner/ScanQueryMatcher level which this > diff avoids. I think the other is at hfs.next() (at the storefile scanner > level) that's happening whenever a HFile scanner servers out a data-- and > perhaps that's the additional seek that we need to avoid. But I want to > tackle this optimization first as the two issues seem unrelated.] > -- > The basic idea of the patch I am working on/testing is as follows. The > ExplicitColumnTracker currently returns "INCLUDE" to the ScanQueryMatcher if > the KV needs to be included and then if done, only in the the next call it > returns the appropriate SEEK_NEXT_COL or SEEK_NEXT_ROW hint. For the cases > when ExplicitColumnTracker knows it is done with a particular column/row, the > patch attempts to combine the INCLUDE code and done hint into a single match > code-- INCLUDE_AND_SEEK_NEXT_COL and INCLUDE_AND_SEEK_NEXT_ROW. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-4430) Disable TestSlabCache and TestSingleSizedCache temporarily to see if these are cause of build box failure though all tests pass
[ https://issues.apache.org/jira/browse/HBASE-4430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reopened HBASE-4430: -- Reopening because TestSlabCache failed in most recent build -- hung. Going to disable this test again. > Disable TestSlabCache and TestSingleSizedCache temporarily to see if these > are cause of build box failure though all tests pass > --- > > Key: HBASE-4430 > URL: https://issues.apache.org/jira/browse/HBASE-4430 > Project: HBase > Issue Type: Task > Components: test >Reporter: stack >Assignee: Li Pi >Priority: Blocker > Fix For: 0.92.0 > > Attachments: TestSlabCache.trace > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-4680) FSUtils.isInSafeMode() checks should operate on HBase root dir, where we have permissions
[
https://issues.apache.org/jira/browse/HBASE-4680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack reopened HBASE-4680:
--
Reopening
> FSUtils.isInSafeMode() checks should operate on HBase root dir, where we have
> permissions
> -
>
> Key: HBASE-4680
> URL: https://issues.apache.org/jira/browse/HBASE-4680
> Project: HBase
> Issue Type: Bug
> Components: util
>Affects Versions: 0.92.0, 0.94.0
>Reporter: Gary Helmling
>Assignee: Gary Helmling
>Priority: Critical
> Fix For: 0.92.0
>
> Attachments: HBASE-4680.patch
>
>
> The HDFS safe mode check workaround introduced by HBASE-4510 performs a
> {{FileSystem.setPermission()}} operation on the root directory ("/") when
> attempting to trigger a {{SafeModeException}}. As a result, it requires
> superuser privileges when running with DFS permission checking enabled.
> Changing the operations to act on the HBase root directory should be safe,
> since the master process must have write access to it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-4298) Support to drain RS nodes through ZK
[ https://issues.apache.org/jira/browse/HBASE-4298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reopened HBASE-4298: -- Reopening. It was applied to 0.92 but we might want to apply to 0.90... moving it over there. > Support to drain RS nodes through ZK > > > Key: HBASE-4298 > URL: https://issues.apache.org/jira/browse/HBASE-4298 > Project: HBase > Issue Type: Improvement > Components: master >Affects Versions: 0.90.4 > Environment: all >Reporter: Aravind Gottipati >Priority: Critical > Labels: patch > Fix For: 0.92.0, 0.90.5 > > Attachments: 4298-trunk-v2.txt, 4298-trunk-v3.txt, 90_hbase.patch, > drainingservertest-v2.txt, drainingservertest.txt, trunk_hbase.patch, > trunk_with_test.txt > > > HDFS currently has a way to exclude certain datanodes and prevent them from > getting new blocks. HDFS goes one step further and even drains these nodes > for you. This enhancement is a step in that direction. > The idea is that we mark nodes in zookeeper as draining nodes. This means > that they don't get any more new regions. These draining nodes look exactly > the same as the corresponding nodes in /rs, except they live under /draining. > Eventually, support for draining them can be added. I am submitting two > patches for review - one for the 0.90 branch and one for trunk (in git). > Here are the two patches > 0.90 - > https://github.com/aravind/hbase/commit/181041e72e7ffe6a4da6d82b431ef7f8c99e62d2 > trunk - > https://github.com/aravind/hbase/commit/e127b25ae3b4034103b185d8380f3b7267bc67d5 > I have tested both these patches and they work as advertised. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-4288) "Server not running" exception during meta verification causes RS abort
[ https://issues.apache.org/jira/browse/HBASE-4288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reopened HBASE-4288: -- Hmm... yeah, still issue in 0.90 branch so leaving open. > "Server not running" exception during meta verification causes RS abort > --- > > Key: HBASE-4288 > URL: https://issues.apache.org/jira/browse/HBASE-4288 > Project: HBase > Issue Type: Bug > Components: master >Affects Versions: 0.90.4 >Reporter: Todd Lipcon >Priority: Critical > Fix For: 0.90.5 > > Attachments: 4288-v2.txt, 4288.txt > > > The master tried to verify the META location just as that server was shutting > down due to an abort. This caused the "Server not running" exception to get > thrown, which wasn't handled properly in the master, causing it to abort. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-4772) Utility to Create StoreFiles
[ https://issues.apache.org/jira/browse/HBASE-4772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reopened HBASE-4772: -- I don't see this patch in TRUNK or in 0.92 yet the issue is resolved? > Utility to Create StoreFiles > > > Key: HBASE-4772 > URL: https://issues.apache.org/jira/browse/HBASE-4772 > Project: HBase > Issue Type: Test >Affects Versions: 0.94.0 >Reporter: Nicolas Spiegelberg >Assignee: Mikhail Bautin >Priority: Minor > Fix For: 0.94.0 > > Attachments: HBASE-4772-B.patch, HBASE-4772.patch > > > Add a tool to create a StoreFile with the specified number of key/value > pairs, with the specified compression and Bloom filter type. This is useful > for creating HFileV1 & HFileV2 store files for testing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-4352) Apply version of hbase-4015 to branch
[ https://issues.apache.org/jira/browse/HBASE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reopened HBASE-4352: -- Reopened after reverting patch from 0.90 branch. > Apply version of hbase-4015 to branch > - > > Key: HBASE-4352 > URL: https://issues.apache.org/jira/browse/HBASE-4352 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: ramkrishna.s.vasudevan >Priority: Blocker > Attachments: HBASE-4352_0.90.patch, HBASE-4352_0.90_1.patch > > > Consider adding a version of hbase-4015 to 0.90. It changes HRegionInterface > so would need move change to end of the Interface and then test that it > doesn't break rolling restart. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-4922) [packaging] Assembly tars up hbase in a subdir; i.e. after untar hbase-0.92.0 has a subdir named 0.92.0
[ https://issues.apache.org/jira/browse/HBASE-4922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reopened HBASE-4922: -- This patch seems to do wrong thing. When I untar, I have: hbase-0.92.0/hbase-0.92.0/* This is one hbase-0.92.0 too many. Instead I would think we want after untarring: hbase-0.92.0/* > [packaging] Assembly tars up hbase in a subdir; i.e. after untar hbase-0.92.0 > has a subdir named 0.92.0 > --- > > Key: HBASE-4922 > URL: https://issues.apache.org/jira/browse/HBASE-4922 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: Roman Shaposhnik >Priority: Blocker > Fix For: 0.92.0 > > Attachments: HBASE-4922.patch.txt > > > Reported by Roman. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-4935) hbase 0.92.0 doesn't work going against 0.20.205.0, its packaged hadoop
[ https://issues.apache.org/jira/browse/HBASE-4935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reopened HBASE-4935: -- Assignee: (was: Mikhail Bautin) Reverting because breaks building against hadoop 0.22. > hbase 0.92.0 doesn't work going against 0.20.205.0, its packaged hadoop > --- > > Key: HBASE-4935 > URL: https://issues.apache.org/jira/browse/HBASE-4935 > Project: HBase > Issue Type: Bug >Reporter: stack > Fix For: 0.92.0 > > Attachments: 4935-reverse.txt, 4935.txt > > > See this Mikhail thread up on the list: > http://search-hadoop.com/m/WMUZR24EAJ1/%2522SequenceFileLogReader+uses+a+reflection+hack+resulting+in+runtime+failures%2522&subj=Re+SequenceFileLogReader+uses+a+reflection+hack+resulting+in+runtime+failures > Dig into it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-5081) Distributed log splitting deleteNode races againsth splitLog retry
[ https://issues.apache.org/jira/browse/HBASE-5081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reopened HBASE-5081: -- Reopening. I committed though there was review still going on. Backed out change from trunk and 0.92. > Distributed log splitting deleteNode races againsth splitLog retry > --- > > Key: HBASE-5081 > URL: https://issues.apache.org/jira/browse/HBASE-5081 > Project: HBase > Issue Type: Bug > Components: wal >Affects Versions: 0.92.0, 0.94.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang > Fix For: 0.92.0 > > Attachments: distributed-log-splitting-screenshot.png, > hbase-5081-patch-v6.txt, hbase-5081-patch-v7.txt, > hbase-5081_patch_for_92_v4.txt, hbase-5081_patch_v5.txt, patch_for_92.txt, > patch_for_92_v2.txt, patch_for_92_v3.txt > > > Recently, during 0.92 rc testing, we found distributed log splitting hangs > there forever. Please see attached screen shot. > I looked into it and here is what happened I think: > 1. One rs died, the servershutdownhandler found it out and started the > distributed log splitting; > 2. All three tasks failed, so the three tasks were deleted, asynchronously; > 3. Servershutdownhandler retried the log splitting; > 4. During the retrial, it created these three tasks again, and put them in a > hashmap (tasks); > 5. The asynchronously deletion in step 2 finally happened for one task, in > the callback, it removed one > task in the hashmap; > 6. One of the newly submitted tasks' zookeeper watcher found out that task is > unassigned, and it is not > in the hashmap, so it created a new orphan task. > 7. All three tasks failed, but that task created in step 6 is an orphan so > the batch.err counter was one short, > so the log splitting hangs there and keeps waiting for the last task to > finish which is never going to happen. > So I think the problem is step 2. The fix is to make deletion sync, instead > of async, so that the retry will have > a clean start. > Async deleteNode will mess up with split log retrial. In extreme situation, > if async deleteNode doesn't happen > soon enough, some node created during the retrial could be deleted. > deleteNode should be sync. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-5221) bin/hbase script doesn't look for Hadoop jars in the right place in trunk layout
[ https://issues.apache.org/jira/browse/HBASE-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reopened HBASE-5221: -- I reverted the change Jimmy. Should we close as won't resolve or as dup of the issue that Roman refers to? > bin/hbase script doesn't look for Hadoop jars in the right place in trunk > layout > > > Key: HBASE-5221 > URL: https://issues.apache.org/jira/browse/HBASE-5221 > Project: HBase > Issue Type: Bug >Affects Versions: 0.92.0 >Reporter: Todd Lipcon >Assignee: Jimmy Xiang > Fix For: 0.94.0 > > Attachments: hbase-5221.txt > > > Running against an 0.24.0-SNAPSHOT hadoop: > ls: cannot access > /home/todd/ha-demo/hadoop-0.24.0-SNAPSHOT/hadoop-common*.jar: No such file or > directory > ls: cannot access /home/todd/ha-demo/hadoop-0.24.0-SNAPSHOT/hadoop-hdfs*.jar: > No such file or directory > ls: cannot access > /home/todd/ha-demo/hadoop-0.24.0-SNAPSHOT/hadoop-mapred*.jar: No such file or > directory > The jars are rooted deeper in the heirarchy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
