[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14214031#comment-14214031 ] Lars Hofhansl commented on HBASE-7336: -- See patch in HBASE-12411, relatively easy way

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14214002#comment-14214002 ] Vladimir Rodionov commented on HBASE-7336: -- [~lhofhansl] Yes that would be helpf

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14213870#comment-14213870 ] Lars Hofhansl commented on HBASE-7336: -- [~vrodionov], I have a quite simple patch tha

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14074849#comment-14074849 ] Vladimir Rodionov commented on HBASE-7336: -- [~lhofhansl] The major priority righ

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14074812#comment-14074812 ] Lars Hofhansl commented on HBASE-7336: -- [~vrodionov], curious about how you will find

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14063662#comment-14063662 ] Vladimir Rodionov commented on HBASE-7336: -- [~xieliang007]], I am working on effi

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14063652#comment-14063652 ] Vladimir Rodionov commented on HBASE-7336: -- HBase mini cluster runs HDFS and code

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14063630#comment-14063630 ] Lars Hofhansl commented on HBASE-7336: -- "Local mode" is not the same. It does not act

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14063209#comment-14063209 ] Liang Xie commented on HBASE-7336: -- Yes, i observed similar problem. Long time ago i had

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14062814#comment-14062814 ] Vladimir Rodionov commented on HBASE-7336: -- I monitor thread stack traces during

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14062749#comment-14062749 ] stack commented on HBASE-7336: -- [~vrodionov] Can you try on hdfs? > HFileBlock.readAtOffset

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14062740#comment-14062740 ] Vladimir Rodionov commented on HBASE-7336: -- Forgot to mention: HBase 0.98.3 hado

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14062733#comment-14062733 ] Vladimir Rodionov commented on HBASE-7336: -- DFSInputStream class is heavily synch

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14062356#comment-14062356 ] Vladimir Rodionov commented on HBASE-7336: -- h3. Effect of compaction on large sca

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14050791#comment-14050791 ] Lars Hofhansl commented on HBASE-7336: -- I agree with your assessment on the seek+read

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14050778#comment-14050778 ] Lars Hofhansl commented on HBASE-7336: -- DFSInputStream *does* override this method (c

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14050777#comment-14050777 ] Vladimir Rodionov commented on HBASE-7336: -- I was not right, Lars. *DFSInputStrea

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14050768#comment-14050768 ] Lars Hofhansl commented on HBASE-7336: -- The trylock stuff is from the patch here: # t

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14050541#comment-14050541 ] Vladimir Rodionov commented on HBASE-7336: -- Upd. The code is not dangerous - it j

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14050538#comment-14050538 ] Vladimir Rodionov commented on HBASE-7336: -- I am looking into this stuff now, try

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13544426#comment-13544426 ] Hudson commented on HBASE-7336: --- Integrated in HBase-0.94-security-on-Hadoop-23 #10 (See [h

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13537918#comment-13537918 ] Hudson commented on HBASE-7336: --- Integrated in HBase-0.94-security #87 (See [https://builds

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13534423#comment-13534423 ] Hudson commented on HBASE-7336: --- Integrated in HBase-0.94 #635 (See [https://builds.apache.

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13534149#comment-13534149 ] Lars Hofhansl commented on HBASE-7336: -- The 0.94 tests fail with the same OOM even wi

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533726#comment-13533726 ] Hudson commented on HBASE-7336: --- Integrated in HBase-0.94 #632 (See [https://builds.apache.

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533655#comment-13533655 ] Lars Hofhansl commented on HBASE-7336: -- Actually that test is not performance represe

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533608#comment-13533608 ] Lars Hofhansl commented on HBASE-7336: -- Somewhat tempted to revert this patch.

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533606#comment-13533606 ] Lars Hofhansl commented on HBASE-7336: -- The latest test run passed. Looking at testC

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533583#comment-13533583 ] Lars Hofhansl commented on HBASE-7336: -- Running the test locally also seems to consum

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533567#comment-13533567 ] Lars Hofhansl commented on HBASE-7336: -- TestHFileBlock.testConcurrentReading[1] is fa

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13532736#comment-13532736 ] Lars Hofhansl commented on HBASE-7336: -- Note that the previous tests all were scans t

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13531664#comment-13531664 ] Hudson commented on HBASE-7336: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #295 (See [https

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13531506#comment-13531506 ] Hudson commented on HBASE-7336: --- Integrated in HBase-TRUNK #3618 (See [https://builds.apach

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13531500#comment-13531500 ] Hudson commented on HBASE-7336: --- Integrated in HBase-0.94 #625 (See [https://builds.apache.

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13531265#comment-13531265 ] stack commented on HBASE-7336: -- +1 on committing this for now. On Reader per long-running sc

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530612#comment-13530612 ] liang xie commented on HBASE-7336: -- +1, LGTM,a nice test case, Lars > HF

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530580#comment-13530580 ] Lars Hofhansl commented on HBASE-7336: -- Numbers with patch and read short circuit ena

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530512#comment-13530512 ] Lars Hofhansl commented on HBASE-7336: -- Any objections to committing this (0.94 and 0

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13530074#comment-13530074 ] Lars Hofhansl commented on HBASE-7336: -- TestMultiParallel passed locally.

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529785#comment-13529785 ] Hadoop QA commented on HBASE-7336: -- {color:red}-1 overall{color}. Here are the results o

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529730#comment-13529730 ] Lars Hofhansl commented on HBASE-7336: -- bq. Compactions should go get their own Reade

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529712#comment-13529712 ] stack commented on HBASE-7336: -- bq. tryLock without a timeout returns immediately. Sorry. T

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529682#comment-13529682 ] Lars Hofhansl commented on HBASE-7336: -- tryLock without a timeout returns immediately

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529669#comment-13529669 ] stack commented on HBASE-7336: -- Should the tryLock have a timeout on it? Else patch is an im

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529647#comment-13529647 ] Lars Hofhansl commented on HBASE-7336: -- I have a patch, that changes that synchronize

[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads

[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529642#comment-13529642 ] Lars Hofhansl commented on HBASE-7336: -- Here's my test case: 20m rows, single column