[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16291911#comment-16291911
]
Hudson commented on HBASE-19289:
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4226 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/4226/])
HBASE-19289 Add flag to disable stream capability enforcement (mdrob: rev
2c9ef8a471148ece655b881cc490b6b685d634f4)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogWriter.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/io/asyncfs/AsyncFSOutputHelper.java
* (edit)
hbase-common/src/main/java/org/apache/hadoop/hbase/util/CommonFSUtils.java
* (edit)
hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/WALProcedureStore.java
* (edit) hbase-procedure/src/test/resources/hbase-site.xml
* (edit) hbase-server/src/test/resources/hbase-site.xml
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mike Drob
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 19289.v1.txt, 19289.v2.txt, HBASE-19289.patch,
> HBASE-19289.v2.patch, HBASE-19289.v3.patch, HBASE-19289.v4.patch,
> HBASE-19289.v5.patch
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16291296#comment-16291296
]
Mike Drob commented on HBASE-19289:
---
Yea, been testing this locally. pushed to branch-2 and master, thanks for the
reviews!
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mike Drob
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 19289.v1.txt, 19289.v2.txt, HBASE-19289.patch,
> HBASE-19289.v2.patch, HBASE-19289.v3.patch, HBASE-19289.v4.patch,
> HBASE-19289.v5.patch
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288447#comment-16288447
]
Josh Elser commented on HBASE-19289:
LGTM. I assume you're running these tests successfully using H3 on your local
machine (as I don't think PreCommit is, right?) -- if that's the case, push it!
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mike Drob
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 19289.v1.txt, 19289.v2.txt, HBASE-19289.patch,
> HBASE-19289.v2.patch, HBASE-19289.v3.patch, HBASE-19289.v4.patch,
> HBASE-19289.v5.patch
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288443#comment-16288443
]
Josh Elser commented on HBASE-19289:
bq. Do you mind if we handle the bit about standalone cluster in a follow on
issue? Currently we're blocking tests, and while that should definitely be a
release blocker I'm not sure it needs to be addressed with equal urgency.
Not at all. Just wanted to make sure we didn't miss this aspect :)
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mike Drob
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 19289.v1.txt, 19289.v2.txt, HBASE-19289.patch,
> HBASE-19289.v2.patch, HBASE-19289.v3.patch, HBASE-19289.v4.patch,
> HBASE-19289.v5.patch
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288420#comment-16288420
]
Mike Drob commented on HBASE-19289:
---
bq. How about instead: "Controls whether or not HBase will require stream
capabilities (hflush) in tests using the LocalFileSystem class".
I'll wordsmith this a bit.
bq. Nit: TestHStore.java has a whitespace-only change.
Fixed.
Do you mind if we handle the bit about standalone cluster in a follow on issue?
Currently we're blocking tests, and while that should definitely be a release
blocker I'm not sure it needs to be addressed with equal urgency.
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mike Drob
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 19289.v1.txt, 19289.v2.txt, HBASE-19289.patch,
> HBASE-19289.v2.patch, HBASE-19289.v3.patch, HBASE-19289.v4.patch
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16287868#comment-16287868
]
Josh Elser commented on HBASE-19289:
bq. put the property in test/resources/hbase-site.xml
That's a nice trick!
{code}
We don't need to enforce stream capabilities in tests using local
FS
{code}
How about instead: "Controls whether or not HBase will require stream
capabilities (hflush) in tests using the LocalFileSystem class".
Nit: TestHStore.java has a whitespace-only change.
bq. AFAICT anybody starting HBase on local file system is going to run straight
into a brick wall.
Need to return to this too, eh? The
[book|https://hbase.apache.org/book.html#_get_started_with_hbase] does have
instructions for users to create an hbase-site.xml -- is updating this
sufficient? A release note is obviously good to warn users. Is the error
message a user would find if they _didn't_ set this property sufficient to lead
them to somewhere that will clearly show them how to fix the problem (e.g. via
a web-search)?
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mike Drob
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 19289.v1.txt, 19289.v2.txt, HBASE-19289.patch,
> HBASE-19289.v2.patch, HBASE-19289.v3.patch, HBASE-19289.v4.patch
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16284175#comment-16284175
]
Ted Yu commented on HBASE-19289:
How about putting the config in:
./hbase-server/src/test/resources/hbase-site.xml
This way, you don't need to modify individual tests.
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mike Drob
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 19289.v1.txt, 19289.v2.txt, HBASE-19289.patch,
> HBASE-19289.v2.patch, HBASE-19289.v3.patch
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16284123#comment-16284123
]
Mike Drob commented on HBASE-19289:
---
Setting a property on the conf doesn't work for the tests in hbase-server
because the FS is already created and then gets pulled out of the cache in
{{FSHLogProvider.createWAL()}}, need to do the DFSCluster approach here...
v4 coming soon.
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mike Drob
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 19289.v1.txt, 19289.v2.txt, HBASE-19289.patch,
> HBASE-19289.v2.patch, HBASE-19289.v3.patch
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16284044#comment-16284044
]
Ted Yu commented on HBASE-19289:
lgtm
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mike Drob
>Priority: Critical
> Fix For: 2.0.0-beta-1
>
> Attachments: 19289.v1.txt, 19289.v2.txt, HBASE-19289.patch,
> HBASE-19289.v2.patch, HBASE-19289.v3.patch
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16283768#comment-16283768
]
Ted Yu commented on HBASE-19289:
{code}
if (!(CommonFSUtils.hasCapability(newStream, durability))) {
- throw new IllegalStateException("The procedure WAL relies on the ability
to " + durability +
+ if (fs.getConf().getBoolean("hbase.stream.capability.enforce", true))
{code}
Since the value of the new config is unlikely to change during the lifetime of
server, you can save the value in an instance variable.
The order of {{if (!(CommonFSUtils.hasCapability(}} and the config check should
be reverted.
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mike Drob
> Attachments: 19289.v1.txt, 19289.v2.txt, HBASE-19289.patch,
> HBASE-19289.v2.patch
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16283703#comment-16283703
]
Mike Drob commented on HBASE-19289:
---
{{hbase.unsafe.stream.capability.enforce.disabled}}?
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mike Drob
> Attachments: 19289.v1.txt, 19289.v2.txt, HBASE-19289.patch,
> HBASE-19289.v2.patch
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16283472#comment-16283472
]
Steve Loughran commented on HBASE-19289:
What about giving the property some name to make clear its experimental/risky?
""hbase.experimental.stream.capability.enforce.disabled"
Then if people set it, well, "told you so"
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mike Drob
> Attachments: 19289.v1.txt, 19289.v2.txt, HBASE-19289.patch,
> HBASE-19289.v2.patch
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16282217#comment-16282217
]
Josh Elser commented on HBASE-19289:
bq. That doesn't seem right to me. We have a bunch of tests that check for data
correctness, that we're apparently running with a file system that does not
offer the guarantees that we expect.
Sorry, the corollary got lost. For tests in which we're explicitly testing for
correctness in the face of failure (e.g. CM doing CM things), we should
definitely use something with can have the guarantees we require.
My assertion was that not all tests are testing for correctness in the face of
failure -- in these cases, MiniDFS would be overkill. This is still based on my
previous understanding that XSumFileSystem does not have inherent issues under
normal circumstances (e.g. those in which we're not abruptly
killing/interrupting things). Maybe this has changed since Hadoop2 and I'm
coming from bad knowledge?
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mike Drob
> Attachments: 19289.v1.txt, 19289.v2.txt, HBASE-19289.patch
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16282197#comment-16282197
]
Mike Drob commented on HBASE-19289:
---
[~elserj] - let me see if I understand your proposed solution.
# Introduce new flag to not fail on the StreamCapability flush/sync checks (but
still print appropriate warnings)
# Apply this flag to unit tests that don't use an underlying DFS cluster
# Apply this flag to standalone clusters
# Update documentation to include blatant caveats about data loss in standalone
cluster
That doesn't seem right to me. We have a bunch of tests that check for data
correctness, that we're apparently running with a file system that does not
offer the guarantees that we expect. This is like the difference between
SloppyMath and StrictMath - one is fast; the other is accurate. Do we gain any
confidence from running tests quickly, seeing them pass, and then discovering
that we don't have the coverage we thought we did at a later date?
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mike Drob
> Attachments: 19289.v1.txt, 19289.v2.txt, HBASE-19289.patch
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280708#comment-16280708
]
Steve Loughran commented on HBASE-19289:
looking at the patch
* it can take time to start and stop the serve, good to see you making this a
class rule.
* failures in stop should be caught & logged, in case raising them could iding
the underlying exception triggering a test failure. (I don't know enough about
custom rules here to know for sure, just based on test teardown method
experience)
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mike Drob
> Attachments: 19289.v1.txt, 19289.v2.txt, HBASE-19289.patch
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280415#comment-16280415
]
Josh Elser commented on HBASE-19289:
bq. That doesn't work for our reliance on LocalFileSystem for standalone mode
(ref the quickstart guide). We don't actually call sync / flush or anything
like that for the local OS, so the WAL is essentially useless.
Understood. I still think it's something that's easily caveat'ed: "The use of
HDFS (or a comparable FileSystem implementation) is required to ensure that
HBase does not experience data loss. This quickstart example may lose data in
unexpected-failure conditions -- this should only be used for
testing/investigation purposes, never in a production environment."
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mike Drob
> Attachments: 19289.v1.txt, 19289.v2.txt, HBASE-19289.patch
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279621#comment-16279621
]
Sean Busbey commented on HBASE-19289:
-
{quote}
bq. unless we want to expressly tell folks not to run HBase on top of
LocalFileSystem, in which case why are we running tests against it in the first
place?
bq. ps: Are people using HBase against file:// today? If so, they've not been
getting the persistence/durability HBase needs. Tell them to stop it.
Is this solvable by a flag that says "yes I acknowledge that I may lose data"?
I think we're well aware that we may experience "data loss" with XSumFileSystem
and this is OK because we just don't care (because it's a short lived test). We
don't want to wait the extra 5+secs for a full MiniDFSCluster.
{quote}
That doesn't work for our reliance on LocalFileSystem for standalone mode ([ref
the quickstart
guide|http://hbase.apache.org/book.html#_get_started_with_hbase]). We don't
actually call sync / flush or anything like that for the local OS, so the WAL
is essentially useless.
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mike Drob
> Attachments: 19289.v1.txt, 19289.v2.txt, HBASE-19289.patch
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279296#comment-16279296
]
Ted Yu commented on HBASE-19289:
I tried the patch on both Linux and Mac.
I got the same error:
{code}
[ERROR] org.apache.hadoop.hbase.regionserver.TestHStore Time elapsed: 2.663 s
<<< ERROR!
java.lang.NoSuchMethodError:
org.eclipse.jetty.server.session.SessionHandler.setHttpOnly(Z)V
{code}
{code}
mvn clean install -DskipTests -Dhadoop.profile=3.0
-Dhadoop-three.version=3.0.0-beta1 -Dhadoop-two.version=3.0.0-beta1
mvn test -Dtest=TestHStore\#testRefreshStoreFiles -Dhadoop.profile=3.0
-Dhadoop-three.version=3.0.0-beta1 -Dhadoop-two.version=3.0.0-beta1 -pl
hbase-server -am
{code}
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mike Drob
> Attachments: 19289.v1.txt, 19289.v2.txt, HBASE-19289.patch
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279270#comment-16279270
]
Josh Elser commented on HBASE-19289:
bq. unless we want to expressly tell folks not to run HBase on top of
LocalFileSystem, in which case why are we running tests against it in the first
place?
bq. ps: Are people using HBase against file:// today? If so, they've not been
getting the persistence/durability HBase needs. Tell them to stop it.
Is this solvable by a flag that says "yes I acknowledge that I may lose data"?
I think we're well aware that we may experience "data loss" with XSumFileSystem
and this is OK because we just don't care (because it's a short lived test). We
don't want to wait the extra 5+secs for a full MiniDFSCluster.
bq. Anyway, I'm attaching a patch that will start a DFS cluster for tests where
we need to use this. So far I've only done it for one test, but I know there
are lots of failures out there
Your approach is a nice idea as well, but I'm worried about the tail in
tracking all of these down.
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mike Drob
> Attachments: 19289.v1.txt, 19289.v2.txt, HBASE-19289.patch
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279199#comment-16279199
]
Mike Drob commented on HBASE-19289:
---
My proposed solution works for hbase-server, but we also have failing tests in
the same vein in hbase-procedure. I would prefer not to introduce a dependency
on hdfs that low, but maybe there's no other way around it?
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mike Drob
> Attachments: 19289.v1.txt, 19289.v2.txt, HBASE-19289.patch
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279156#comment-16279156
]
Mike Drob commented on HBASE-19289:
---
HBASE-19369 is related to it, but I don't think it will fix the cases where we
run on LocalFS
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Mike Drob
> Attachments: 19289.v1.txt, 19289.v2.txt, HBASE-19289.patch
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279142#comment-16279142
]
Mike Drob commented on HBASE-19289:
---
Cool, I get this to fail locally now. I tried setting file system to
{{raw:///}} then I realized that you were suggesting that as a future fix, not
something that's available now, [~steve_l].
At least for our tests here, since we're testing higher level functionality in
HBase we shouldn't need to care about the underlying FS implementation - there
is an expectation that the local fs (ext3 or whatever) supports flush and sync,
and beyond that there's no reason to start a full DFS cluster - this will add a
lot of time to the test suite if it's not strictly necessary.
Anyway, I'm attaching a patch that will start a DFS cluster for tests where we
need to use this. So far I've only done it for one test, but I know there are
lots of failures out there... Attaching the WIP for initial review of the
approach. [~elserj], [~tedyu] - WDYT?
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: 19289.v1.txt, 19289.v2.txt, HBASE-19289.patch
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16277591#comment-16277591
]
Ted Yu commented on HBASE-19289:
Mike:
You may need the patch from HBASE-19422.
Here is the command I used:
{code}
mvn test -Dtest=TestHStore\#testRefreshStoreFiles -Dhadoop-profile=3.0
-Dhadoop-three.version=3.0.0-beta1 -Dhadoop-two.version=3.0.0-beta1 -pl
hbase-serr -am
{code}
{code}
[ERROR] testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore)
Time elapsed: 0.718 s <<< ERROR!
java.io.IOException: cannot get log writer
at
org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
at
org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
at
org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
at
org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
at
org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
at
org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
at
org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
at
org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
Caused by:
org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
hflush
at
org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
at
org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
at
org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
at
org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
at
org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
at
org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
at
org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
at
org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
{code}
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: 19289.v1.txt, 19289.v2.txt
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16277358#comment-16277358
]
Sean Busbey commented on HBASE-19289:
-
bq. ps: Are people using HBase against file:// today? If so, they've not been
getting the persistence/durability HBase needs. Tell them to stop it.
I believe we do this in standalone mode.
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: 19289.v1.txt, 19289.v2.txt
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16277352#comment-16277352
]
Mike Drob commented on HBASE-19289:
---
I can't get this to fail locally... Did somebody fix it in a separate issue?
My command is
{noformat}
mvn clean test -Dtest=TestHStore\#testRefreshStoreFiles -Dhadoop-profile=3.0
-Dhadoop-three.version=3.0.0-beta1 -pl hbase-server -am
{noformat}
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: 19289.v1.txt, 19289.v2.txt
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261454#comment-16261454
]
Steve Loughran commented on HBASE-19289:
ps: Are people using HBase against file:// today? If so, they've not been
getting the persistence/durability HBase needs. Tell them to stop it.
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: 19289.v1.txt, 19289.v2.txt
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261449#comment-16261449
]
Steve Loughran commented on HBASE-19289:
If people really want hbase -> file://
* they'd need a distributed file:// or some shared NFS server
* it'd presumably need its own RAID > 0 to do checksumming; so checksum fs is
moot
I'd look at seeing whether checksumfs could actually bypass its checksum, say
if we set the property "bytes per checksum == 0" as the secret no, turn me off"
switch. But people would probably then use it for performance and then be upset
when all their data got corrupted without anything noticing. It's too critical
a layer under HDFS really.
I was thinking about what if we added a raw:// URL which bonded directly to raw
local fs, but RawLocalFileSystem has an expectation that file:// is its schema
and returns it in getURI(), so forcing you back to CheckummedFS
I believe the way to do this is
* subclass RawLocalFileSystem
* give it a new schema, like say "raw"
* have it remember its URI in initialize() and return it in getURI()
* register it (statically, dynamically)
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: 19289.v1.txt, 19289.v2.txt
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16259947#comment-16259947
]
Sean Busbey commented on HBASE-19289:
-
[~yuzhih...@gmail.com] v1 is expressly unsafe based on the discussion so far,
please don't commit it.
How onerous is converting the failed tests to use MiniDFSCluster instead of
LocalFileSystem?
Does this mean we're also broken in standalone mode?
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: 19289.v1.txt
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16258707#comment-16258707
]
Sean Busbey commented on HBASE-19289:
-
Thanks for weighing in [~ste...@apache.org]. This helps a ton; I had overlooked
that limitation of ChecksumFileSystem.
Would updating RawLocalFileSystem to create output streams that do implement
hflush/hsync be viable? I think long term HBase needs a FileSystem
implementation that can be used with the local disks on a single node. I'd
rather it get maintained in a place where folks other than us can use it and
help with maintenance.
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: 19289.v1.txt
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16258446#comment-16258446
]
Steve Loughran commented on HBASE-19289:
Closed HADOOP-15051 as wontfix. LocalFS output streams don't declare their
support for hflush/sync for the following reason, as covered in HADOOP-13327
(oustanding, reviews welcome)
h3. Output streams which do not implement the flush/persitence semantics of
hflush/hsync MUST NOT declare that their streams have that capability.
LocalFileSystem is a subclass of ChecksumFileSystem; ChecksumFileSystem output
streams don't implement hflush/hsync, therefore it's the correct behaviour in
the Hadoop code.
If HBase requires the methods for the correct persistence of its data, then it
cannot safely use localFS as destination of its output. It's check is therefore
also the correct behavior
In which case, "expressly tell folks not to run HBase on top of
LocalFileSystem," is the correct action on your part. People must not be using
the local FS as a direct destination of HDFS output.
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: 19289.v1.txt
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16258012#comment-16258012
]
Ted Yu commented on HBASE-19289:
Patch v1 uses approach #2.
This would allow me to run test suite against hadoop3.
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
> Attachments: 19289.v1.txt
>
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257578#comment-16257578
]
Ted Yu commented on HBASE-19289:
Logged HADOOP-15051
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257437#comment-16257437
]
Sean Busbey commented on HBASE-19289:
-
The more I think about this, the more I think changing LocalFileSystem needs to
be the answer. unless we want to expressly tell folks not to run HBase on top
of LocalFileSystem, in which case why are we running tests against it in the
first place?
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257404#comment-16257404
]
Ted Yu commented on HBASE-19289:
bq. updating LocalFileSystem to support hflush/hsync
This would be done in hadoop, right ?
Consider logging a HADOOP issue.
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257397#comment-16257397
]
Sean Busbey commented on HBASE-19289:
-
How would you propose relaxing it?
Two solutions I see:
* updating LocalFileSystem to support hflush/hsync
* telling CommonFSUtils to ignore the check on LocalFileSystem
The former is probably going to be easier due to the way we wrap FileSystems.
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257158#comment-16257158
]
Ted Yu commented on HBASE-19289:
[~busbey]:
To unblock testing against hadoop3, how about relaxing the following for unit
tests ?
{code}
if (!(CommonFSUtils.hasCapability(output, "hflush"))) {
throw new StreamLacksCapabilityException("hflush");
{code}
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19289) CommonFSUtils$StreamLacksCapabilityException: hflush when running test against hadoop3 beta1
[
https://issues.apache.org/jira/browse/HBASE-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256200#comment-16256200
]
Sean Busbey commented on HBASE-19289:
-
Sounds like LocalFileSystem doesn't support flush/sync. That's odd. Let's do a
quick check if Hadoop can provide that and then update tests to deal with it as
necessary.
> CommonFSUtils$StreamLacksCapabilityException: hflush when running test
> against hadoop3 beta1
>
>
> Key: HBASE-19289
> URL: https://issues.apache.org/jira/browse/HBASE-19289
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>
> As of commit d8fb10c8329b19223c91d3cda6ef149382ad4ea0 , I encountered the
> following exception when running unit test against hadoop3 beta1:
> {code}
> testRefreshStoreFiles(org.apache.hadoop.hbase.regionserver.TestHStore) Time
> elapsed: 0.061 sec <<< ERROR!
> java.io.IOException: cannot get log writer
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> Caused by:
> org.apache.hadoop.hbase.util.CommonFSUtils$StreamLacksCapabilityException:
> hflush
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.initHRegion(TestHStore.java:215)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:220)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:195)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:190)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:185)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:179)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.init(TestHStore.java:173)
> at
> org.apache.hadoop.hbase.regionserver.TestHStore.testRefreshStoreFiles(TestHStore.java:962)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
