[jira] [Commented] (HBASE-4553) The update of .tableinfo is not atomic; we remove then rename
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144579#comment-13144579 ] Hudson commented on HBASE-4553: --- Integrated in HBase-TRUNK #2411 (See [https://builds.apache.org/job/HBase-TRUNK/2411/]) HBASE-4553 The update of .tableinfo is not atomic; we remove then rename stack : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/HConstants.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/HMerge.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/Merge.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestFSTableDescriptorForceCreation.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestFSTableDescriptors.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestFSUtils.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/util/TestMergeTool.java > The update of .tableinfo is not atomic; we remove then rename > - > > Key: HBASE-4553 > URL: https://issues.apache.org/jira/browse/HBASE-4553 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 0.92.0 > > Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v12.txt, > 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v13.txt, 4553-v5.txt, > 4553-v9.txt, HBase-4553-TestAvroServer.patch > > > This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists > already. In 0.20+ its better but still 'some' issues if existing reader when > file is renamed. This issue is about fixing this (though we depend on fix > first being in hdfs). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not atomic; we remove then rename
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144497#comment-13144497 ] Hudson commented on HBASE-4553: --- Integrated in HBase-0.92 #109 (See [https://builds.apache.org/job/HBase-0.92/109/]) HBASE-4553 The update of .tableinfo is not atomic; we remove then rename stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/HConstants.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/HMerge.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/Merge.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestDrainingServer.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestFSTableDescriptorForceCreation.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/TestFSTableDescriptors.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/TestFSUtils.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/util/TestMergeTool.java > The update of .tableinfo is not atomic; we remove then rename > - > > Key: HBASE-4553 > URL: https://issues.apache.org/jira/browse/HBASE-4553 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 0.92.0 > > Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v12.txt, > 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v13.txt, 4553-v5.txt, > 4553-v9.txt, HBase-4553-TestAvroServer.patch > > > This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists > already. In 0.20+ its better but still 'some' issues if existing reader when > file is renamed. This issue is about fixing this (though we depend on fix > first being in hdfs). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not atomic; we remove then rename
[
https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144435#comment-13144435
]
stack commented on HBASE-4553:
--
I ran test suite on local ubuntu and all passed except roll log; it had this:
{code}
Tests in error:
testLogRollOnDatanodeDeath(org.apache.hadoop.hbase.regionserver.wal.TestLogRolling):
Cannot lock storage
/export1/stack/hbase/target/test-data/74208e88-deaa-4d3c-a6ed-d316c5aa5ef2/dfscluster_0515d0e6-8fcd-47c0-8b34-9126722fc1cb/dfs/data/data3.
The directory is already locked.
{code}
which is unrelated. Committing v13.
> The update of .tableinfo is not atomic; we remove then rename
> -
>
> Key: HBASE-4553
> URL: https://issues.apache.org/jira/browse/HBASE-4553
> Project: HBase
> Issue Type: Task
>Reporter: stack
>Assignee: stack
>Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v12.txt,
> 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v13.txt, 4553-v5.txt,
> 4553-v9.txt, HBase-4553-TestAvroServer.patch
>
>
> This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists
> already. In 0.20+ its better but still 'some' issues if existing reader when
> file is renamed. This issue is about fixing this (though we depend on fix
> first being in hdfs).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not atomic; we remove then rename
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144321#comment-13144321 ] Hadoop QA commented on HBASE-4553: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12502501/4553-v13.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 27 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -164 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 48 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.thrift2.TestThriftHBaseServiceHandler org.apache.hadoop.hbase.client.TestAdmin org.apache.hadoop.hbase.master.TestDistributedLogSplitting org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort org.apache.hadoop.hbase.master.TestMasterFailover Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/177//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/177//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/177//console This message is automatically generated. > The update of .tableinfo is not atomic; we remove then rename > - > > Key: HBASE-4553 > URL: https://issues.apache.org/jira/browse/HBASE-4553 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 0.92.0 > > Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v12.txt, > 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v13.txt, 4553-v5.txt, > 4553-v9.txt, HBase-4553-TestAvroServer.patch > > > This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists > already. In 0.20+ its better but still 'some' issues if existing reader when > file is renamed. This issue is about fixing this (though we depend on fix > first being in hdfs). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not atomic; we remove then rename
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13143615#comment-13143615 ] Hadoop QA commented on HBASE-4553: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12502233/4553-v12.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 24 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -164 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 48 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/166//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/166//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/166//console This message is automatically generated. > The update of .tableinfo is not atomic; we remove then rename > - > > Key: HBASE-4553 > URL: https://issues.apache.org/jira/browse/HBASE-4553 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 0.92.0 > > Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v12.txt, > 4553-v12.txt, 4553-v12.txt, 4553-v12.txt, 4553-v5.txt, 4553-v9.txt, > HBase-4553-TestAvroServer.patch > > > This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists > already. In 0.20+ its better but still 'some' issues if existing reader when > file is renamed. This issue is about fixing this (though we depend on fix > first being in hdfs). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not atomic; we remove then rename
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13143572#comment-13143572 ] Hadoop QA commented on HBASE-4553: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12502227/4553-v12.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 24 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -164 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 48 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/164//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/164//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/164//console This message is automatically generated. > The update of .tableinfo is not atomic; we remove then rename > - > > Key: HBASE-4553 > URL: https://issues.apache.org/jira/browse/HBASE-4553 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 0.92.0 > > Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v12.txt, > 4553-v12.txt, 4553-v12.txt, 4553-v5.txt, 4553-v9.txt, > HBase-4553-TestAvroServer.patch > > > This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists > already. In 0.20+ its better but still 'some' issues if existing reader when > file is renamed. This issue is about fixing this (though we depend on fix > first being in hdfs). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not atomic; we remove then rename
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13143466#comment-13143466 ] Hadoop QA commented on HBASE-4553: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12502178/4553-v12.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 24 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -164 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 48 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.thrift2.TestThriftHBaseServiceHandler org.apache.hadoop.hbase.client.TestAdmin org.apache.hadoop.hbase.master.TestDistributedLogSplitting org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint org.apache.hadoop.hbase.TestGlobalMemStoreSize org.apache.hadoop.hbase.master.TestMasterFailover org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/158//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/158//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/158//console This message is automatically generated. > The update of .tableinfo is not atomic; we remove then rename > - > > Key: HBASE-4553 > URL: https://issues.apache.org/jira/browse/HBASE-4553 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 0.92.0 > > Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v12.txt, > 4553-v12.txt, 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch > > > This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists > already. In 0.20+ its better but still 'some' issues if existing reader when > file is renamed. This issue is about fixing this (though we depend on fix > first being in hdfs). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not atomic; we remove then rename
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13143312#comment-13143312 ] Hadoop QA commented on HBASE-4553: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12502166/4553-v12.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 24 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -164 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 48 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestAdmin org.apache.hadoop.hbase.replication.TestReplication org.apache.hadoop.hbase.master.TestMasterFailover Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/155//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/155//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/155//console This message is automatically generated. > The update of .tableinfo is not atomic; we remove then rename > - > > Key: HBASE-4553 > URL: https://issues.apache.org/jira/browse/HBASE-4553 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 0.92.0 > > Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v12.txt, > 4553-v5.txt, 4553-v9.txt, HBase-4553-TestAvroServer.patch > > > This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists > already. In 0.20+ its better but still 'some' issues if existing reader when > file is renamed. This issue is about fixing this (though we depend on fix > first being in hdfs). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not atomic; we remove then rename
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13142829#comment-13142829 ] Hadoop QA commented on HBASE-4553: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12501986/4553-v11.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 24 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/151//console This message is automatically generated. > The update of .tableinfo is not atomic; we remove then rename > - > > Key: HBASE-4553 > URL: https://issues.apache.org/jira/browse/HBASE-4553 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 0.92.0 > > Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v11.txt, 4553-v5.txt, > 4553-v9.txt, HBase-4553-TestAvroServer.patch > > > This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists > already. In 0.20+ its better but still 'some' issues if existing reader when > file is renamed. This issue is about fixing this (though we depend on fix > first being in hdfs). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not atomic; we remove then rename
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13141501#comment-13141501 ] Hadoop QA commented on HBASE-4553: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12501803/4553-v10.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 24 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -166 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.wal.TestLogRolling org.apache.hadoop.hbase.master.TestDistributedLogSplitting org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/125//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/125//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/125//console This message is automatically generated. > The update of .tableinfo is not atomic; we remove then rename > - > > Key: HBASE-4553 > URL: https://issues.apache.org/jira/browse/HBASE-4553 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 0.92.0 > > Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v5.txt, 4553-v9.txt, > HBase-4553-TestAvroServer.patch > > > This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists > already. In 0.20+ its better but still 'some' issues if existing reader when > file is renamed. This issue is about fixing this (though we depend on fix > first being in hdfs). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not atomic; we remove then rename
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13141387#comment-13141387 ] jirapos...@reviews.apache.org commented on HBASE-4553: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2656/ --- (Updated 2011-11-01 17:52:00.184660) Review request for hbase. Changes --- Updated diff; more javadoc and tests. Summary --- Retry. This addresses bug hbase-4553. https://issues.apache.org/jira/browse/hbase-4553 Diffs (updated) - src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 46ca765 src/main/java/org/apache/hadoop/hbase/HConstants.java 76278a2 src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java b9d4b90 src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java 24570c9 src/main/java/org/apache/hadoop/hbase/util/FSUtils.java 36d90c3 src/main/java/org/apache/hadoop/hbase/util/HMerge.java a6f6b69 src/main/java/org/apache/hadoop/hbase/util/Merge.java 3aa980f src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java 0a6f703 src/test/java/org/apache/hadoop/hbase/TestFSTableDescriptorForceCreation.java 8404d1b src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java 49bcf02 src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java b2cb233 src/test/java/org/apache/hadoop/hbase/util/TestFSTableDescriptors.java 8fce6ec src/test/java/org/apache/hadoop/hbase/util/TestFSUtils.java 1ad30e6 src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java 21a57ca src/test/java/org/apache/hadoop/hbase/util/TestMergeTool.java e4b3d75 Diff: https://reviews.apache.org/r/2656/diff Testing --- Thanks, Michael > The update of .tableinfo is not atomic; we remove then rename > - > > Key: HBASE-4553 > URL: https://issues.apache.org/jira/browse/HBASE-4553 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 0.92.0 > > Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v5.txt, 4553-v9.txt, > HBase-4553-TestAvroServer.patch > > > This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists > already. In 0.20+ its better but still 'some' issues if existing reader when > file is renamed. This issue is about fixing this (though we depend on fix > first being in hdfs). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not atomic; we remove then rename
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13141382#comment-13141382 ] stack commented on HBASE-4553: -- The above TestLogRolling was because could not get lock on dir. > The update of .tableinfo is not atomic; we remove then rename > - > > Key: HBASE-4553 > URL: https://issues.apache.org/jira/browse/HBASE-4553 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 0.92.0 > > Attachments: 3446-v8.txt, 4553-v10.txt, 4553-v5.txt, 4553-v9.txt, > HBase-4553-TestAvroServer.patch > > > This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists > already. In 0.20+ its better but still 'some' issues if existing reader when > file is renamed. This issue is about fixing this (though we depend on fix > first being in hdfs). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not atomic; we remove then rename
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13141013#comment-13141013 ] Hadoop QA commented on HBASE-4553: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12501740/4553-v9.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 24 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -166 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.wal.TestLogRolling org.apache.hadoop.hbase.master.TestDistributedLogSplitting org.apache.hadoop.hbase.master.TestMasterFailover Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/119//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/119//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/119//console This message is automatically generated. > The update of .tableinfo is not atomic; we remove then rename > - > > Key: HBASE-4553 > URL: https://issues.apache.org/jira/browse/HBASE-4553 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 0.92.0 > > Attachments: 3446-v8.txt, 4553-v5.txt, 4553-v9.txt, > HBase-4553-TestAvroServer.patch > > > This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists > already. In 0.20+ its better but still 'some' issues if existing reader when > file is renamed. This issue is about fixing this (though we depend on fix > first being in hdfs). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not atomic; we remove then rename
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13140959#comment-13140959 ] stack commented on HBASE-4553: -- Retried upload to rb (Sorry for the no diff LarsH). > The update of .tableinfo is not atomic; we remove then rename > - > > Key: HBASE-4553 > URL: https://issues.apache.org/jira/browse/HBASE-4553 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 0.92.0 > > Attachments: 3446-v8.txt, 4553-v5.txt, 4553-v9.txt, > HBase-4553-TestAvroServer.patch > > > This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists > already. In 0.20+ its better but still 'some' issues if existing reader when > file is renamed. This issue is about fixing this (though we depend on fix > first being in hdfs). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not atomic; we remove then rename
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13140958#comment-13140958 ] jirapos...@reviews.apache.org commented on HBASE-4553: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2656/ --- Review request for hbase. Summary --- Retry. This addresses bug hbase-4553. https://issues.apache.org/jira/browse/hbase-4553 Diffs - Diff: https://reviews.apache.org/r/2656/diff Testing --- Thanks, Michael > The update of .tableinfo is not atomic; we remove then rename > - > > Key: HBASE-4553 > URL: https://issues.apache.org/jira/browse/HBASE-4553 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 0.92.0 > > Attachments: 3446-v8.txt, 4553-v5.txt, 4553-v9.txt, > HBase-4553-TestAvroServer.patch > > > This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists > already. In 0.20+ its better but still 'some' issues if existing reader when > file is renamed. This issue is about fixing this (though we depend on fix > first being in hdfs). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not atomic; we remove then rename
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13140957#comment-13140957 ] jirapos...@reviews.apache.org commented on HBASE-4553: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2656/ --- (Updated 2011-11-01 05:47:41.332884) Review request for hbase. Summary --- Retry. This addresses bug hbase-4553. https://issues.apache.org/jira/browse/hbase-4553 Diffs (updated) - src/main/java/org/apache/hadoop/hbase/HConstants.java 943b774 src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 46ca765 src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java b9d4b90 src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java 24570c9 src/main/java/org/apache/hadoop/hbase/util/FSUtils.java 36d90c3 src/main/java/org/apache/hadoop/hbase/util/HMerge.java a6f6b69 src/main/java/org/apache/hadoop/hbase/util/Merge.java 3aa980f src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java 1cfeea8 src/test/java/org/apache/hadoop/hbase/TestFSTableDescriptorForceCreation.java 8404d1b src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java 49bcf02 src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java b2cb233 src/test/java/org/apache/hadoop/hbase/util/TestFSTableDescriptors.java 8fce6ec src/test/java/org/apache/hadoop/hbase/util/TestFSUtils.java 1ad30e6 src/test/java/org/apache/hadoop/hbase/util/TestMergeTable.java ffc8d9d src/test/java/org/apache/hadoop/hbase/util/TestMergeTool.java 88dc9de Diff: https://reviews.apache.org/r/2656/diff Testing --- Thanks, Michael > The update of .tableinfo is not atomic; we remove then rename > - > > Key: HBASE-4553 > URL: https://issues.apache.org/jira/browse/HBASE-4553 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 0.92.0 > > Attachments: 3446-v8.txt, 4553-v5.txt, 4553-v9.txt, > HBase-4553-TestAvroServer.patch > > > This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists > already. In 0.20+ its better but still 'some' issues if existing reader when > file is renamed. This issue is about fixing this (though we depend on fix > first being in hdfs). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not atomic; we remove then rename
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13140679#comment-13140679 ] stack commented on HBASE-4553: -- I need to put back the bit where .tableinfo is made in a tmp dir Will make a new patch later. > The update of .tableinfo is not atomic; we remove then rename > - > > Key: HBASE-4553 > URL: https://issues.apache.org/jira/browse/HBASE-4553 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 0.92.0 > > Attachments: 3446-v8.txt, 4553-v5.txt, HBase-4553-TestAvroServer.patch > > > This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists > already. In 0.20+ its better but still 'some' issues if existing reader when > file is renamed. This issue is about fixing this (though we depend on fix > first being in hdfs). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not atomic; we remove then rename
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13140666#comment-13140666 ] Hadoop QA commented on HBASE-4553: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12501695/3446-v8.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 18 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/115//console This message is automatically generated. > The update of .tableinfo is not atomic; we remove then rename > - > > Key: HBASE-4553 > URL: https://issues.apache.org/jira/browse/HBASE-4553 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 0.92.0 > > Attachments: 3446-v8.txt, 4553-v5.txt, HBase-4553-TestAvroServer.patch > > > This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists > already. In 0.20+ its better but still 'some' issues if existing reader when > file is renamed. This issue is about fixing this (though we depend on fix > first being in hdfs). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not atomic; we remove then rename
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13130956#comment-13130956 ] Akash Ashok commented on HBASE-4553: https://issues.apache.org/jira/browse/HBASE-4621 opened for the TestCaseFix . Shall modify and submit the patch out there. Thanks > The update of .tableinfo is not atomic; we remove then rename > - > > Key: HBASE-4553 > URL: https://issues.apache.org/jira/browse/HBASE-4553 > Project: HBase > Issue Type: Task >Reporter: stack >Priority: Critical > Fix For: 0.92.0 > > Attachments: 4553-v5.txt, HBase-4553-TestAvroServer.patch > > > This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists > already. In 0.20+ its better but still 'some' issues if existing reader when > file is renamed. This issue is about fixing this (though we depend on fix > first being in hdfs). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not atomic; we remove then rename
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129660#comment-13129660 ] stack commented on HBASE-4553: -- Patch looks fine Akash (long as you ensure an upper bound on test as per Ted suggestion). I'd suggest you do it in another issue though rather than here. This one is about .tableinfo updating. Thanks boss. > The update of .tableinfo is not atomic; we remove then rename > - > > Key: HBASE-4553 > URL: https://issues.apache.org/jira/browse/HBASE-4553 > Project: HBase > Issue Type: Task >Reporter: stack >Priority: Critical > Fix For: 0.92.0 > > Attachments: HBase-4553-TestAvroServer.patch > > > This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists > already. In 0.20+ its better but still 'some' issues if existing reader when > file is renamed. This issue is about fixing this (though we depend on fix > first being in hdfs). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not atomic; we remove then rename
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129188#comment-13129188 ] Ted Yu commented on HBASE-4553: --- We should impose an upper limit on the amount of time we wait in the loop. > The update of .tableinfo is not atomic; we remove then rename > - > > Key: HBASE-4553 > URL: https://issues.apache.org/jira/browse/HBASE-4553 > Project: HBase > Issue Type: Task >Reporter: stack >Priority: Critical > Fix For: 0.92.0 > > Attachments: HBase-4553-TestAvroServer.patch > > > This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists > already. In 0.20+ its better but still 'some' issues if existing reader when > file is renamed. This issue is about fixing this (though we depend on fix > first being in hdfs). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not atomic; we remove then rename
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13128074#comment-13128074 ] Ted Yu commented on HBASE-4553: --- I don't see FNFE in TestAvroServer-output although the test failure is consistent. > The update of .tableinfo is not atomic; we remove then rename > - > > Key: HBASE-4553 > URL: https://issues.apache.org/jira/browse/HBASE-4553 > Project: HBase > Issue Type: Task >Reporter: stack >Priority: Critical > Fix For: 0.92.0 > > > This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists > already. In 0.20+ its better but still 'some' issues if existing reader when > file is renamed. This issue is about fixing this (though we depend on fix > first being in hdfs). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not atomic; we remove then rename
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123616#comment-13123616 ] stack commented on HBASE-4553: -- @Todd Thanks boss. > The update of .tableinfo is not atomic; we remove then rename > - > > Key: HBASE-4553 > URL: https://issues.apache.org/jira/browse/HBASE-4553 > Project: HBase > Issue Type: Task >Reporter: stack > > This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists > already. In 0.20+ its better but still 'some' issues if existing reader when > file is renamed. This issue is about fixing this (though we depend on fix > first being in hdfs). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not atomic; we remove then rename
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123603#comment-13123603 ] Todd Lipcon commented on HBASE-4553: atomic operations on filesystems are tricky... to do this "correctly" in the face of crashes, we need to have some process either do a rollback or roll-forward to recover from failures. Something like: writer: - create tableinfo.tmp - delete tableinfo - rename tableinfo.tmp to tableinfo reader: - try to read tableinfo - on IOE (block missing, etc), that means that the file was deleted underneath. So spin until the file open succeeds. if the writer crashes between the delete and rename, we need someone else to come in and "finish" the operation. IMO we need some general purpose way of allowing the master to keep an "intent log" in ZK for this kind of thing - and then if the master fails over, it can complete the operation. > The update of .tableinfo is not atomic; we remove then rename > - > > Key: HBASE-4553 > URL: https://issues.apache.org/jira/browse/HBASE-4553 > Project: HBase > Issue Type: Task >Reporter: stack > > This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists > already. In 0.20+ its better but still 'some' issues if existing reader when > file is renamed. This issue is about fixing this (though we depend on fix > first being in hdfs). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4553) The update of .tableinfo is not atomic; we remove then rename
[ https://issues.apache.org/jira/browse/HBASE-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123567#comment-13123567 ] stack commented on HBASE-4553: -- So, I see this in this failure up on jenkins in https://builds.apache.org/job/HBase-0.92/53/artifact/trunk/target/surefire-reports/org.apache.hadoop.hbase.avro.TestAvroServer-output.txt I see that we updated .tableinfo and a check on the files modtime at about the same time fails with filenotfound. Todd, should we spin here till file shows up? > The update of .tableinfo is not atomic; we remove then rename > - > > Key: HBASE-4553 > URL: https://issues.apache.org/jira/browse/HBASE-4553 > Project: HBase > Issue Type: Task >Reporter: stack > > This comes of HBASE-4547. The rename in 0.20 hdfs fails if file exists > already. In 0.20+ its better but still 'some' issues if existing reader when > file is renamed. This issue is about fixing this (though we depend on fix > first being in hdfs). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
