[jira] [Commented] (HDFS-8480) Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs instead of copying them

2015-07-31 Thread Ming Ma (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14650156#comment-14650156
 ] 

Ming Ma commented on HDFS-8480:
---

Make sense. It is easier to check in an edit log with an older version. Thanks 
[~zhz] and [~cmccabe].

> Fix performance and timeout issues in HDFS-7929 by using hard-links to 
> preserve old edit logs instead of copying them
> -
>
> Key: HDFS-8480
> URL: https://issues.apache.org/jira/browse/HDFS-8480
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 2.7.0
>Reporter: Zhe Zhang
>Assignee: Zhe Zhang
>Priority: Critical
> Fix For: 2.7.1
>
> Attachments: HDFS-8480.00.patch, HDFS-8480.01.patch, 
> HDFS-8480.02.patch, HDFS-8480.03.patch
>
>
> HDFS-7929 copies existing edit logs to the storage directory of the upgraded 
> {{NameNode}}. This slows down the upgrade process. This JIRA aims to use 
> hard-linking instead of per-op copying to achieve the same goal.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8480) Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs instead of copying them

2015-07-31 Thread Colin Patrick McCabe (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14649924#comment-14649924
 ] 

Colin Patrick McCabe commented on HDFS-8480:


Thanks, [~zhz].

> Fix performance and timeout issues in HDFS-7929 by using hard-links to 
> preserve old edit logs instead of copying them
> -
>
> Key: HDFS-8480
> URL: https://issues.apache.org/jira/browse/HDFS-8480
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 2.7.0
>Reporter: Zhe Zhang
>Assignee: Zhe Zhang
>Priority: Critical
> Fix For: 2.7.1
>
> Attachments: HDFS-8480.00.patch, HDFS-8480.01.patch, 
> HDFS-8480.02.patch, HDFS-8480.03.patch
>
>
> HDFS-7929 copies existing edit logs to the storage directory of the upgraded 
> {{NameNode}}. This slows down the upgrade process. This JIRA aims to use 
> hard-linking instead of per-op copying to achieve the same goal.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8480) Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs instead of copying them

2015-07-31 Thread Zhe Zhang (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14649875#comment-14649875
 ] 

Zhe Zhang commented on HDFS-8480:
-

Thanks for the discussion Ming and Colin!

I created HDFS-8846 to add old-version edit logs for testing.

> Fix performance and timeout issues in HDFS-7929 by using hard-links to 
> preserve old edit logs instead of copying them
> -
>
> Key: HDFS-8480
> URL: https://issues.apache.org/jira/browse/HDFS-8480
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 2.7.0
>Reporter: Zhe Zhang
>Assignee: Zhe Zhang
>Priority: Critical
> Fix For: 2.7.1
>
> Attachments: HDFS-8480.00.patch, HDFS-8480.01.patch, 
> HDFS-8480.02.patch, HDFS-8480.03.patch
>
>
> HDFS-7929 copies existing edit logs to the storage directory of the upgraded 
> {{NameNode}}. This slows down the upgrade process. This JIRA aims to use 
> hard-linking instead of per-op copying to achieve the same goal.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8480) Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs instead of copying them

2015-07-30 Thread Colin Patrick McCabe (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14648076#comment-14648076
 ] 

Colin Patrick McCabe commented on HDFS-8480:


Good catch, [~mingma].  If we make a change later to the edit log format, this 
test will break because we'll be writing out an edit log with the new format, 
but the old version ID.

I think we should simply check in a small binary edit log in order to do this 
testing.  This is similar to what we do with other "ugprade scenario" tests.  
As [~zhz] mentioned, it would be a lot of effort to support writing out edit 
logs in older formats (we have literally dozens of versions).

> Fix performance and timeout issues in HDFS-7929 by using hard-links to 
> preserve old edit logs instead of copying them
> -
>
> Key: HDFS-8480
> URL: https://issues.apache.org/jira/browse/HDFS-8480
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 2.7.0
>Reporter: Zhe Zhang
>Assignee: Zhe Zhang
>Priority: Critical
> Fix For: 2.7.1
>
> Attachments: HDFS-8480.00.patch, HDFS-8480.01.patch, 
> HDFS-8480.02.patch, HDFS-8480.03.patch
>
>
> HDFS-7929 copies existing edit logs to the storage directory of the upgraded 
> {{NameNode}}. This slows down the upgrade process. This JIRA aims to use 
> hard-linking instead of per-op copying to achieve the same goal.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8480) Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs instead of copying them

2015-07-28 Thread Zhe Zhang (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14645511#comment-14645511
 ] 

Zhe Zhang commented on HDFS-8480:
-

[~mingma] Thanks for the good catch! Yes this is indeed a flaw in the patch.

I can't think of another use case requiring us to extend {{FSEditLogOp}} as you 
suggested. But testing upgrade scenarios itself is a pretty strong motivation. 
The only alternative is to manually create some old edit log files.

Once we agree upon a plan I'm happy to make the change.

> Fix performance and timeout issues in HDFS-7929 by using hard-links to 
> preserve old edit logs instead of copying them
> -
>
> Key: HDFS-8480
> URL: https://issues.apache.org/jira/browse/HDFS-8480
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 2.7.0
>Reporter: Zhe Zhang
>Assignee: Zhe Zhang
>Priority: Critical
> Fix For: 2.7.1
>
> Attachments: HDFS-8480.00.patch, HDFS-8480.01.patch, 
> HDFS-8480.02.patch, HDFS-8480.03.patch
>
>
> HDFS-7929 copies existing edit logs to the storage directory of the upgraded 
> {{NameNode}}. This slows down the upgrade process. This JIRA aims to use 
> hard-linking instead of per-op copying to achieve the same goal.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8480) Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs instead of copying them

2015-07-28 Thread Ming Ma (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14645399#comment-14645399
 ] 

Ming Ma commented on HDFS-8480:
---

Thanks [~zhz].

The test code simulates older version edit by creating EditLogFileOutputStream 
with the prior version. However, the FSEditLogOp's writeFields implementation 
doesn't distinguish different versions and always write with the latest 
features.

For example, if this patch is applied directly on the top of the 2.6 release 
where CURRENT_LAYOUT_VERSION is set to -60, the test will fail because even 
though EditLogFileOutputStream is created with version -59, OP_ADD's 
writeFields will add storagePolicyId to the edit. During the edit loading, it 
will skip the reading of storagePolicyId given it has version -59 and thus 
cause checksum error.

To simulates old edit for test purpose, it seems we need to have FSEditLogOp 
support writing optional data depending on the version requested. So in this 
case, if the version is set to -59, it shouldn't write storagePolicyId. This 
functionality seems only useful for testing upgrade scenario.

Thought?

> Fix performance and timeout issues in HDFS-7929 by using hard-links to 
> preserve old edit logs instead of copying them
> -
>
> Key: HDFS-8480
> URL: https://issues.apache.org/jira/browse/HDFS-8480
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 2.7.0
>Reporter: Zhe Zhang
>Assignee: Zhe Zhang
>Priority: Critical
> Fix For: 2.7.1
>
> Attachments: HDFS-8480.00.patch, HDFS-8480.01.patch, 
> HDFS-8480.02.patch, HDFS-8480.03.patch
>
>
> HDFS-7929 copies existing edit logs to the storage directory of the upgraded 
> {{NameNode}}. This slows down the upgrade process. This JIRA aims to use 
> hard-linking instead of per-op copying to achieve the same goal.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Commented] (HDFS-8480) Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs instead of copying them

2015-06-23 Thread wsb
您的邮件已收到!谢谢!

[jira] [Commented] (HDFS-8480) Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs instead of copying them

2015-06-23 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14597822#comment-14597822
 ] 

Hudson commented on HDFS-8480:
--

FAILURE: Integrated in Hadoop-Mapreduce-trunk #2183 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2183/])
HDFS-8480. Fix performance and timeout issues in HDFS-7929 by using hard-links 
to preserve old edit logs, instead of copying them. (Zhe Zhang via Colin P. 
McCabe) (cmccabe: rev 7b424f938c3c306795d574792b086d84e4f06425)
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNUpgradeUtil.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSUpgrade.java


> Fix performance and timeout issues in HDFS-7929 by using hard-links to 
> preserve old edit logs instead of copying them
> -
>
> Key: HDFS-8480
> URL: https://issues.apache.org/jira/browse/HDFS-8480
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 2.7.0
>Reporter: Zhe Zhang
>Assignee: Zhe Zhang
>Priority: Critical
> Fix For: 2.7.1
>
> Attachments: HDFS-8480.00.patch, HDFS-8480.01.patch, 
> HDFS-8480.02.patch, HDFS-8480.03.patch
>
>
> HDFS-7929 copies existing edit logs to the storage directory of the upgraded 
> {{NameNode}}. This slows down the upgrade process. This JIRA aims to use 
> hard-linking instead of per-op copying to achieve the same goal.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Auto-Re: [jira] [Commented] (HDFS-8480) Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs instead of copying them

2015-06-23 Thread wsb
您的邮件已收到!谢谢!

[jira] [Commented] (HDFS-8480) Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs instead of copying them

2015-06-23 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14597780#comment-14597780
 ] 

Hudson commented on HDFS-8480:
--

FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #235 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/235/])
HDFS-8480. Fix performance and timeout issues in HDFS-7929 by using hard-links 
to preserve old edit logs, instead of copying them. (Zhe Zhang via Colin P. 
McCabe) (cmccabe: rev 7b424f938c3c306795d574792b086d84e4f06425)
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSUpgrade.java
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNUpgradeUtil.java


> Fix performance and timeout issues in HDFS-7929 by using hard-links to 
> preserve old edit logs instead of copying them
> -
>
> Key: HDFS-8480
> URL: https://issues.apache.org/jira/browse/HDFS-8480
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 2.7.0
>Reporter: Zhe Zhang
>Assignee: Zhe Zhang
>Priority: Critical
> Fix For: 2.7.1
>
> Attachments: HDFS-8480.00.patch, HDFS-8480.01.patch, 
> HDFS-8480.02.patch, HDFS-8480.03.patch
>
>
> HDFS-7929 copies existing edit logs to the storage directory of the upgraded 
> {{NameNode}}. This slows down the upgrade process. This JIRA aims to use 
> hard-linking instead of per-op copying to achieve the same goal.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8480) Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs instead of copying them

2015-06-23 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14597719#comment-14597719
 ] 

Hudson commented on HDFS-8480:
--

FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #226 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/226/])
HDFS-8480. Fix performance and timeout issues in HDFS-7929 by using hard-links 
to preserve old edit logs, instead of copying them. (Zhe Zhang via Colin P. 
McCabe) (cmccabe: rev 7b424f938c3c306795d574792b086d84e4f06425)
* 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSUpgrade.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNUpgradeUtil.java


> Fix performance and timeout issues in HDFS-7929 by using hard-links to 
> preserve old edit logs instead of copying them
> -
>
> Key: HDFS-8480
> URL: https://issues.apache.org/jira/browse/HDFS-8480
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 2.7.0
>Reporter: Zhe Zhang
>Assignee: Zhe Zhang
>Priority: Critical
> Fix For: 2.7.1
>
> Attachments: HDFS-8480.00.patch, HDFS-8480.01.patch, 
> HDFS-8480.02.patch, HDFS-8480.03.patch
>
>
> HDFS-7929 copies existing edit logs to the storage directory of the upgraded 
> {{NameNode}}. This slows down the upgrade process. This JIRA aims to use 
> hard-linking instead of per-op copying to achieve the same goal.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8480) Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs instead of copying them

2015-06-23 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14597705#comment-14597705
 ] 

Hudson commented on HDFS-8480:
--

FAILURE: Integrated in Hadoop-Hdfs-trunk #2165 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/2165/])
HDFS-8480. Fix performance and timeout issues in HDFS-7929 by using hard-links 
to preserve old edit logs, instead of copying them. (Zhe Zhang via Colin P. 
McCabe) (cmccabe: rev 7b424f938c3c306795d574792b086d84e4f06425)
* 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSUpgrade.java
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNUpgradeUtil.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt


> Fix performance and timeout issues in HDFS-7929 by using hard-links to 
> preserve old edit logs instead of copying them
> -
>
> Key: HDFS-8480
> URL: https://issues.apache.org/jira/browse/HDFS-8480
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 2.7.0
>Reporter: Zhe Zhang
>Assignee: Zhe Zhang
>Priority: Critical
> Fix For: 2.7.1
>
> Attachments: HDFS-8480.00.patch, HDFS-8480.01.patch, 
> HDFS-8480.02.patch, HDFS-8480.03.patch
>
>
> HDFS-7929 copies existing edit logs to the storage directory of the upgraded 
> {{NameNode}}. This slows down the upgrade process. This JIRA aims to use 
> hard-linking instead of per-op copying to achieve the same goal.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8480) Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs instead of copying them

2015-06-23 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14597531#comment-14597531
 ] 

Hudson commented on HDFS-8480:
--

SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #237 (See 
[https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/237/])
HDFS-8480. Fix performance and timeout issues in HDFS-7929 by using hard-links 
to preserve old edit logs, instead of copying them. (Zhe Zhang via Colin P. 
McCabe) (cmccabe: rev 7b424f938c3c306795d574792b086d84e4f06425)
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSUpgrade.java
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNUpgradeUtil.java


> Fix performance and timeout issues in HDFS-7929 by using hard-links to 
> preserve old edit logs instead of copying them
> -
>
> Key: HDFS-8480
> URL: https://issues.apache.org/jira/browse/HDFS-8480
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 2.7.0
>Reporter: Zhe Zhang
>Assignee: Zhe Zhang
>Priority: Critical
> Fix For: 2.7.1
>
> Attachments: HDFS-8480.00.patch, HDFS-8480.01.patch, 
> HDFS-8480.02.patch, HDFS-8480.03.patch
>
>
> HDFS-7929 copies existing edit logs to the storage directory of the upgraded 
> {{NameNode}}. This slows down the upgrade process. This JIRA aims to use 
> hard-linking instead of per-op copying to achieve the same goal.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8480) Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs instead of copying them

2015-06-23 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14597509#comment-14597509
 ] 

Hudson commented on HDFS-8480:
--

FAILURE: Integrated in Hadoop-Yarn-trunk #967 (See 
[https://builds.apache.org/job/Hadoop-Yarn-trunk/967/])
HDFS-8480. Fix performance and timeout issues in HDFS-7929 by using hard-links 
to preserve old edit logs, instead of copying them. (Zhe Zhang via Colin P. 
McCabe) (cmccabe: rev 7b424f938c3c306795d574792b086d84e4f06425)
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNUpgradeUtil.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSUpgrade.java


> Fix performance and timeout issues in HDFS-7929 by using hard-links to 
> preserve old edit logs instead of copying them
> -
>
> Key: HDFS-8480
> URL: https://issues.apache.org/jira/browse/HDFS-8480
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 2.7.0
>Reporter: Zhe Zhang
>Assignee: Zhe Zhang
>Priority: Critical
> Fix For: 2.7.1
>
> Attachments: HDFS-8480.00.patch, HDFS-8480.01.patch, 
> HDFS-8480.02.patch, HDFS-8480.03.patch
>
>
> HDFS-7929 copies existing edit logs to the storage directory of the upgraded 
> {{NameNode}}. This slows down the upgrade process. This JIRA aims to use 
> hard-linking instead of per-op copying to achieve the same goal.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8480) Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs instead of copying them

2015-06-22 Thread Zhe Zhang (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14596786#comment-14596786
 ] 

Zhe Zhang commented on HDFS-8480:
-

Thanks Colin for the review! And helpful comments from Vinod, Andrew, and Arpit.

> Fix performance and timeout issues in HDFS-7929 by using hard-links to 
> preserve old edit logs instead of copying them
> -
>
> Key: HDFS-8480
> URL: https://issues.apache.org/jira/browse/HDFS-8480
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 2.7.0
>Reporter: Zhe Zhang
>Assignee: Zhe Zhang
>Priority: Critical
> Fix For: 2.7.1
>
> Attachments: HDFS-8480.00.patch, HDFS-8480.01.patch, 
> HDFS-8480.02.patch, HDFS-8480.03.patch
>
>
> HDFS-7929 copies existing edit logs to the storage directory of the upgraded 
> {{NameNode}}. This slows down the upgrade process. This JIRA aims to use 
> hard-linking instead of per-op copying to achieve the same goal.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (HDFS-8480) Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs instead of copying them

2015-06-22 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/HDFS-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14596703#comment-14596703
 ] 

Hudson commented on HDFS-8480:
--

FAILURE: Integrated in Hadoop-trunk-Commit #8046 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/8046/])
HDFS-8480. Fix performance and timeout issues in HDFS-7929 by using hard-links 
to preserve old edit logs, instead of copying them. (Zhe Zhang via Colin P. 
McCabe) (cmccabe: rev 7b424f938c3c306795d574792b086d84e4f06425)
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNUpgradeUtil.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSUpgrade.java


> Fix performance and timeout issues in HDFS-7929 by using hard-links to 
> preserve old edit logs instead of copying them
> -
>
> Key: HDFS-8480
> URL: https://issues.apache.org/jira/browse/HDFS-8480
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 2.7.0
>Reporter: Zhe Zhang
>Assignee: Zhe Zhang
>Priority: Critical
> Attachments: HDFS-8480.00.patch, HDFS-8480.01.patch, 
> HDFS-8480.02.patch, HDFS-8480.03.patch
>
>
> HDFS-7929 copies existing edit logs to the storage directory of the upgraded 
> {{NameNode}}. This slows down the upgrade process. This JIRA aims to use 
> hard-linking instead of per-op copying to achieve the same goal.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)