[jira] [Commented] (HDFS-8480) Fix performance and timeout issues in HDFS-7929: use hard-links instead of copying edit logs
[
https://issues.apache.org/jira/browse/HDFS-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14596671#comment-14596671
]
Colin Patrick McCabe commented on HDFS-8480:
Thanks, [~zhz]. +1.
Fix performance and timeout issues in HDFS-7929: use hard-links instead of
copying edit logs
Key: HDFS-8480
URL: https://issues.apache.org/jira/browse/HDFS-8480
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.7.0
Reporter: Zhe Zhang
Assignee: Zhe Zhang
Priority: Critical
Attachments: HDFS-8480.00.patch, HDFS-8480.01.patch,
HDFS-8480.02.patch, HDFS-8480.03.patch
HDFS-7929 copies existing edit logs to the storage directory of the upgraded
{{NameNode}}. This slows down the upgrade process. This JIRA aims to use
hard-linking instead of per-op copying to achieve the same goal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8480) Fix performance and timeout issues in HDFS-7929: use hard-links instead of copying edit logs
[
https://issues.apache.org/jira/browse/HDFS-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14594400#comment-14594400
]
Zhe Zhang commented on HDFS-8480:
-
The test in the patch verifies that the earliest edit log op in the pre-created
{{/current}} directory (a mkdir of path {{/TestUpgrade}}) was successfully
retrieved.
Fix performance and timeout issues in HDFS-7929: use hard-links instead of
copying edit logs
Key: HDFS-8480
URL: https://issues.apache.org/jira/browse/HDFS-8480
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.7.0
Reporter: Zhe Zhang
Assignee: Zhe Zhang
Priority: Critical
Attachments: HDFS-8480.00.patch, HDFS-8480.01.patch,
HDFS-8480.02.patch, HDFS-8480.03.patch
HDFS-7929 copies existing edit logs to the storage directory of the upgraded
{{NameNode}}. This slows down the upgrade process. This JIRA aims to use
hard-linking instead of per-op copying to achieve the same goal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8480) Fix performance and timeout issues in HDFS-7929: use hard-links instead of copying edit logs
[
https://issues.apache.org/jira/browse/HDFS-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14594385#comment-14594385
]
Hadoop QA commented on HDFS-8480:
-
\\
\\
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 17m 55s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 32s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 40s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 2m 12s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 34s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 35s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 3m 15s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 14s | Pre-build of native portion |
| {color:green}+1{color} | hdfs tests | 161m 44s | Tests passed in hadoop-hdfs.
|
| | | 208m 7s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12740571/HDFS-8480.03.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 49f5d20 |
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11415/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/11415/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11415/console |
This message was automatically generated.
Fix performance and timeout issues in HDFS-7929: use hard-links instead of
copying edit logs
Key: HDFS-8480
URL: https://issues.apache.org/jira/browse/HDFS-8480
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.7.0
Reporter: Zhe Zhang
Assignee: Zhe Zhang
Priority: Critical
Attachments: HDFS-8480.00.patch, HDFS-8480.01.patch,
HDFS-8480.02.patch, HDFS-8480.03.patch
HDFS-7929 copies existing edit logs to the storage directory of the upgraded
{{NameNode}}. This slows down the upgrade process. This JIRA aims to use
hard-linking instead of per-op copying to achieve the same goal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8480) Fix performance and timeout issues in HDFS-7929: use hard-links instead of copying edit logs
[
https://issues.apache.org/jira/browse/HDFS-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14590891#comment-14590891
]
Colin Patrick McCabe commented on HDFS-8480:
Hi [~zhz],
We have to distinguish between two cases where the edit log is read:
1. when the edit log is read NN during startup to catch up with edits that
didn't make it into the fsimage yet
2. when the NN reads the edit log to handle an inotify request
Case #1 should be validating that the version is the newest version, since the
upgrade process ensures this.
Case #2 should not, since otherwise we will fail when reading old edit logs.
Fix performance and timeout issues in HDFS-7929: use hard-links instead of
copying edit logs
Key: HDFS-8480
URL: https://issues.apache.org/jira/browse/HDFS-8480
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.7.0
Reporter: Zhe Zhang
Assignee: Zhe Zhang
Priority: Critical
Attachments: HDFS-8480.00.patch, HDFS-8480.01.patch,
HDFS-8480.02.patch
HDFS-7929 copies existing edit logs to the storage directory of the upgraded
{{NameNode}}. This slows down the upgrade process. This JIRA aims to use
hard-linking instead of per-op copying to achieve the same goal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8480) Fix performance and timeout issues in HDFS-7929: use hard-links instead of copying edit logs
[
https://issues.apache.org/jira/browse/HDFS-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14587754#comment-14587754
]
Hadoop QA commented on HDFS-8480:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 18m 11s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 20s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 33s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 2m 18s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 31s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 3m 13s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 15s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 102m 48s | Tests failed in hadoop-hdfs. |
| | | 149m 7s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.hdfs.TestDFSUpgrade |
| | hadoop.hdfs.TestAppendSnapshotTruncate |
| Timed out tests |
org.apache.hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12739790/HDFS-8480.02.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 32e39d8 |
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11367/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/11367/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11367/console |
This message was automatically generated.
Fix performance and timeout issues in HDFS-7929: use hard-links instead of
copying edit logs
Key: HDFS-8480
URL: https://issues.apache.org/jira/browse/HDFS-8480
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.7.0
Reporter: Zhe Zhang
Assignee: Zhe Zhang
Priority: Critical
Attachments: HDFS-8480.00.patch, HDFS-8480.01.patch,
HDFS-8480.02.patch
HDFS-7929 copies existing edit logs to the storage directory of the upgraded
{{NameNode}}. This slows down the upgrade process. This JIRA aims to use
hard-linking instead of per-op copying to achieve the same goal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8480) Fix performance and timeout issues in HDFS-7929: use hard-links instead of copying edit logs
[
https://issues.apache.org/jira/browse/HDFS-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14587279#comment-14587279
]
Arpit Agarwal commented on HDFS-8480:
-
[~zhz] oddly I cannot reproduce the Jenkins test failure with your patch
applied locally.
Fix performance and timeout issues in HDFS-7929: use hard-links instead of
copying edit logs
Key: HDFS-8480
URL: https://issues.apache.org/jira/browse/HDFS-8480
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.7.0
Reporter: Zhe Zhang
Assignee: Zhe Zhang
Priority: Critical
Attachments: HDFS-8480.00.patch, HDFS-8480.01.patch
HDFS-7929 copies existing edit logs to the storage directory of the upgraded
{{NameNode}}. This slows down the upgrade process. This JIRA aims to use
hard-linking instead of per-op copying to achieve the same goal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8480) Fix performance and timeout issues in HDFS-7929: use hard-links instead of copying edit logs
[
https://issues.apache.org/jira/browse/HDFS-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14587427#comment-14587427
]
Zhe Zhang commented on HDFS-8480:
-
Thanks for the comments Vinod and Arpit.
I tried the patch again and got the same failure. To verify that the failure is
caused by edit log version I changed the following line and removed the {{-1}}:
{code}
out.create(NameNodeLayoutVersion.CURRENT_LAYOUT_VERSION - 1); -
out.create(NameNodeLayoutVersion.CURRENT_LAYOUT_VERSION);
{code}
And then the test passes. I think this expected when
{{EditLogFileInputStream#init()}} is called with {{verifyLayoutVersion}} as
true.
Fix performance and timeout issues in HDFS-7929: use hard-links instead of
copying edit logs
Key: HDFS-8480
URL: https://issues.apache.org/jira/browse/HDFS-8480
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.7.0
Reporter: Zhe Zhang
Assignee: Zhe Zhang
Priority: Critical
Attachments: HDFS-8480.00.patch, HDFS-8480.01.patch
HDFS-7929 copies existing edit logs to the storage directory of the upgraded
{{NameNode}}. This slows down the upgrade process. This JIRA aims to use
hard-linking instead of per-op copying to achieve the same goal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8480) Fix performance and timeout issues in HDFS-7929: use hard-links instead of copying edit logs
[
https://issues.apache.org/jira/browse/HDFS-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14586919#comment-14586919
]
Vinod Kumar Vavilapalli commented on HDFS-8480:
---
Appreciate the movement on this [~zhz] and [~cmccabe]!
[~zhz], assuming everything else is addressed, can you look at the test issue
reported by Jenkins so that we can get this in 2.7.1? Thanks again!
Fix performance and timeout issues in HDFS-7929: use hard-links instead of
copying edit logs
Key: HDFS-8480
URL: https://issues.apache.org/jira/browse/HDFS-8480
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.7.0
Reporter: Zhe Zhang
Assignee: Zhe Zhang
Priority: Critical
Attachments: HDFS-8480.00.patch, HDFS-8480.01.patch
HDFS-7929 copies existing edit logs to the storage directory of the upgraded
{{NameNode}}. This slows down the upgrade process. This JIRA aims to use
hard-linking instead of per-op copying to achieve the same goal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8480) Fix performance and timeout issues in HDFS-7929: use hard-links instead of copying edit logs
[
https://issues.apache.org/jira/browse/HDFS-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14584380#comment-14584380
]
Hadoop QA commented on HDFS-8480:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 18m 23s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 47s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 55s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 21s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 2m 16s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 34s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 3m 17s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 18s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 161m 15s | Tests failed in hadoop-hdfs. |
| | | 208m 43s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.hdfs.TestDFSUpgrade |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12739349/HDFS-8480.01.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / eef7b50 |
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11333/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/11333/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11333/console |
This message was automatically generated.
Fix performance and timeout issues in HDFS-7929: use hard-links instead of
copying edit logs
Key: HDFS-8480
URL: https://issues.apache.org/jira/browse/HDFS-8480
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.7.0
Reporter: Zhe Zhang
Assignee: Zhe Zhang
Priority: Critical
Attachments: HDFS-8480.00.patch, HDFS-8480.01.patch
HDFS-7929 copies existing edit logs to the storage directory of the upgraded
{{NameNode}}. This slows down the upgrade process. This JIRA aims to use
hard-linking instead of per-op copying to achieve the same goal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8480) Fix performance and timeout issues in HDFS-7929: use hard-links instead of copying edit logs
[
https://issues.apache.org/jira/browse/HDFS-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14577993#comment-14577993
]
Colin Patrick McCabe commented on HDFS-8480:
+1, pending adding the unit test you talked about earlier. Thanks, [~zhz].
Fix performance and timeout issues in HDFS-7929: use hard-links instead of
copying edit logs
Key: HDFS-8480
URL: https://issues.apache.org/jira/browse/HDFS-8480
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.7.0
Reporter: Zhe Zhang
Assignee: Zhe Zhang
Priority: Critical
Attachments: HDFS-8480.00.patch
HDFS-7929 copies existing edit logs to the storage directory of the upgraded
{{NameNode}}. This slows down the upgrade process. This JIRA aims to use
hard-linking instead of per-op copying to achieve the same goal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8480) Fix performance and timeout issues in HDFS-7929: use hard-links instead of copying edit logs
[
https://issues.apache.org/jira/browse/HDFS-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14575169#comment-14575169
]
Hadoop QA commented on HDFS-8480:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 15m 5s | Findbugs (version ) appears to
be broken on trunk. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 30s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 34s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 50s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 35s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 35s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 3m 15s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 13s | Pre-build of native portion |
| {color:green}+1{color} | hdfs tests | 160m 59s | Tests passed in hadoop-hdfs.
|
| | | 203m 1s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12737987/HDFS-8480.00.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 7588585 |
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/11245/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/11245/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/11245/console |
This message was automatically generated.
Fix performance and timeout issues in HDFS-7929: use hard-links instead of
copying edit logs
Key: HDFS-8480
URL: https://issues.apache.org/jira/browse/HDFS-8480
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.7.0
Reporter: Zhe Zhang
Assignee: Zhe Zhang
Priority: Critical
Attachments: HDFS-8480.00.patch
HDFS-7929 copies existing edit logs to the storage directory of the upgraded
{{NameNode}}. This slows down the upgrade process. This JIRA aims to use
hard-linking instead of per-op copying to achieve the same goal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8480) Fix performance and timeout issues in HDFS-7929: use hard-links instead of copying edit logs
[
https://issues.apache.org/jira/browse/HDFS-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573947#comment-14573947
]
Colin Patrick McCabe commented on HDFS-8480:
Maybe I can take a look in a little bit...
Fix performance and timeout issues in HDFS-7929: use hard-links instead of
copying edit logs
Key: HDFS-8480
URL: https://issues.apache.org/jira/browse/HDFS-8480
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.7.0
Reporter: Zhe Zhang
Assignee: Zhe Zhang
Priority: Critical
HDFS-7929 copies existing edit logs to the storage directory of the upgraded
{{NameNode}}. This slows down the upgrade process. This JIRA aims to use
hard-linking instead of per-op copying to achieve the same goal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8480) Fix performance and timeout issues in HDFS-7929: use hard-links instead of copying edit logs
[
https://issues.apache.org/jira/browse/HDFS-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573881#comment-14573881
]
Andrew Wang commented on HDFS-8480:
---
If we don't get this in, we should probably revert HDFS-7929 from branch-2.7,
since it can greatly slow down JN startup. Can retarget this for 2.8.0 then.
Fix performance and timeout issues in HDFS-7929: use hard-links instead of
copying edit logs
Key: HDFS-8480
URL: https://issues.apache.org/jira/browse/HDFS-8480
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.7.0
Reporter: Zhe Zhang
Assignee: Zhe Zhang
Priority: Critical
HDFS-7929 copies existing edit logs to the storage directory of the upgraded
{{NameNode}}. This slows down the upgrade process. This JIRA aims to use
hard-linking instead of per-op copying to achieve the same goal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8480) Fix performance and timeout issues in HDFS-7929: use hard-links instead of copying edit logs
[
https://issues.apache.org/jira/browse/HDFS-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573818#comment-14573818
]
Vinod Kumar Vavilapalli commented on HDFS-8480:
---
[~zhz]/[~andrew.wang],
The root-cause HDFS-7929 was in 2.7.0. Do we have any proposals that we can get
in shortly for 2.7.1 for this issue. Otherwise, we should consider moving this
out, assuming this is not a major regression. Please comment, tx.
Fix performance and timeout issues in HDFS-7929: use hard-links instead of
copying edit logs
Key: HDFS-8480
URL: https://issues.apache.org/jira/browse/HDFS-8480
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.7.0
Reporter: Zhe Zhang
Assignee: Zhe Zhang
Priority: Critical
HDFS-7929 copies existing edit logs to the storage directory of the upgraded
{{NameNode}}. This slows down the upgrade process. This JIRA aims to use
hard-linking instead of per-op copying to achieve the same goal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
