[jira] Updated: (MAPREDUCE-2132) Need a command line option in RaidShell to fix blocks using raid
[ https://issues.apache.org/jira/browse/MAPREDUCE-2132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramkumar Vadali updated MAPREDUCE-2132: --- Attachment: MAPREDUCE-2132.3.patch Reverted unintentional change to CHANGES.txt > Need a command line option in RaidShell to fix blocks using raid > > > Key: MAPREDUCE-2132 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2132 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/raid >Reporter: Ramkumar Vadali >Assignee: Ramkumar Vadali > Fix For: 0.22.0 > > Attachments: MAPREDUCE-2132.2.patch, MAPREDUCE-2132.3.patch, > MAPREDUCE-2132.patch > > > RaidShell currently has an option to recover a file and return the path to > the recovered file. The administrator can then rename the recovered file to > the damaged file. > The problem with this is that the file metadata is altered, specifically the > modification time. Instead we need a way to just repair the damaged blocks > and send the fixed blocks to a data node. > Once this is done, we can put automation around it. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2132) Need a command line option in RaidShell to fix blocks using raid
[
https://issues.apache.org/jira/browse/MAPREDUCE-2132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923343#action_12923343
]
Scott Chen commented on MAPREDUCE-2132:
---
Hey Ram: Can you remove the change in CHANGES.txt.
{code}
-MAPREDUCE-2140. Regenerate fair scheduler design doc PDF. (matei)
-
{code}
> Need a command line option in RaidShell to fix blocks using raid
>
>
> Key: MAPREDUCE-2132
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2132
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: contrib/raid
>Reporter: Ramkumar Vadali
>Assignee: Ramkumar Vadali
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-2132.2.patch, MAPREDUCE-2132.patch
>
>
> RaidShell currently has an option to recover a file and return the path to
> the recovered file. The administrator can then rename the recovered file to
> the damaged file.
> The problem with this is that the file metadata is altered, specifically the
> modification time. Instead we need a way to just repair the damaged blocks
> and send the fixed blocks to a data node.
> Once this is done, we can put automation around it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2134) ant binary-system is broken in mapreduce project.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923342#action_12923342 ] Ranjit Mathew commented on MAPREDUCE-2134: -- Yeah, so much for ivy/maven easing the build process. :-( I nuked "~/.ivy2", applied your patch and re-built using "ant binary-system" and this time it works. Thanks a lot. +1 for your patch. > ant binary-system is broken in mapreduce project. > - > > Key: MAPREDUCE-2134 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2134 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: build >Affects Versions: 0.21.1 >Reporter: Vinay Kumar Thota >Assignee: Konstantin Boudnik > Attachments: MAPREDUCE-2134.patch > > > Build failed due to unable to copy the commons instrumented jar. I could see > the following error in the log. > binary-system: > [copy] Copying 5 files to > /home/vinay/mapreduce/build-fi/system/hadoop-mapred-0.22.0-SNAPSHOT > BUILD FAILED > /home/vinay/mapreduce/build.xml:1307: Warning: Could not find file > /home/vinay/mapreduce/build-fi/ivy/lib/Hadoop/system/hadoop-common-instrumented-0.22.0-SNAPSHOT.jar > to copy. > It's pointing to the wrong path to copy the file. Actually the correct path > is, > /home/vinay/mapreduce/build-fi/system/ivy/Hadoop/system -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2134) ant binary-system is broken in mapreduce project.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923337#action_12923337 ] Konstantin Boudnik commented on MAPREDUCE-2134: --- Ranjit, your problem seems to caused by some outdated artifacts left either in ivy cache or in your local m2 repository. I have ran trunk build on a pristine machine (where I have built MR before) and your issue doesn't seem to exist. If you keep seeing it - please feel free to open a new JIRA. > ant binary-system is broken in mapreduce project. > - > > Key: MAPREDUCE-2134 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2134 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: build >Affects Versions: 0.21.1 >Reporter: Vinay Kumar Thota >Assignee: Konstantin Boudnik > Attachments: MAPREDUCE-2134.patch > > > Build failed due to unable to copy the commons instrumented jar. I could see > the following error in the log. > binary-system: > [copy] Copying 5 files to > /home/vinay/mapreduce/build-fi/system/hadoop-mapred-0.22.0-SNAPSHOT > BUILD FAILED > /home/vinay/mapreduce/build.xml:1307: Warning: Could not find file > /home/vinay/mapreduce/build-fi/ivy/lib/Hadoop/system/hadoop-common-instrumented-0.22.0-SNAPSHOT.jar > to copy. > It's pointing to the wrong path to copy the file. Actually the correct path > is, > /home/vinay/mapreduce/build-fi/system/ivy/Hadoop/system -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2134) ant binary-system is broken in mapreduce project.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Boudnik updated MAPREDUCE-2134: -- Component/s: (was: test) build Tags: herriot > ant binary-system is broken in mapreduce project. > - > > Key: MAPREDUCE-2134 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2134 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: build >Affects Versions: 0.21.1 >Reporter: Vinay Kumar Thota >Assignee: Konstantin Boudnik > Attachments: MAPREDUCE-2134.patch > > > Build failed due to unable to copy the commons instrumented jar. I could see > the following error in the log. > binary-system: > [copy] Copying 5 files to > /home/vinay/mapreduce/build-fi/system/hadoop-mapred-0.22.0-SNAPSHOT > BUILD FAILED > /home/vinay/mapreduce/build.xml:1307: Warning: Could not find file > /home/vinay/mapreduce/build-fi/ivy/lib/Hadoop/system/hadoop-common-instrumented-0.22.0-SNAPSHOT.jar > to copy. > It's pointing to the wrong path to copy the file. Actually the correct path > is, > /home/vinay/mapreduce/build-fi/system/ivy/Hadoop/system -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2134) ant binary-system is broken in mapreduce project.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923336#action_12923336 ] Konstantin Boudnik commented on MAPREDUCE-2134: --- Awesome, I'll commit it in the morning after usual validation. > ant binary-system is broken in mapreduce project. > - > > Key: MAPREDUCE-2134 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2134 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: test >Affects Versions: 0.21.1 >Reporter: Vinay Kumar Thota >Assignee: Konstantin Boudnik > Attachments: MAPREDUCE-2134.patch > > > Build failed due to unable to copy the commons instrumented jar. I could see > the following error in the log. > binary-system: > [copy] Copying 5 files to > /home/vinay/mapreduce/build-fi/system/hadoop-mapred-0.22.0-SNAPSHOT > BUILD FAILED > /home/vinay/mapreduce/build.xml:1307: Warning: Could not find file > /home/vinay/mapreduce/build-fi/ivy/lib/Hadoop/system/hadoop-common-instrumented-0.22.0-SNAPSHOT.jar > to copy. > It's pointing to the wrong path to copy the file. Actually the correct path > is, > /home/vinay/mapreduce/build-fi/system/ivy/Hadoop/system -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2134) ant binary-system is broken in mapreduce project.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923330#action_12923330 ] Vinay Kumar Thota commented on MAPREDUCE-2134: -- Patch works for me. +1. > ant binary-system is broken in mapreduce project. > - > > Key: MAPREDUCE-2134 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2134 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: test >Affects Versions: 0.21.1 >Reporter: Vinay Kumar Thota >Assignee: Konstantin Boudnik > Attachments: MAPREDUCE-2134.patch > > > Build failed due to unable to copy the commons instrumented jar. I could see > the following error in the log. > binary-system: > [copy] Copying 5 files to > /home/vinay/mapreduce/build-fi/system/hadoop-mapred-0.22.0-SNAPSHOT > BUILD FAILED > /home/vinay/mapreduce/build.xml:1307: Warning: Could not find file > /home/vinay/mapreduce/build-fi/ivy/lib/Hadoop/system/hadoop-common-instrumented-0.22.0-SNAPSHOT.jar > to copy. > It's pointing to the wrong path to copy the file. Actually the correct path > is, > /home/vinay/mapreduce/build-fi/system/ivy/Hadoop/system -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (MAPREDUCE-2144) CLONE -ant binary-system is broken
CLONE -ant binary-system is broken -- Key: MAPREDUCE-2144 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2144 Project: Hadoop Map/Reduce Issue Type: Bug Components: test Affects Versions: 0.21.1 Reporter: Konstantin Boudnik Assignee: Konstantin Boudnik Build failed due to unable to copy the commons instrumented jar. I could see the following error in the log. binary-system: [copy] Copying 5 files to /home/vinay/mapreduce/build-fi/system/hadoop-mapred-0.22.0-SNAPSHOT BUILD FAILED /home/vinay/mapreduce/build.xml:1307: Warning: Could not find file /home/vinay/mapreduce/build-fi/ivy/lib/Hadoop/system/hadoop-common-instrumented-0.22.0-SNAPSHOT.jar to copy. It's pointing to the wrong path to copy the file. Actually the correct path is, /home/vinay/mapreduce/build-fi/system/ivy/Hadoop/system -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (MAPREDUCE-2134) ant binary-system is broken in mapreduce project.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Boudnik reassigned MAPREDUCE-2134: - Assignee: Konstantin Boudnik > ant binary-system is broken in mapreduce project. > - > > Key: MAPREDUCE-2134 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2134 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: test >Affects Versions: 0.21.1 >Reporter: Vinay Kumar Thota >Assignee: Konstantin Boudnik > Attachments: MAPREDUCE-2134.patch > > > Build failed due to unable to copy the commons instrumented jar. I could see > the following error in the log. > binary-system: > [copy] Copying 5 files to > /home/vinay/mapreduce/build-fi/system/hadoop-mapred-0.22.0-SNAPSHOT > BUILD FAILED > /home/vinay/mapreduce/build.xml:1307: Warning: Could not find file > /home/vinay/mapreduce/build-fi/ivy/lib/Hadoop/system/hadoop-common-instrumented-0.22.0-SNAPSHOT.jar > to copy. > It's pointing to the wrong path to copy the file. Actually the correct path > is, > /home/vinay/mapreduce/build-fi/system/ivy/Hadoop/system -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2134) ant binary-system is broken in mapreduce project.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Boudnik updated MAPREDUCE-2134: -- Affects Version/s: 0.21.1 > ant binary-system is broken in mapreduce project. > - > > Key: MAPREDUCE-2134 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2134 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: test >Affects Versions: 0.21.1 >Reporter: Vinay Kumar Thota >Assignee: Konstantin Boudnik > Attachments: MAPREDUCE-2134.patch > > > Build failed due to unable to copy the commons instrumented jar. I could see > the following error in the log. > binary-system: > [copy] Copying 5 files to > /home/vinay/mapreduce/build-fi/system/hadoop-mapred-0.22.0-SNAPSHOT > BUILD FAILED > /home/vinay/mapreduce/build.xml:1307: Warning: Could not find file > /home/vinay/mapreduce/build-fi/ivy/lib/Hadoop/system/hadoop-common-instrumented-0.22.0-SNAPSHOT.jar > to copy. > It's pointing to the wrong path to copy the file. Actually the correct path > is, > /home/vinay/mapreduce/build-fi/system/ivy/Hadoop/system -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2134) ant binary-system is broken in mapreduce project.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Boudnik updated MAPREDUCE-2134: -- Attachment: MAPREDUCE-2134.patch Yes, I can see the issue with 'missing' jar file too. The patch addresses the problem. Thanks for catching it, Vinay. Please confirm that it works for you too. > ant binary-system is broken in mapreduce project. > - > > Key: MAPREDUCE-2134 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2134 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: test >Reporter: Vinay Kumar Thota > Attachments: MAPREDUCE-2134.patch > > > Build failed due to unable to copy the commons instrumented jar. I could see > the following error in the log. > binary-system: > [copy] Copying 5 files to > /home/vinay/mapreduce/build-fi/system/hadoop-mapred-0.22.0-SNAPSHOT > BUILD FAILED > /home/vinay/mapreduce/build.xml:1307: Warning: Could not find file > /home/vinay/mapreduce/build-fi/ivy/lib/Hadoop/system/hadoop-common-instrumented-0.22.0-SNAPSHOT.jar > to copy. > It's pointing to the wrong path to copy the file. Actually the correct path > is, > /home/vinay/mapreduce/build-fi/system/ivy/Hadoop/system -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2132) Need a command line option in RaidShell to fix blocks using raid
[ https://issues.apache.org/jira/browse/MAPREDUCE-2132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923216#action_12923216 ] Scott Chen commented on MAPREDUCE-2132: --- +1 The patch looks good. > Need a command line option in RaidShell to fix blocks using raid > > > Key: MAPREDUCE-2132 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2132 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/raid >Reporter: Ramkumar Vadali >Assignee: Ramkumar Vadali > Fix For: 0.22.0 > > Attachments: MAPREDUCE-2132.2.patch, MAPREDUCE-2132.patch > > > RaidShell currently has an option to recover a file and return the path to > the recovered file. The administrator can then rename the recovered file to > the damaged file. > The problem with this is that the file metadata is altered, specifically the > modification time. Instead we need a way to just repair the damaged blocks > and send the fixed blocks to a data node. > Once this is done, we can put automation around it. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1970) Reed-Solomon code implementation to be used in raid
[ https://issues.apache.org/jira/browse/MAPREDUCE-1970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923184#action_12923184 ] Ramkumar Vadali commented on MAPREDUCE-1970: +1 This looks good. Could you post the ant test and ant test-patch results? > Reed-Solomon code implementation to be used in raid > --- > > Key: MAPREDUCE-1970 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1970 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: contrib/raid >Reporter: Scott Chen >Assignee: Scott Chen > Attachments: MAPREDUCE-1970-v2.txt, MAPREDUCE-1970.txt > > > A Reed-Solomon erasure code implementation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1970) Reed-Solomon code implementation to be used in raid
[ https://issues.apache.org/jira/browse/MAPREDUCE-1970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923182#action_12923182 ] dhruba borthakur commented on MAPREDUCE-1970: - At this point, I would just like to see the ReedSolomon code make it into trunk. Once that is done, we ca figure out how we can make it pluggable so that other implementations of the same algorithm can be used. > Reed-Solomon code implementation to be used in raid > --- > > Key: MAPREDUCE-1970 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1970 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: contrib/raid >Reporter: Scott Chen >Assignee: Scott Chen > Attachments: MAPREDUCE-1970-v2.txt, MAPREDUCE-1970.txt > > > A Reed-Solomon erasure code implementation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-987) Exposing MiniDFS and MiniMR clusters as a single process command-line
[
https://issues.apache.org/jira/browse/MAPREDUCE-987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923157#action_12923157
]
Konstantin Boudnik commented on MAPREDUCE-987:
--
Phillip, is there any intention to address last comments from Chris, so this
can be committed into trunk?
> Exposing MiniDFS and MiniMR clusters as a single process command-line
> -
>
> Key: MAPREDUCE-987
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-987
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: build, test
>Reporter: Philip Zeyliger
>Assignee: Philip Zeyliger
>Priority: Minor
> Attachments: HDFS-621-0.20-patch, HDFS-621.patch, MAPREDUCE-987.patch
>
>
> It's hard to test non-Java programs that rely on significant mapreduce
> functionality. The patch I'm proposing shortly will let you just type
> "bin/hadoop jar hadoop-hdfs-hdfswithmr-test.jar minicluster" to start a
> cluster (internally, it's using Mini{MR,HDFS}Cluster) with a specified number
> of daemons, etc. A test that checks how some external process interacts with
> Hadoop might start minicluster as a subprocess, run through its thing, and
> then simply kill the java subprocess.
> I've been using just such a system for a couple of weeks, and I like it.
> It's significantly easier than developing a lot of scripts to start a
> pseudo-distributed cluster, and then clean up after it. I figure others
> might find it useful as well.
> I'm at a bit of a loss as to where to put it in 0.21. hdfs-with-mr tests
> have all the required libraries, so I've put it there. I could conceivably
> split this into "minimr" and "minihdfs", but it's specifically the fact that
> they're configured to talk to each other that I like about having them
> together. And one JVM is better than two for my test programs.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2132) Need a command line option in RaidShell to fix blocks using raid
[ https://issues.apache.org/jira/browse/MAPREDUCE-2132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923087#action_12923087 ] Ramkumar Vadali commented on MAPREDUCE-2132: The latest patch passes testing: [exec] [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 7 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 system tests framework. The patch passed system tests framework compile. [exec] [exec] [exec] [exec] [exec] == [exec] == [exec] Finished build. [exec] == [exec] == [exec] test-junit: [junit] WARNING: multiple versions of ant detected in path for junit [junit] jar:file:/home/rvadali/local/external/ant/lib/ant.jar!/org/apache/tools/ant/Project.class [junit] and jar:file:/home/rvadali/.ivy2/cache/ant/ant/jars/ant-1.6.5.jar!/org/apache/tools/ant/Project.class [junit] Running org.apache.hadoop.hdfs.TestRaidDfs [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 31.133 sec [junit] Running org.apache.hadoop.raid.TestDirectoryTraversal [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 8.116 sec [junit] Running org.apache.hadoop.raid.TestHarIndexParser [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.052 sec [junit] Running org.apache.hadoop.raid.TestRaidHar [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 68.637 sec [junit] Running org.apache.hadoop.raid.TestRaidNode [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 523.123 sec [junit] Running org.apache.hadoop.raid.TestRaidPurge [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 43.853 sec [junit] Running org.apache.hadoop.raid.TestRaidShell [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 24.41 sec test: BUILD SUCCESSFUL Total time: 11 minutes 55 seconds > Need a command line option in RaidShell to fix blocks using raid > > > Key: MAPREDUCE-2132 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2132 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/raid >Reporter: Ramkumar Vadali >Assignee: Ramkumar Vadali > Fix For: 0.22.0 > > Attachments: MAPREDUCE-2132.2.patch, MAPREDUCE-2132.patch > > > RaidShell currently has an option to recover a file and return the path to > the recovered file. The administrator can then rename the recovered file to > the damaged file. > The problem with this is that the file metadata is altered, specifically the > modification time. Instead we need a way to just repair the damaged blocks > and send the fixed blocks to a data node. > Once this is done, we can put automation around it. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1970) Reed-Solomon code implementation to be used in raid
[ https://issues.apache.org/jira/browse/MAPREDUCE-1970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923061#action_12923061 ] Scott Chen commented on MAPREDUCE-1970: --- Hey Dhruba, I don't know about the overhead of JNI. But I feel using the intel one may be faster. Does this works on other CPU like AMD? Do we need license to use it? > Reed-Solomon code implementation to be used in raid > --- > > Key: MAPREDUCE-1970 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1970 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: contrib/raid >Reporter: Scott Chen >Assignee: Scott Chen > Attachments: MAPREDUCE-1970-v2.txt, MAPREDUCE-1970.txt > > > A Reed-Solomon erasure code implementation. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2104) Rumen TraceBuilder Does Not Emit CPU/Memory Usage Details in Traces
[ https://issues.apache.org/jira/browse/MAPREDUCE-2104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12922880#action_12922880 ] Ravi Gummadi commented on MAPREDUCE-2104: - Some comments on the patch: (1) Why are the variable names and method names start with the word "expected" ? Since these are the values that are already seen(and not the values expected), please remove the word "expected" everywhere. (2) You missed making changes to HadoopLogAnalyzer.incorporateCounters(). (3) You missed making changes to LoggedTaskAttempt.deepCompare(). (4) Empty comments as javadoc are seen for the new methods. Please add some comments there. (5) Better to change the new(and may be existing ones also ?) data members of LoggedTaskAttempt private and use setters and getters everywhere because even if I change getExpectedVirtualMemorySize() to return expectedVirtualMemorySizeInBytes-2, testcase is not failing. > Rumen TraceBuilder Does Not Emit CPU/Memory Usage Details in Traces > --- > > Key: MAPREDUCE-2104 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2104 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: tools/rumen >Affects Versions: 0.22.0 >Reporter: Ranjit Mathew >Assignee: Amar Kamat > Attachments: mapreduce-2104-v1.1.patch > > > Via MAPREDUCE-220, we now have CPU/Memory usage information in MapReduce > JobHistory files. However, Rumen's TraceBuilder > does not emit this information in the JSON traces. Without this information, > GridMix3 cannot emulate CPU/Memory usage correctly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2132) Need a command line option in RaidShell to fix blocks using raid
[ https://issues.apache.org/jira/browse/MAPREDUCE-2132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramkumar Vadali updated MAPREDUCE-2132: --- Attachment: MAPREDUCE-2132.2.patch Incorporated Scott's comments. Scott, using endsWith() does not work since the corrupt file is under the RAID HAR directory, so the path will end with part- > Need a command line option in RaidShell to fix blocks using raid > > > Key: MAPREDUCE-2132 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2132 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/raid >Reporter: Ramkumar Vadali >Assignee: Ramkumar Vadali > Fix For: 0.22.0 > > Attachments: MAPREDUCE-2132.2.patch, MAPREDUCE-2132.patch > > > RaidShell currently has an option to recover a file and return the path to > the recovered file. The administrator can then rename the recovered file to > the damaged file. > The problem with this is that the file metadata is altered, specifically the > modification time. Instead we need a way to just repair the damaged blocks > and send the fixed blocks to a data node. > Once this is done, we can put automation around it. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
