+1
Testing done and passed:
Unit tests:
- Linux
Benchmarks on 20, 100 and 500 nodes:
- TestDFSIO
- MRBench (aka SmallJobs)
- NNBench
- Sort (RandomWriter, Sort, SortValidation)
Thanks,
Mukund
Nigel Daley wrote:
I've created a candidate build for Hadoop 0.15.3.
http://people.apache.org/~n
[
https://issues.apache.org/jira/browse/HADOOP-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12558710#action_12558710
]
Mukund Madhugiri commented on HADOOP-2582:
--
+1. The output looks good
>
[
https://issues.apache.org/jira/browse/HADOOP-2570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Madhugiri updated HADOOP-2570:
-
Status: Patch Available (was: Open)
trying to trigger the patch process to pick it up
[
https://issues.apache.org/jira/browse/HADOOP-2570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Madhugiri updated HADOOP-2570:
-
Status: Open (was: Patch Available)
> streaming jobs fail after HADOOP-2
[
https://issues.apache.org/jira/browse/HADOOP-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12558162#action_12558162
]
Mukund Madhugiri commented on HADOOP-1650:
--
I did a re-run on the 500
[
https://issues.apache.org/jira/browse/HADOOP-2481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12558084#action_12558084
]
Mukund Madhugiri commented on HADOOP-2481:
--
+1
I tested it on a 500
[
https://issues.apache.org/jira/browse/HADOOP-1707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12558041#action_12558041
]
Mukund Madhugiri commented on HADOOP-1707:
--
Running on a 100 node cluster,
Issue Type: Bug
Components: mapred
Affects Versions: 0.16.0
Environment: solaris
Reporter: Mukund Madhugiri
Priority: Critical
Fix For: 0.16.0
Unit test failed in the nightly build
[
https://issues.apache.org/jira/browse/HADOOP-2449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12556639#action_12556639
]
Mukund Madhugiri commented on HADOOP-2449:
--
+1
> Restore the old NN Ben
+1
Testing done and passed:
Unit tests:
- Linux
Benchmarks on 20, 100 and 500 nodes:
- TestDFSIO
- MRBench (aka SmallJobs)
- Sort (RandomWriter, Sort, SortValidation)
Thanks,
Mukund
Nigel Daley wrote:
I've created a candidate build for Hadoop 0.15.2.
http://people.apache.org/~nigel/hadoop
Issue Type: Bug
Components: dfs
Affects Versions: 0.16.0
Environment: Solaris
Reporter: Mukund Madhugiri
Priority: Critical
Fix For: 0.16.0
Unit test failed on Solaris:
org.apache.hadoop.dfs.TestDFSStorageStateRecovery.unknown
Components: dfs
Affects Versions: 0.16.0
Environment: Windows
Reporter: Mukund Madhugiri
Priority: Critical
Fix For: 0.16.0
Unit test fails on Windows:
org.apache.hadoop.dfs.TestDFSShell.testFilePermissions
junit.framework.ComparisonFailure: expected
[
https://issues.apache.org/jira/browse/HADOOP-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12555368#action_12555368
]
Mukund Madhugiri commented on HADOOP-1298:
--
I ran the tests using the defa
[
https://issues.apache.org/jira/browse/HADOOP-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12554946
]
Mukund Madhugiri commented on HADOOP-1298:
--
|*20 nodes*|trunk|trunk + patch|
|Random Writer (hrs)|0.27
[
https://issues.apache.org/jira/browse/HADOOP-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12554868
]
Mukund Madhugiri commented on HADOOP-1650:
--
I ran the sort benchmark on 500 nodes and here is the data:
1
Components: fs
Affects Versions: 0.16.0
Environment: Windows
Reporter: Mukund Madhugiri
Priority: Blocker
Fix For: 0.16.0
Unit test failed on Windows:
org.apache.hadoop.fs.TestCopyFiles.testCopyFromLocalToDfs
Exception
Components: fs
Affects Versions: 0.16.0
Environment: Windows
Reporter: Mukund Madhugiri
Priority: Blocker
Fix For: 0.16.0
Unit test fails on Windows:
org.apache.hadoop.fs.TestCopyFiles.testCopyFromLocalToLocal
Exception
[
https://issues.apache.org/jira/browse/HADOOP-1650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12552554
]
Mukund Madhugiri commented on HADOOP-1650:
--
The patch failed to apply to trunk. Can you please attach a
Project: Hadoop
Issue Type: Bug
Components: fs
Affects Versions: 0.16.0
Environment: Linux
Reporter: Mukund Madhugiri
Priority: Critical
Fix For: 0.16.0
Unit test failed on Linux:
Test
[
https://issues.apache.org/jira/browse/HADOOP-1707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12551977
]
Mukund Madhugiri commented on HADOOP-1707:
--
I ran sort benchmark on 500 nodes and here is the data:
trunk
Affects Versions: 0.16.0
Reporter: Mukund Madhugiri
Fix For: 0.16.0
Unit test failed: TestInjectionForSimulatedStorage failed in the nightly build
with a timeout:
tail from the console:
[junit] 2007-12-12 12:02:18,674 INFO dfs.TestInjectionForSimulatedStorage
[
https://issues.apache.org/jira/browse/HADOOP-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Madhugiri updated HADOOP-2000:
-
Status: Patch Available (was: Open)
> Re-write NNBench to use MapRed
[
https://issues.apache.org/jira/browse/HADOOP-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Madhugiri updated HADOOP-2000:
-
Status: Open (was: Patch Available)
> Re-write NNBench to use MapRed
[
https://issues.apache.org/jira/browse/HADOOP-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Madhugiri updated HADOOP-2000:
-
Attachment: HADOOP-2000.patch
Updated patch addressing comments from Konstantin
>
[
https://issues.apache.org/jira/browse/HADOOP-2368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12549148
]
Mukund Madhugiri commented on HADOOP-2368:
--
is there a requirement that java has to be installed in cygwin
Issue Type: Bug
Components: dfs
Affects Versions: 0.16.0
Environment: windows
Reporter: Mukund Madhugiri
Priority: Critical
Fix For: 0.16.0
A lot (68) unit tests failed on Windows due to an exception: Login failed:
CreateProcess
Project: Hadoop
Issue Type: Bug
Components: dfs
Affects Versions: 0.16.0
Environment: windows
Reporter: Mukund Madhugiri
Fix For: 0.16.0
Output from failed test
[
https://issues.apache.org/jira/browse/HADOOP-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12548019
]
Mukund Madhugiri commented on HADOOP-1841:
--
I ran sort benchmark on 500 nodes and here is the data:
trunk
[
https://issues.apache.org/jira/browse/HADOOP-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12546858
]
Mukund Madhugiri commented on HADOOP-1841:
--
I ran nnbench benchmark on 500 nodes and here is the data
+1
Testing done and passed:
Unit tests:
- Linux
Benchmarks on 20, 100 and 500 nodes:
- TestDFSIO
- MRBench (aka SmallJobs)
- NNBench
- Sort (RandomWriter, Sort, SortValidation)
Thanks,
Mukund
Nigel Daley wrote:
I've created a candidate build for Hadoop 0.15.1.
http://people.apache.o
+1
Testing done and passed:
Unit tests:
- Linux
Benchmarks on 20, 100 and 500 nodes:
- TestDFSIO
- MRBench (aka SmallJobs)
- NNBench
- Sort (RandomWriter, Sort, SortValidation)
Thanks,
Mukund
Nigel Daley wrote:
I've created a candidate build for Hadoop 0.14.4.
http://people.apac
: 0.16.0
Environment: Windows
Reporter: Mukund Madhugiri
Priority: Critical
Fix For: 0.16.0
These 4 mapred unit tests are failing on Windows:
org.apache.hadoop.mapred.TestMiniMRClasspath.unknown
org.apache.hadoop.mapred.TestMiniMRLocalFS.unknown
[
https://issues.apache.org/jira/browse/HADOOP-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Madhugiri updated HADOOP-2000:
-
Status: Patch Available (was: Open)
> Re-write NNBench to use MapRed
[
https://issues.apache.org/jira/browse/HADOOP-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Madhugiri updated HADOOP-2000:
-
Attachment: HADOOP-2000.patch
Uploading new patch to address comments from Konstantin
[
https://issues.apache.org/jira/browse/HADOOP-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Madhugiri updated HADOOP-2000:
-
Status: Open (was: Patch Available)
> Re-write NNBench to use MapRed
[
https://issues.apache.org/jira/browse/HADOOP-2190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12542930
]
Mukund Madhugiri commented on HADOOP-2190:
--
Can the command output be along the lines of what we see on
[
https://issues.apache.org/jira/browse/HADOOP-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12541999
]
Mukund Madhugiri commented on HADOOP-2195:
--
for the first case, it needs to be documented that it follows
Versions: 0.15.1, 0.16.0
Reporter: Mukund Madhugiri
Fix For: 0.16.0
Assuming the dfs commands follow POSIX standards, there are some problems with
the DFS mkdir command. I compared the DFS output with that of RHEL 4u5:
1. mkdir a directory that exists:
Linux: mkdir: cannot
Components: dfs
Affects Versions: 0.15.1, 0.16.0
Reporter: Mukund Madhugiri
Fix For: 0.16.0
The dfs cat command throws an IOException when the file does not exist:
bin/hadoop dfs -cat doesnotexist
cat: java.io.IOException: Cannot open filename/doesnotexist]
In
Affects Versions: 0.15.1, 0.16.0
Reporter: Mukund Madhugiri
Fix For: 0.16.0
Assuming the dfs commands follow POSIX standards, there are some problems with
the DFS rm and rmr commands. I compared the DFS output with that of RHEL 4u5:
In both cases, if the file/directory
: 0.15.1, 0.16.0
Reporter: Mukund Madhugiri
Fix For: 0.16.0
Assuming the dfs commands follow POSIX standards, there are some problems with
the DFS mv command. I compared the DFS output with that of RHEL 4u5
1. mv a file:
Linux: No output
DFS: Renamed /testarea/hadoop
[
https://issues.apache.org/jira/browse/HADOOP-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Madhugiri updated HADOOP-2191:
-
Description:
Assuming the dfs commands follow POSIX standards, there are some problems
Affects Versions: 0.15.1, 0.16.0
Reporter: Mukund Madhugiri
Fix For: 0.16.0
Assuming the dfs commands follow POSIX standards, there are some problems with
the DFS ls and lsr commands. I compared the DFS output with that of RHEL 4u5
1. ls a directory when there are no
Affects Versions: 0.15.1, 0.16.0
Reporter: Mukund Madhugiri
Fix For: 0.16.0
Assuming the dfs commands follow POSIX standards, there are some problems with
the DFS ls and lsr commands. I compared the DFS output with that of RHEL 4u5
1. ls a directory when there are no
[
https://issues.apache.org/jira/browse/HADOOP-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Madhugiri updated HADOOP-1989:
-
Status: Patch Available (was: Open)
> Add support for simulated Data Nodes - help
[
https://issues.apache.org/jira/browse/HADOOP-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Madhugiri updated HADOOP-1989:
-
Status: Open (was: Patch Available)
> Add support for simulated Data Nodes - help
After further investigation, and looking at re-run results of the
benchmarks, I change my vote to +1.
Sort on 500 nodes, re-run #1: 2.43 hrs
Sort on 500 nodes, re-run #2: 2.36 hrs
Thanks,
Mukund
Mukund Madhugiri wrote:
I am voting -1 for this as I am seeing performance degradation in the
I am voting -1 for this as I am seeing performance degradation in the Sort
benchmarks with 500 nodes. I expect to have an update on the performance
problem by tomorrow.
Here is the data on the performance degradation:
Time taken for Sort benchmark on 500 nodes:
- Oct 23: 2.4 hrs
- Oct 25: 2.
[
https://issues.apache.org/jira/browse/HADOOP-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Madhugiri updated HADOOP-2000:
-
Status: Patch Available (was: Open)
> Re-write NNBench to use MapRed
[
https://issues.apache.org/jira/browse/HADOOP-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Madhugiri updated HADOOP-2000:
-
Status: Open (was: Patch Available)
> Re-write NNBench to use MapRed
[
https://issues.apache.org/jira/browse/HADOOP-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Madhugiri updated HADOOP-2000:
-
Attachment: HADOOP-2000.patch
> Re-write NNBench to use MapRed
[
https://issues.apache.org/jira/browse/HADOOP-2087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Madhugiri updated HADOOP-2087:
-
Description:
task task_200710200555_0005_m_000725_0 started writing a file and the Node
[
https://issues.apache.org/jira/browse/HADOOP-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Madhugiri updated HADOOP-2000:
-
Status: Patch Available (was: Open)
> Re-write NNBench to use MapRed
[
https://issues.apache.org/jira/browse/HADOOP-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Madhugiri updated HADOOP-2000:
-
Status: Open (was: Patch Available)
> Re-write NNBench to use MapRed
[
https://issues.apache.org/jira/browse/HADOOP-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Madhugiri updated HADOOP-2000:
-
Status: Patch Available (was: Open)
> Re-write NNBench to use MapRed
[
https://issues.apache.org/jira/browse/HADOOP-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Madhugiri updated HADOOP-2000:
-
Attachment: HADOOP-2000.patch
Updated patch addressing review comments, including
[
https://issues.apache.org/jira/browse/HADOOP-1942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12537381
]
Mukund Madhugiri commented on HADOOP-1942:
--
Please ignore my previous comment. The run did not actually go
[
https://issues.apache.org/jira/browse/HADOOP-2087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12537184
]
Mukund Madhugiri commented on HADOOP-2087:
--
I don't see this on every run of the TestDFSIO benchmark
[
https://issues.apache.org/jira/browse/HADOOP-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Madhugiri updated HADOOP-2000:
-
Status: Patch Available (was: Open)
Updated the patch to address review feedback
>
[
https://issues.apache.org/jira/browse/HADOOP-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Madhugiri updated HADOOP-2000:
-
Attachment: HADOOP-2000.patch
Updated patch addressing review comments
> Re-wr
Versions: 0.15.0
Environment: Linux
Reporter: Mukund Madhugiri
Priority: Blocker
Fix For: 0.15.0
The streaming tests failed after the fix for HADOOP-2080
Failing tests:
org.apache.hadoop.streaming.TestGzipInput.testCommandLine
Components: pipes
Affects Versions: 0.15.0
Environment: Linux, Solaris
Reporter: Mukund Madhugiri
Priority: Blocker
Fix For: 0.15.0
The test org.apache.hadoop.mapred.pipes.TestPipes.unknown failed on Solaris and
Linux.
Error output
+1
Unit tests passed on:
- Windows
- Linux
The following benchmark jobs passed on cluster sizes of 20, 100 and 500
nodes:
- TestDFSIO
- MRBench (aka SmallJobs)
- NNBench
- Sort (RandomWriter, Sort, SortValidation) - Also passed on 900 node cluster
Thanks,
Mukund
Doug Cuttin
[
https://issues.apache.org/jira/browse/HADOOP-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Madhugiri updated HADOOP-2000:
-
Attachment: HADOOP-2000.patch
Updated code to adhere to coding guidelines
> Re-wr
[
https://issues.apache.org/jira/browse/HADOOP-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Madhugiri updated HADOOP-1973:
-
Fix Version/s: 0.15.0
> NPE at JobTracker star
[
https://issues.apache.org/jira/browse/HADOOP-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Madhugiri updated HADOOP-1973:
-
Affects Version/s: (was: 0.14.1)
0.15.0
> NPE at JobTrac
[
https://issues.apache.org/jira/browse/HADOOP-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Madhugiri updated HADOOP-1973:
-
Priority: Blocker (was: Major)
Marking it a blocker. I see it with the sort benchmarks
[
https://issues.apache.org/jira/browse/HADOOP-1942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12534193
]
Mukund Madhugiri commented on HADOOP-1942:
--
Here is more data:
1. TPS without this fix, 40 threads
[
https://issues.apache.org/jira/browse/HADOOP-2020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Madhugiri updated HADOOP-2020:
-
Priority: Blocker (was: Major)
I feel this should to go into 0.15. Marking it a
Environment: Windows
Reporter: Mukund Madhugiri
Assignee: Hairong Kuang
Priority: Blocker
The following tests failed on Windows. It passed on Linux
org.apache.hadoop.fs.TestLocalDirAllocator.test0
org.apache.hadoop.fs.TestLocalDirAllocator.test1
[
https://issues.apache.org/jira/browse/HADOOP-1942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12533538
]
Mukund Madhugiri commented on HADOOP-1942:
--
I ran NNBench on a 500 node cluster and here is the comparison
[
https://issues.apache.org/jira/browse/HADOOP-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Madhugiri updated HADOOP-2000:
-
Attachment: HADOOP-2000.patch
> Re-write NNBench to use MapRed
[
https://issues.apache.org/jira/browse/HADOOP-2000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mukund Madhugiri updated HADOOP-2000:
-
Status: Patch Available (was: Open)
Attaching patch for code review
Here is a sample
+1
Unit tests pass on:
- Windows
- Linux
The following benchmark jobs pass on cluster sizes of 20, 100, 500 and 900
nodes:
- TestDFSIO
- MRBench (aka SmallJobs)
- NNBench
- Sort (RandomWriter, Sort, SortValidation)
Thanks,
Mukund
On 10/5/07 3:44 PM, "Doug Cutting" <[EM
Reporter: Mukund Madhugiri
Assignee: Mukund Madhugiri
The proposal is to re-write the NNBench benchmark/test to measure Namenode
operations using MapReduce. Two buckets of measurements will be done:
1. Transactions per second
2. Average latency
for these operations
- Create and
: mapred
Affects Versions: 0.15.0
Environment: Linux
Reporter: Mukund Madhugiri
Priority: Blocker
Sort validation has slowed down compared to previous benchmark runs
Here are the data points:
20 nodes: 0.39hr -> 1.36hr
100 nodes: 0.44hr -> 1.39hr
500
Hi,
There are 8 open blockers (3 with patch available) targeted for 0.14.2. Can
we get the blockers resolved by Wed (Oct 3) so we can wrap up the release,
barring any new issues?
Thanks,
Mukund
(Works for Nigel)
[
https://issues.apache.org/jira/browse/HADOOP-1894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12530189
]
Mukund Madhugiri commented on HADOOP-1894:
--
I will test it with the sort 500 benchmark
> Add fancy gra
Components: test
Affects Versions: 0.14.2
Environment: Windows
Reporter: Mukund Madhugiri
Fails with this assert error:
junit.framework.AssertionFailedError: filestatus.dat should be of size 16384
at
org.apache.hadoop.dfs.TestFileCreation.testFileCreation
Components: dfs
Affects Versions: 0.13.1
Reporter: Mukund Madhugiri
Running the du command without specifying a directory throws an exception. It
runs fine if a directory is specified. Here is the output
[EMAIL PROTECTED] hadoop]$ bin/hadoop dfs -du
Exception in thread
80 matches
Mail list logo
