http://git-wip-us.apache.org/repos/asf/hadoop/blob/d759b4bd/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.0/CHANGES.0.20.0.md ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.0/CHANGES.0.20.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.0/CHANGES.0.20.0.md new file mode 100644 index 0000000..2e03f9e --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.0/CHANGES.0.20.0.md @@ -0,0 +1,347 @@ + +<!--- +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +--> +# Apache Hadoop Changelog + +## Release 0.20.0 - 2009-04-22 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-5531](https://issues.apache.org/jira/browse/HADOOP-5531) | Remove Chukwa on branch-0.20 | Blocker | . | Nigel Daley | Nigel Daley | +| [HADOOP-4970](https://issues.apache.org/jira/browse/HADOOP-4970) | Use the full path when move files to .Trash/Current | Major | . | Prasad Chakka | Prasad Chakka | +| [HADOOP-4826](https://issues.apache.org/jira/browse/HADOOP-4826) | Admin command saveNamespace. | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-4789](https://issues.apache.org/jira/browse/HADOOP-4789) | Change fair scheduler to share between pools by default, not between invidual jobs | Minor | . | Matei Zaharia | Matei Zaharia | +| [HADOOP-4783](https://issues.apache.org/jira/browse/HADOOP-4783) | History files are given world readable permissions. | Blocker | . | Hemanth Yamijala | Amareshwari Sriramadasu | +| [HADOOP-4631](https://issues.apache.org/jira/browse/HADOOP-4631) | Split the default configurations into 3 parts | Major | conf | Owen O'Malley | Sharad Agarwal | +| [HADOOP-4618](https://issues.apache.org/jira/browse/HADOOP-4618) | Move http server from FSNamesystem into NameNode. | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-4576](https://issues.apache.org/jira/browse/HADOOP-4576) | Modify pending tasks count in the UI to pending jobs count in the UI | Major | . | Hemanth Yamijala | Sreekanth Ramakrishnan | +| [HADOOP-4572](https://issues.apache.org/jira/browse/HADOOP-4572) | INode and its sub-classes should be package private | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4567](https://issues.apache.org/jira/browse/HADOOP-4567) | GetFileBlockLocations should return the NetworkTopology information of the machines that hosts those blocks | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-4445](https://issues.apache.org/jira/browse/HADOOP-4445) | Wrong number of running map/reduce tasks are displayed in queue information. | Major | . | Karam Singh | Sreekanth Ramakrishnan | +| [HADOOP-4435](https://issues.apache.org/jira/browse/HADOOP-4435) | The JobTracker should display the amount of heap memory used | Minor | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-4422](https://issues.apache.org/jira/browse/HADOOP-4422) | S3 file systems should not create bucket | Major | fs/s3 | David Phillips | David Phillips | +| [HADOOP-4253](https://issues.apache.org/jira/browse/HADOOP-4253) | Fix warnings generated by FindBugs | Major | conf, fs, record | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-4210](https://issues.apache.org/jira/browse/HADOOP-4210) | Findbugs warnings are printed related to equals implementation of several classes | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-4188](https://issues.apache.org/jira/browse/HADOOP-4188) | Remove Task's dependency on concrete file systems | Major | . | Tom White | Sharad Agarwal | +| [HADOOP-4103](https://issues.apache.org/jira/browse/HADOOP-4103) | Alert for missing blocks | Major | . | Christian Kunz | Raghu Angadi | +| [HADOOP-4035](https://issues.apache.org/jira/browse/HADOOP-4035) | Modify the capacity scheduler (HADOOP-3445) to schedule tasks based on memory requirements and task trackers free memory | Blocker | . | Hemanth Yamijala | Vinod Kumar Vavilapalli | +| [HADOOP-4029](https://issues.apache.org/jira/browse/HADOOP-4029) | NameNode should report status and performance for each replica of image and log | Major | . | Robert Chansler | Boris Shkolnik | +| [HADOOP-3986](https://issues.apache.org/jira/browse/HADOOP-3986) | JobClient should not have a static configuration | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-3923](https://issues.apache.org/jira/browse/HADOOP-3923) | Deprecate org.apache.hadoop.mapred.StatusHttpServer | Minor | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3750](https://issues.apache.org/jira/browse/HADOOP-3750) | Fix and enforce module dependencies | Major | . | Tom White | Sharad Agarwal | +| [HADOOP-3497](https://issues.apache.org/jira/browse/HADOOP-3497) | File globbing with a PathFilter is too restrictive | Major | fs | Tom White | Tom White | +| [HADOOP-3422](https://issues.apache.org/jira/browse/HADOOP-3422) | Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen | Major | metrics | Jason | stack | +| [HADOOP-3344](https://issues.apache.org/jira/browse/HADOOP-3344) | libhdfs: always builds 32bit, even when x86\_64 Java used | Major | build | Craig Macdonald | Giridharan Kesavan | +| [HADOOP-1650](https://issues.apache.org/jira/browse/HADOOP-1650) | Upgrade Jetty to 6.x | Major | . | Devaraj Das | Chris Douglas | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-4937](https://issues.apache.org/jira/browse/HADOOP-4937) | [HOD] Include ringmaster RPC port information in the notes attribute | Major | contrib/hod | Hemanth Yamijala | Peeyush Bishnoi | +| [HADOOP-4804](https://issues.apache.org/jira/browse/HADOOP-4804) | Create Forrest documentation for the fair scheduler | Minor | . | Matei Zaharia | Sreekanth Ramakrishnan | +| [HADOOP-4661](https://issues.apache.org/jira/browse/HADOOP-4661) | distch: a tool for distributed ch{mod,own} | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4622](https://issues.apache.org/jira/browse/HADOOP-4622) | Explicitly specify interpretor for non-native binaries | Minor | . | Fredrik Hedberg | Fredrik Hedberg | +| [HADOOP-4575](https://issues.apache.org/jira/browse/HADOOP-4575) | An independent HTTPS proxy for HDFS | Major | security | Kan Zhang | Kan Zhang | +| [HADOOP-4348](https://issues.apache.org/jira/browse/HADOOP-4348) | Adding service-level authorization to Hadoop | Major | security | Kan Zhang | Arun C Murthy | +| [HADOOP-4284](https://issues.apache.org/jira/browse/HADOOP-4284) | Support for user configurable global filters on HttpServer | Major | security | Kan Zhang | Kan Zhang | +| [HADOOP-4179](https://issues.apache.org/jira/browse/HADOOP-4179) | Hadoop-Vaidya : Rule based performance diagnostic tool for Map/Reduce jobs | Major | . | Suhas Gogate | Suhas Gogate | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-6150](https://issues.apache.org/jira/browse/HADOOP-6150) | Need to be able to instantiate a comparator instance from a comparator string without creating a TFile.Reader object | Minor | io | Hong Tang | Hong Tang | +| [HADOOP-5521](https://issues.apache.org/jira/browse/HADOOP-5521) | Remove dependency of testcases on RESTART\_COUNT | Major | test | Amar Kamat | Ravi Gummadi | +| [HADOOP-5520](https://issues.apache.org/jira/browse/HADOOP-5520) | Typo in diskQuota help documentation | Trivial | documentation | Ravi Phulari | Ravi Phulari | +| [HADOOP-5519](https://issues.apache.org/jira/browse/HADOOP-5519) | Remove claims from mapred-default.xml that prime numbers of tasks are helpful. | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-5514](https://issues.apache.org/jira/browse/HADOOP-5514) | Add waiting/failed tasks to JobTracker metrics | Major | metrics | Chris Douglas | Chris Douglas | +| [HADOOP-5468](https://issues.apache.org/jira/browse/HADOOP-5468) | Change Hadoop doc menu to sub-menus | Major | documentation | Corinne Chandel | Corinne Chandel | +| [HADOOP-5466](https://issues.apache.org/jira/browse/HADOOP-5466) | CSS Style Changes for Hadoop Doc Headers and Code | Major | documentation | Corinne Chandel | Corinne Chandel | +| [HADOOP-5395](https://issues.apache.org/jira/browse/HADOOP-5395) | When queue ACLs are enabled, the error message shown when a job is submitted to a non-existent queue is misleading | Major | . | Hemanth Yamijala | rahul k singh | +| [HADOOP-5379](https://issues.apache.org/jira/browse/HADOOP-5379) | Throw exception instead of writing to System.err when there is a CRC error on CBZip2InputStream | Minor | io | Rodrigo Schmidt | Rodrigo Schmidt | +| [HADOOP-5275](https://issues.apache.org/jira/browse/HADOOP-5275) | ivy directory should be there in hadoop tar ball | Major | build | Suman Sehgal | Giridharan Kesavan | +| [HADOOP-5122](https://issues.apache.org/jira/browse/HADOOP-5122) | libhdfs test conf uses deprecated fs.default.name value | Trivial | test | Craig Macdonald | Craig Macdonald | +| [HADOOP-5030](https://issues.apache.org/jira/browse/HADOOP-5030) | Chukwa RPM build improvements | Major | . | Eric Yang | Eric Yang | +| [HADOOP-4980](https://issues.apache.org/jira/browse/HADOOP-4980) | Cleanup the Capacity Scheduler code | Major | . | Vivek Ratan | Vivek Ratan | +| [HADOOP-4950](https://issues.apache.org/jira/browse/HADOOP-4950) | CompressorStream and BlockCompressorStream should be public | Major | io | Owen O'Malley | Owen O'Malley | +| [HADOOP-4944](https://issues.apache.org/jira/browse/HADOOP-4944) | Allow Xinclude in hadoop config file | Minor | conf | Rama, Ramasamy | Rama, Ramasamy | +| [HADOOP-4920](https://issues.apache.org/jira/browse/HADOOP-4920) | do not keep forrest output in subversion | Major | documentation | Doug Cutting | Doug Cutting | +| [HADOOP-4916](https://issues.apache.org/jira/browse/HADOOP-4916) | adjust build.xml to reference external properties when building the "init.d" for the rpm | Major | . | Eric Yang | Eric Yang | +| [HADOOP-4909](https://issues.apache.org/jira/browse/HADOOP-4909) | Clean up map/reduce api to take JobContext in the static methods | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-4880](https://issues.apache.org/jira/browse/HADOOP-4880) | Improvements to TestJobTrackerRestart | Major | test | Arun C Murthy | Amar Kamat | +| [HADOOP-4843](https://issues.apache.org/jira/browse/HADOOP-4843) | Collect Job History log file and Job Conf file into Chukwa | Major | . | Eric Yang | Eric Yang | +| [HADOOP-4838](https://issues.apache.org/jira/browse/HADOOP-4838) | Cleanup Metrics and the MBeans | Major | . | Sanjay Radia | Sanjay Radia | +| [HADOOP-4837](https://issues.apache.org/jira/browse/HADOOP-4837) | Move HADOOP\_CONF\_DIR to chukwa-env.sh | Major | . | Jerome Boulon | Jerome Boulon | +| [HADOOP-4828](https://issues.apache.org/jira/browse/HADOOP-4828) | Update documentation for default configuration | Major | documentation | Sharad Agarwal | Sharad Agarwal | +| [HADOOP-4827](https://issues.apache.org/jira/browse/HADOOP-4827) | Improve data aggregation in database | Major | . | Eric Yang | Eric Yang | +| [HADOOP-4758](https://issues.apache.org/jira/browse/HADOOP-4758) | Add a splitter for metrics contexts | Minor | metrics | Chris Douglas | Chris Douglas | +| [HADOOP-4753](https://issues.apache.org/jira/browse/HADOOP-4753) | gridmix2 code can be condensed | Minor | benchmarks | Chris Douglas | Chris Douglas | +| [HADOOP-4749](https://issues.apache.org/jira/browse/HADOOP-4749) | reducer should output input data size when shuffling is done | Major | . | Zheng Shao | He Yongqiang | +| [HADOOP-4747](https://issues.apache.org/jira/browse/HADOOP-4747) | Reuse FileStatus in FsShell where possible | Minor | fs | David Phillips | David Phillips | +| [HADOOP-4708](https://issues.apache.org/jira/browse/HADOOP-4708) | Add support for dfsadmin commands for test TestCLI unit test | Major | test | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-4705](https://issues.apache.org/jira/browse/HADOOP-4705) | [HOD] Grant read permissions for files/directories created by hod as part of provisioning hadoop clusters | Major | contrib/hod | Hemanth Yamijala | Peeyush Bishnoi | +| [HADOOP-4699](https://issues.apache.org/jira/browse/HADOOP-4699) | Change TaskTracker.MapOutputServlet to send only the IFile segment, validate checksum in Reduce | Major | . | Chris Douglas | Chris Douglas | +| [HADOOP-4690](https://issues.apache.org/jira/browse/HADOOP-4690) | fuse-dfs - create source file/function + utils + config + main source files | Minor | . | Pete Wyckoff | Pete Wyckoff | +| [HADOOP-4668](https://issues.apache.org/jira/browse/HADOOP-4668) | Improve JavaDoc on JobConf.setCombinerClass to better document restrictions on combiners | Major | documentation | Owen O'Malley | Owen O'Malley | +| [HADOOP-4666](https://issues.apache.org/jira/browse/HADOOP-4666) | Launch reduces only after a few maps have run in the Fair Scheduler | Major | . | Matei Zaharia | Matei Zaharia | +| [HADOOP-4664](https://issues.apache.org/jira/browse/HADOOP-4664) | Parallelize job initialization | Blocker | . | Matei Zaharia | Jothi Padmanabhan | +| [HADOOP-4649](https://issues.apache.org/jira/browse/HADOOP-4649) | Improve abstraction for spill indices | Minor | . | Chris Douglas | Chris Douglas | +| [HADOOP-4640](https://issues.apache.org/jira/browse/HADOOP-4640) | Add ability to split text files compressed with lzo | Trivial | io | Johan Oskarsson | Johan Oskarsson | +| [HADOOP-4628](https://issues.apache.org/jira/browse/HADOOP-4628) | Move Hive out of Core | Major | . | Owen O'Malley | | +| [HADOOP-4599](https://issues.apache.org/jira/browse/HADOOP-4599) | BlocksMap and BlockInfo should be package private. | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-4565](https://issues.apache.org/jira/browse/HADOOP-4565) | MultiFileInputSplit can use data locality information to create splits | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-4545](https://issues.apache.org/jira/browse/HADOOP-4545) | Add an example of a secondary sort | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-4523](https://issues.apache.org/jira/browse/HADOOP-4523) | Enhance how memory-intensive user tasks are handled | Major | . | Vivek Ratan | Vinod Kumar Vavilapalli | +| [HADOOP-4485](https://issues.apache.org/jira/browse/HADOOP-4485) | ant compile-native shorthand | Trivial | build | Enis Soztutar | Enis Soztutar | +| [HADOOP-4454](https://issues.apache.org/jira/browse/HADOOP-4454) | Support comments in 'slaves' file | Minor | conf, scripts | Rama, Ramasamy | Rama, Ramasamy | +| [HADOOP-4453](https://issues.apache.org/jira/browse/HADOOP-4453) | Improve ssl handling for distcp | Major | . | Chris Douglas | Kan Zhang | +| [HADOOP-4440](https://issues.apache.org/jira/browse/HADOOP-4440) | TestJobInProgressListener should also test for jobs killed in queued state | Minor | test | Amar Kamat | Amar Kamat | +| [HADOOP-4437](https://issues.apache.org/jira/browse/HADOOP-4437) | Use qMC sequence to improve the accuracy of PiEstimator | Minor | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4339](https://issues.apache.org/jira/browse/HADOOP-4339) | Improve FsShell -du/-dus and FileSystem.getContentSummary efficiency | Major | fs | David Phillips | David Phillips | +| [HADOOP-4305](https://issues.apache.org/jira/browse/HADOOP-4305) | repeatedly blacklisted tasktrackers should get declared dead | Major | . | Christian Kunz | Amareshwari Sriramadasu | +| [HADOOP-4276](https://issues.apache.org/jira/browse/HADOOP-4276) | The mapred.\*ID classes are inefficient for hashCode and serialization | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-4244](https://issues.apache.org/jira/browse/HADOOP-4244) | In "ant test-patch", runContribTestOnEclipseFiles should not be run on Hudson only | Major | build, test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4234](https://issues.apache.org/jira/browse/HADOOP-4234) | KFS: Allow KFS layer to interface with multiple KFS namenodes | Minor | fs | Sriram Rao | Sriram Rao | +| [HADOOP-4226](https://issues.apache.org/jira/browse/HADOOP-4226) | LineReader::readLine cleanup | Minor | . | Yuri Pradkin | Yuri Pradkin | +| [HADOOP-4178](https://issues.apache.org/jira/browse/HADOOP-4178) | The capacity scheduler's defaults for queues should be configurable. | Blocker | . | Owen O'Malley | Sreekanth Ramakrishnan | +| [HADOOP-4126](https://issues.apache.org/jira/browse/HADOOP-4126) | Allow access to HDFS web UI on EC2 | Major | contrib/cloud | Tom White | Tom White | +| [HADOOP-3770](https://issues.apache.org/jira/browse/HADOOP-3770) | improve composition, submission and result collection of gridmix | Major | benchmarks | Lingyun Yang | Runping Qi | +| [HADOOP-3461](https://issues.apache.org/jira/browse/HADOOP-3461) | Remove dfs.StringBytesWritbale | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3180](https://issues.apache.org/jira/browse/HADOOP-3180) | add name of faulty class to WritableName.getClass IOException upon ClassNotFoundException | Trivial | io | Pete Wyckoff | Pete Wyckoff | +| [HADOOP-3136](https://issues.apache.org/jira/browse/HADOOP-3136) | Assign multiple tasks per TaskTracker heartbeat | Major | . | Devaraj Das | Arun C Murthy | +| [HADOOP-3063](https://issues.apache.org/jira/browse/HADOOP-3063) | BloomMapFile - fail-fast version of MapFile for sparsely populated key space | Major | io | Andrzej Bialecki | Andrzej Bialecki | +| [HADOOP-2774](https://issues.apache.org/jira/browse/HADOOP-2774) | Add counters to show number of key/values that have been sorted and merged in the maps and reduces | Major | . | Owen O'Malley | Ravi Gummadi | +| [HADOOP-1230](https://issues.apache.org/jira/browse/HADOOP-1230) | Replace parameters with context objects in Mapper, Reducer, Partitioner, InputFormat, and OutputFormat classes | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-466](https://issues.apache.org/jira/browse/HADOOP-466) | Startup scripts will not start instances of Hadoop daemons w/different configs w/o setting separate PID directories | Major | conf | Vetle Roeim | | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-5645](https://issues.apache.org/jira/browse/HADOOP-5645) | After HADOOP-4920 we need a place to checkin releasenotes.html | Major | build | Nigel Daley | Nigel Daley | +| [HADOOP-5644](https://issues.apache.org/jira/browse/HADOOP-5644) | Namnode is stuck in safe mode | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-5607](https://issues.apache.org/jira/browse/HADOOP-5607) | TestCapacityScheduler fails with NPE | Major | test | Chris Douglas | Chris Douglas | +| [HADOOP-5605](https://issues.apache.org/jira/browse/HADOOP-5605) | All the replicas incorrectly got marked as corrupt. | Blocker | . | Raghu Angadi | Hairong Kuang | +| [HADOOP-5588](https://issues.apache.org/jira/browse/HADOOP-5588) | hadoop commands seem extremely slow in 0.20 branch | Blocker | fs | Koji Noguchi | Hairong Kuang | +| [HADOOP-5585](https://issues.apache.org/jira/browse/HADOOP-5585) | FileSystem statistic counters are too high when JVM reuse is enabled. | Blocker | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-5577](https://issues.apache.org/jira/browse/HADOOP-5577) | The Job.waitForCompletion doesn't print status as the job runs | Blocker | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-5576](https://issues.apache.org/jira/browse/HADOOP-5576) | LocalJobRunner does not run jobs using new MapReduce API | Blocker | . | Tom White | Tom White | +| [HADOOP-5571](https://issues.apache.org/jira/browse/HADOOP-5571) | TupleWritable can return incorrect results if it contains more than 32 values | Major | . | Jingkei Ly | Jingkei Ly | +| [HADOOP-5565](https://issues.apache.org/jira/browse/HADOOP-5565) | The job instrumentation API needs to have a method for finalizeJob, | Major | . | Jerome Boulon | Jerome Boulon | +| [HADOOP-5564](https://issues.apache.org/jira/browse/HADOOP-5564) | hadoop command uses large JVM heap size | Blocker | build | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-5557](https://issues.apache.org/jira/browse/HADOOP-5557) | Two minor problems in TestOverReplicatedBlocks | Minor | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-5554](https://issues.apache.org/jira/browse/HADOOP-5554) | DataNodeCluster should create blocks with the same generation stamp as the blocks created in CreateEditsLog | Major | test | Hairong Kuang | Hairong Kuang | +| [HADOOP-5551](https://issues.apache.org/jira/browse/HADOOP-5551) | Namenode permits directory destruction on overwrite | Critical | . | Brian Bockelman | Brian Bockelman | +| [HADOOP-5549](https://issues.apache.org/jira/browse/HADOOP-5549) | ReplicationMonitor should schedule both replication and deletion work in one iteration | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-5548](https://issues.apache.org/jira/browse/HADOOP-5548) | Observed negative running maps on the job tracker | Blocker | . | Owen O'Malley | Amareshwari Sriramadasu | +| [HADOOP-5534](https://issues.apache.org/jira/browse/HADOOP-5534) | Deadlock triggered by FairScheduler scheduler's servlet due to changes from HADOOP-5214. | Blocker | . | Vinod Kumar Vavilapalli | rahul k singh | +| [HADOOP-5522](https://issues.apache.org/jira/browse/HADOOP-5522) | Document job setup/cleaup tasks and task cleanup tasks in mapred tutorial | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-5516](https://issues.apache.org/jira/browse/HADOOP-5516) | TaskMemoryManagerThread crashes in a corner case | Blocker | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [HADOOP-5505](https://issues.apache.org/jira/browse/HADOOP-5505) | TestMissingBlocksAlert fails on 0.20. | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-5499](https://issues.apache.org/jira/browse/HADOOP-5499) | JobTracker metrics do not match job counters | Major | metrics | Chris Douglas | | +| [HADOOP-5493](https://issues.apache.org/jira/browse/HADOOP-5493) | Shuffle copiers do not return Codecs back to the pool at the end of shuffling | Blocker | . | Devaraj Das | Jothi Padmanabhan | +| [HADOOP-5490](https://issues.apache.org/jira/browse/HADOOP-5490) | TestParallelInitialization failed on NoSuchElementException | Blocker | . | Hairong Kuang | Jothi Padmanabhan | +| [HADOOP-5484](https://issues.apache.org/jira/browse/HADOOP-5484) | TestRecoveryManager fails wtih FileAlreadyExistsException | Major | test | Hairong Kuang | Amar Kamat | +| [HADOOP-5483](https://issues.apache.org/jira/browse/HADOOP-5483) | Directory/file cleanup thread throws IllegalStateException | Major | . | Hairong Kuang | | +| [HADOOP-5479](https://issues.apache.org/jira/browse/HADOOP-5479) | NameNode should not send empty block replication request to DataNode | Critical | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-5473](https://issues.apache.org/jira/browse/HADOOP-5473) | Race condition in command-line kill for a task | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-5471](https://issues.apache.org/jira/browse/HADOOP-5471) | SyncLogs thread in Child.java would update wrong file for a cleanup attempt, in some cases. | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-5465](https://issues.apache.org/jira/browse/HADOOP-5465) | Blocks remain under-replicated | Blocker | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-5463](https://issues.apache.org/jira/browse/HADOOP-5463) | Balancer throws "Not a host:port pair" unless port is specified in fs.default.name | Minor | . | Stuart White | Stuart White | +| [HADOOP-5459](https://issues.apache.org/jira/browse/HADOOP-5459) | CRC errors not detected reading intermediate output into memory with problematic length | Major | . | Chris Douglas | Chris Douglas | +| [HADOOP-5440](https://issues.apache.org/jira/browse/HADOOP-5440) | Successful taskid are not removed from TaskMemoryManager | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-5437](https://issues.apache.org/jira/browse/HADOOP-5437) | Unit test for jvm -reuse | Blocker | . | Amareshwari Sriramadasu | Owen O'Malley | +| [HADOOP-5432](https://issues.apache.org/jira/browse/HADOOP-5432) | TestHdfsProxy fails on 0.20 | Major | . | Chris Douglas | Chris Douglas | +| [HADOOP-5417](https://issues.apache.org/jira/browse/HADOOP-5417) | IPC client drops interrupted exceptions | Blocker | ipc | Owen O'Malley | Owen O'Malley | +| [HADOOP-5416](https://issues.apache.org/jira/browse/HADOOP-5416) | Wrong description of " hadoop fs -test " in FS Shell guide . | Trivial | documentation | Ravi Phulari | Ravi Phulari | +| [HADOOP-5414](https://issues.apache.org/jira/browse/HADOOP-5414) | IO exception while executing hadoop fs -touchz fileName | Major | fs | Ravi Phulari | Hairong Kuang | +| [HADOOP-5394](https://issues.apache.org/jira/browse/HADOOP-5394) | JobTracker might schedule 2 attempts of the same task with the same attempt id across restarts | Critical | . | Amar Kamat | Amar Kamat | +| [HADOOP-5390](https://issues.apache.org/jira/browse/HADOOP-5390) | hdfsproxy includes duplicate jars in tarball, source in binary tarball | Major | build | Chris Douglas | zhiyong zhang | +| [HADOOP-5384](https://issues.apache.org/jira/browse/HADOOP-5384) | DataNodeCluster should not create blocks with generationStamp == 1 | Blocker | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-5382](https://issues.apache.org/jira/browse/HADOOP-5382) | The new map/reduce api doesn't support combiners | Blocker | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-5376](https://issues.apache.org/jira/browse/HADOOP-5376) | JobInProgress.obtainTaskCleanupTask() throws an ArrayIndexOutOfBoundsException | Blocker | . | Vinod Kumar Vavilapalli | Amareshwari Sriramadasu | +| [HADOOP-5338](https://issues.apache.org/jira/browse/HADOOP-5338) | Reduce tasks are stuck waiting for map outputs when none are in progress | Blocker | . | Ramya Sunil | Amar Kamat | +| [HADOOP-5337](https://issues.apache.org/jira/browse/HADOOP-5337) | JobTracker greedily schedules tasks without running tasks to join | Major | . | Karam Singh | Amar Kamat | +| [HADOOP-5328](https://issues.apache.org/jira/browse/HADOOP-5328) | Renaming of Job histroy file is incorrect if Jobtracker is restarted multimple times | Major | . | Karam Singh | Amar Kamat | +| [HADOOP-5327](https://issues.apache.org/jira/browse/HADOOP-5327) | Job files for a job failing because of ACLs are not clean from the system directory | Blocker | . | Karam Singh | Amar Kamat | +| [HADOOP-5326](https://issues.apache.org/jira/browse/HADOOP-5326) | bzip2 codec (CBZip2OutputStream) creates corrupted output file for some inputs | Major | io | Rodrigo Schmidt | Rodrigo Schmidt | +| [HADOOP-5320](https://issues.apache.org/jira/browse/HADOOP-5320) | TestMapReduceLocal is missing a close() that is causing it to fail while running the test on NFS | Minor | test | Jothi Padmanabhan | Jothi Padmanabhan | +| [HADOOP-5306](https://issues.apache.org/jira/browse/HADOOP-5306) | Job History file can have empty string as http port after JobTracker Restart in case of lost TT, which can result in NumberFormatException when JT is restarted 2nd time | Blocker | . | Ravi Gummadi | Amar Kamat | +| [HADOOP-5305](https://issues.apache.org/jira/browse/HADOOP-5305) | Unit test org.apache.hadoop.fs.TestCopyFiles.testMapCount fails on trunk | Minor | test | Ramya Sunil | Tsz Wo Nicholas Sze | +| [HADOOP-5298](https://issues.apache.org/jira/browse/HADOOP-5298) | Unit test fails out on trunk org.apache.hadoop.http.TestServletFilter.testServletFilter | Major | test | Lee Tucker | Tsz Wo Nicholas Sze | +| [HADOOP-5292](https://issues.apache.org/jira/browse/HADOOP-5292) | KFS: calling getFileBlockLocations() on 0-length file causes a NPE | Major | . | Sriram Rao | Sriram Rao | +| [HADOOP-5285](https://issues.apache.org/jira/browse/HADOOP-5285) | JobTracker hangs for long periods of time | Blocker | . | Vinod Kumar Vavilapalli | Devaraj Das | +| [HADOOP-5282](https://issues.apache.org/jira/browse/HADOOP-5282) | Running tasks on lost trackers are incorrectly logging the attempt and task failures | Blocker | . | Amar Kamat | Amar Kamat | +| [HADOOP-5281](https://issues.apache.org/jira/browse/HADOOP-5281) | GzipCodec fails second time it is used in a process | Blocker | . | Ben Maurer | Chris Douglas | +| [HADOOP-5280](https://issues.apache.org/jira/browse/HADOOP-5280) | When expiring a lost launched task, JT doesn't remove the attempt from the taskidToTIPMap. | Blocker | . | Vinod Kumar Vavilapalli | Devaraj Das | +| [HADOOP-5274](https://issues.apache.org/jira/browse/HADOOP-5274) | gridmix2 is not getting compiled to generate gridmix.jar | Critical | benchmarks | Suman Sehgal | Chris Douglas | +| [HADOOP-5272](https://issues.apache.org/jira/browse/HADOOP-5272) | JobTracker does not log TIP start information after restart | Critical | . | Amar Kamat | Amar Kamat | +| [HADOOP-5269](https://issues.apache.org/jira/browse/HADOOP-5269) | TaskTracker.runningTasks holding FAILED\_UNCLEAN and KILLED\_UNCLEAN taskStatuses forever in some cases. | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-5259](https://issues.apache.org/jira/browse/HADOOP-5259) | Job with output hdfs:/user/\<username\>/outputpath (no authority) fails with Wrong FS | Major | fs | Koji Noguchi | Doug Cutting | +| [HADOOP-5255](https://issues.apache.org/jira/browse/HADOOP-5255) | Fix for HADOOP-5079 HashFunction inadvertently destroys some randomness | Minor | io | stack | Jonathan Ellis | +| [HADOOP-5248](https://issues.apache.org/jira/browse/HADOOP-5248) | Job directories could remain undeleted in some scenarios after job completes. | Blocker | . | Hemanth Yamijala | Devaraj Das | +| [HADOOP-5247](https://issues.apache.org/jira/browse/HADOOP-5247) | NPEs in JobTracker and JobClient when mapred.jobtracker.completeuserjobs.maximum is set to zero. | Blocker | . | Vinod Kumar Vavilapalli | Amar Kamat | +| [HADOOP-5241](https://issues.apache.org/jira/browse/HADOOP-5241) | Reduce tasks get stuck because of over-estimated task size (regression from 0.18) | Blocker | . | Andy Pavlo | Sharad Agarwal | +| [HADOOP-5233](https://issues.apache.org/jira/browse/HADOOP-5233) | Reducer not Succeded after 100% | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-5227](https://issues.apache.org/jira/browse/HADOOP-5227) | distcp -delete option deletes all files from the destination directory | Major | . | Suhas Gogate | Tsz Wo Nicholas Sze | +| [HADOOP-5214](https://issues.apache.org/jira/browse/HADOOP-5214) | ConcurrentModificationException in FairScheduler.getTotalSlots | Blocker | . | Vinod Kumar Vavilapalli | rahul k singh | +| [HADOOP-5211](https://issues.apache.org/jira/browse/HADOOP-5211) | TestSetupAndCleanupFailure fails with timeout | Major | . | Enis Soztutar | Enis Soztutar | +| [HADOOP-5154](https://issues.apache.org/jira/browse/HADOOP-5154) | 4-way deadlock in FairShare scheduler | Blocker | . | Vinod Kumar Vavilapalli | Matei Zaharia | +| [HADOOP-5146](https://issues.apache.org/jira/browse/HADOOP-5146) | LocalDirAllocator misses files on the local filesystem | Blocker | . | Arun C Murthy | Devaraj Das | +| [HADOOP-5145](https://issues.apache.org/jira/browse/HADOOP-5145) | Balancer sometimes runs out of memory after days or weeks running | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-5142](https://issues.apache.org/jira/browse/HADOOP-5142) | MapWritable#putAll does not store classes | Minor | io | DoÄacan Güney | | +| [HADOOP-5139](https://issues.apache.org/jira/browse/HADOOP-5139) | RPC call throws IllegalArgumentException complaining duplicate metrics registration | Major | ipc | Hairong Kuang | Hairong Kuang | +| [HADOOP-5111](https://issues.apache.org/jira/browse/HADOOP-5111) | Generic mapreduce classes cannot be used with Job::set\* methods | Major | . | Chris Douglas | Chris Douglas | +| [HADOOP-5085](https://issues.apache.org/jira/browse/HADOOP-5085) | Copying a file to local with Crc throws an exception | Major | fs | Hairong Kuang | Hairong Kuang | +| [HADOOP-5079](https://issues.apache.org/jira/browse/HADOOP-5079) | HashFunction inadvertently destroys some randomness | Major | util | Jonathan Ellis | Jonathan Ellis | +| [HADOOP-5075](https://issues.apache.org/jira/browse/HADOOP-5075) | Potential infinite loop in updateMinSlots | Blocker | . | Matei Zaharia | Matei Zaharia | +| [HADOOP-5068](https://issues.apache.org/jira/browse/HADOOP-5068) | testClusterBlockingForLackOfMemory in TestCapacityScheduler fails randomly | Major | . | Sreekanth Ramakrishnan | Vinod Kumar Vavilapalli | +| [HADOOP-5066](https://issues.apache.org/jira/browse/HADOOP-5066) | ant binary should not compile docs | Major | build | Amareshwari Sriramadasu | Giridharan Kesavan | +| [HADOOP-5065](https://issues.apache.org/jira/browse/HADOOP-5065) | setOutputFormatClass in mapreduce.Job fails for SequenceFileOutputFormat | Major | . | Chris Douglas | Chris Douglas | +| [HADOOP-5048](https://issues.apache.org/jira/browse/HADOOP-5048) | Sometimes job is still displayed in jobqueue\_details page for long time after job was killed. | Major | . | Karam Singh | Sreekanth Ramakrishnan | +| [HADOOP-5026](https://issues.apache.org/jira/browse/HADOOP-5026) | Startup scripts should be svn-executable | Minor | . | Andy Konwinski | Andy Konwinski | +| [HADOOP-4999](https://issues.apache.org/jira/browse/HADOOP-4999) | IndexOutOfBoundsException in FSEditLog | Major | . | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-4993](https://issues.apache.org/jira/browse/HADOOP-4993) | chukwa agent startup should be more modular | Major | . | Ari Rabkin | Ari Rabkin | +| [HADOOP-4988](https://issues.apache.org/jira/browse/HADOOP-4988) | An earlier fix, for HADOOP-4373, results in a problem with reclaiming capacity when one or more queues have a capacity equal to zero | Blocker | . | Vivek Ratan | Vivek Ratan | +| [HADOOP-4979](https://issues.apache.org/jira/browse/HADOOP-4979) | Capacity Scheduler does not always return no task to a TT if a job's memry requirements are not met | Major | . | Vivek Ratan | Vivek Ratan | +| [HADOOP-4977](https://issues.apache.org/jira/browse/HADOOP-4977) | Deadlock between reclaimCapacity and assignTasks | Blocker | . | Matei Zaharia | Vivek Ratan | +| [HADOOP-4963](https://issues.apache.org/jira/browse/HADOOP-4963) | Logs saying org.apache.hadoop.util.DiskChecker$DiskErrorException in TaskTracker are not relevant | Minor | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-4956](https://issues.apache.org/jira/browse/HADOOP-4956) | NPE when jobdetails.jsp | Blocker | . | Devaraj Das | Devaraj Das | +| [HADOOP-4949](https://issues.apache.org/jira/browse/HADOOP-4949) | Native compilation is broken | Blocker | . | Chris Douglas | Chris Douglas | +| [HADOOP-4947](https://issues.apache.org/jira/browse/HADOOP-4947) | use regex to parse chukwa commands | Minor | . | Ari Rabkin | Ari Rabkin | +| [HADOOP-4925](https://issues.apache.org/jira/browse/HADOOP-4925) | Chukwa HTTP connection policies are too hard to configure | Major | . | Ari Rabkin | Ari Rabkin | +| [HADOOP-4914](https://issues.apache.org/jira/browse/HADOOP-4914) | Support chkconfig for chukwa start up scripts | Trivial | . | Eric Yang | Eric Yang | +| [HADOOP-4896](https://issues.apache.org/jira/browse/HADOOP-4896) | hdfs fsck does not load hdfs configuration. | Blocker | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-4894](https://issues.apache.org/jira/browse/HADOOP-4894) | fix Chukwa's jettyCollector.sh | Minor | . | Ari Rabkin | Ari Rabkin | +| [HADOOP-4892](https://issues.apache.org/jira/browse/HADOOP-4892) | File descriptor leak in Chukwa's ExecPlugin | Major | . | Ari Rabkin | Ari Rabkin | +| [HADOOP-4889](https://issues.apache.org/jira/browse/HADOOP-4889) | Chukwa RPM needs to chown user/group in build stage instead of post install stage. | Trivial | . | Eric Yang | Eric Yang | +| [HADOOP-4884](https://issues.apache.org/jira/browse/HADOOP-4884) | Change Date format pattern for Time Series graph | Trivial | . | Eric Yang | Eric Yang | +| [HADOOP-4879](https://issues.apache.org/jira/browse/HADOOP-4879) | TestJobTrackerRestart fails on trunk | Blocker | test | Arun C Murthy | Owen O'Malley | +| [HADOOP-4878](https://issues.apache.org/jira/browse/HADOOP-4878) | After introduction of ivy ant test-patch always returns -1 score | Blocker | build | Sreekanth Ramakrishnan | Giridharan Kesavan | +| [HADOOP-4876](https://issues.apache.org/jira/browse/HADOOP-4876) | Capacity reclaimation for queue would not work as intended | Blocker | . | Sreekanth Ramakrishnan | Sreekanth Ramakrishnan | +| [HADOOP-4874](https://issues.apache.org/jira/browse/HADOOP-4874) | Remove bindings to lzo | Blocker | io | Owen O'Malley | Owen O'Malley | +| [HADOOP-4873](https://issues.apache.org/jira/browse/HADOOP-4873) | display minMaps/Reduces on advanced scheduler page | Major | . | Joydeep Sen Sarma | Joydeep Sen Sarma | +| [HADOOP-4869](https://issues.apache.org/jira/browse/HADOOP-4869) | Lost Trackers may not be able to join back | Blocker | . | Devaraj Das | Amareshwari Sriramadasu | +| [HADOOP-4860](https://issues.apache.org/jira/browse/HADOOP-4860) | File Tailing Adaptor test cases causes random failure | Major | . | Eric Yang | Eric Yang | +| [HADOOP-4855](https://issues.apache.org/jira/browse/HADOOP-4855) | Fix help message in MRAdmin | Minor | fs, security | Arun C Murthy | Arun C Murthy | +| [HADOOP-4854](https://issues.apache.org/jira/browse/HADOOP-4854) | Capacity Scheduler should read the reclaim-capacity-interval config value from its own config file | Minor | . | Vivek Ratan | Sreekanth Ramakrishnan | +| [HADOOP-4849](https://issues.apache.org/jira/browse/HADOOP-4849) | Document service level authorization - HADOOP-4348 | Major | documentation, scripts | Arun C Murthy | Arun C Murthy | +| [HADOOP-4845](https://issues.apache.org/jira/browse/HADOOP-4845) | Shuffle counter issues | Major | . | Chris Douglas | He Yongqiang | +| [HADOOP-4844](https://issues.apache.org/jira/browse/HADOOP-4844) | Document deprecation of o.a.h.fs.permission.AccessControlException better | Minor | documentation, fs | Arun C Murthy | Arun C Murthy | +| [HADOOP-4830](https://issues.apache.org/jira/browse/HADOOP-4830) | Have end to end tests based on MiniMRCluster to verify that queue capacities are honoured. | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [HADOOP-4825](https://issues.apache.org/jira/browse/HADOOP-4825) | Chukwa - change jps to ps for process status detection | Major | . | Eric Yang | Eric Yang | +| [HADOOP-4820](https://issues.apache.org/jira/browse/HADOOP-4820) | Remove unused method FSNamesystem.deleteInSafeMode | Minor | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-4818](https://issues.apache.org/jira/browse/HADOOP-4818) | Enable JobTracker Instrumentation API with user job configuration | Major | . | Eric Yang | Eric Yang | +| [HADOOP-4811](https://issues.apache.org/jira/browse/HADOOP-4811) | Import style sheet images for Chukwa | Trivial | . | Eric Yang | Eric Yang | +| [HADOOP-4805](https://issues.apache.org/jira/browse/HADOOP-4805) | Remove black list feature from Chukwa Agent to Chukwa Collector communication | Major | . | Eric Yang | Eric Yang | +| [HADOOP-4796](https://issues.apache.org/jira/browse/HADOOP-4796) | Test target for chukwa build.xml needs to comply to hadoop build.xml test suites | Major | build | Eric Yang | Eric Yang | +| [HADOOP-4792](https://issues.apache.org/jira/browse/HADOOP-4792) | Chukwa build process generates files not tracked by svn | Major | build | Chris Douglas | Chris Douglas | +| [HADOOP-4791](https://issues.apache.org/jira/browse/HADOOP-4791) | Improve rpm build process for Chukwa | Major | . | Eric Yang | Eric Yang | +| [HADOOP-4787](https://issues.apache.org/jira/browse/HADOOP-4787) | TestTrackerBlacklistAcrossJobs fails randomly | Blocker | test | Hemanth Yamijala | Amareshwari Sriramadasu | +| [HADOOP-4786](https://issues.apache.org/jira/browse/HADOOP-4786) | TestTrackerBlacklistAcrossJobs compilation is broken | Blocker | . | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-4785](https://issues.apache.org/jira/browse/HADOOP-4785) | avoid two calls to System.currentTimeMillis() in heartbeat | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-4782](https://issues.apache.org/jira/browse/HADOOP-4782) | [HOD] HOD opens up directory permissions more than required | Blocker | contrib/hod | Hemanth Yamijala | Peeyush Bishnoi | +| [HADOOP-4774](https://issues.apache.org/jira/browse/HADOOP-4774) | CapacityScheduler should work even if capacity-scheduler.xml is not in the classpath i.e with the code defaults | Major | . | Amar Kamat | Sreekanth Ramakrishnan | +| [HADOOP-4771](https://issues.apache.org/jira/browse/HADOOP-4771) | FSImage saveFSImage() will have problem. | Major | . | Ruyue Ma | Ruyue Ma | +| [HADOOP-4770](https://issues.apache.org/jira/browse/HADOOP-4770) | gridmix2 run script doesn't work on trunk | Trivial | benchmarks | Chris Douglas | Chris Douglas | +| [HADOOP-4744](https://issues.apache.org/jira/browse/HADOOP-4744) | Wrong resolution of hostname and port | Blocker | . | Amareshwari Sriramadasu | Jothi Padmanabhan | +| [HADOOP-4738](https://issues.apache.org/jira/browse/HADOOP-4738) | saveVersion.sh could write a package-info.java that cannot be compiled when used with git | Trivial | build | Hemanth Yamijala | Chris Douglas | +| [HADOOP-4737](https://issues.apache.org/jira/browse/HADOOP-4737) | Job end notification should give status as "KILLED" when job gets killed. | Major | . | Ruchir Shah | Amareshwari Sriramadasu | +| [HADOOP-4732](https://issues.apache.org/jira/browse/HADOOP-4732) | connectionTimeout and readTimeout are passed in wrong order in ReduceTask.java | Minor | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-4716](https://issues.apache.org/jira/browse/HADOOP-4716) | testRestartWithLostTracker frequently times out | Blocker | . | Johan Oskarsson | Amar Kamat | +| [HADOOP-4706](https://issues.apache.org/jira/browse/HADOOP-4706) | IFileOutputStream.close should call close of the underlying stream | Minor | . | Jothi Padmanabhan | Jothi Padmanabhan | +| [HADOOP-4704](https://issues.apache.org/jira/browse/HADOOP-4704) | javadoc: "the the" =\> "the" | Trivial | documentation | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4698](https://issues.apache.org/jira/browse/HADOOP-4698) | TestMapRed fails with 64bit JDK | Major | test | Nigel Daley | Nigel Daley | +| [HADOOP-4695](https://issues.apache.org/jira/browse/HADOOP-4695) | TestGlobalFilter.testServletFilter fails | Blocker | test | Tsz Wo Nicholas Sze | Kan Zhang | +| [HADOOP-4692](https://issues.apache.org/jira/browse/HADOOP-4692) | Namenode in infinite loop for replicating/deleting corrupted block | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-4691](https://issues.apache.org/jira/browse/HADOOP-4691) | In the javadoc of IndexedSortable.compare(...), the link is wrong. | Minor | documentation, util | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4683](https://issues.apache.org/jira/browse/HADOOP-4683) | Move the call to getMapCompletionEvents in ReduceTask.ReduceCopier.fetchOutputs to a separate thread | Major | . | Devaraj Das | Jothi Padmanabhan | +| [HADOOP-4677](https://issues.apache.org/jira/browse/HADOOP-4677) | FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect. | Major | fs | Hong Tang | Hong Tang | +| [HADOOP-4676](https://issues.apache.org/jira/browse/HADOOP-4676) | Go back to Job has a wrong URL on blacklisted trackers page | Minor | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-4671](https://issues.apache.org/jira/browse/HADOOP-4671) | Make some "flag" booleans volatile because they are set/read in different threads | Minor | . | Owen O'Malley | Chris Douglas | +| [HADOOP-4669](https://issues.apache.org/jira/browse/HADOOP-4669) | A file-open sometimes may not set the access time of a file | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-4650](https://issues.apache.org/jira/browse/HADOOP-4650) | local.cache.size is set to 10 GB, while DEFAULT\_CACHE\_SIZE is set to 1 MB | Trivial | filecache | Jeff Hammerbacher | Jeff Hammerbacher | +| [HADOOP-4645](https://issues.apache.org/jira/browse/HADOOP-4645) | hdfs proxy doesn't package itself right. | Major | . | Owen O'Malley | Kan Zhang | +| [HADOOP-4638](https://issues.apache.org/jira/browse/HADOOP-4638) | Exception thrown in/from RecoveryManager.recover() should be caught and handled | Blocker | . | Amar Kamat | Amar Kamat | +| [HADOOP-4623](https://issues.apache.org/jira/browse/HADOOP-4623) | Running tasks are not maintained by JobInProgress if speculation is off | Major | . | Amar Kamat | Amar Kamat | +| [HADOOP-4621](https://issues.apache.org/jira/browse/HADOOP-4621) | javadoc: warning - Multiple sources of package comments found for some packages | Major | documentation | Tsz Wo Nicholas Sze | Kan Zhang | +| [HADOOP-4608](https://issues.apache.org/jira/browse/HADOOP-4608) | Examples -Driver does not check first argument. | Major | . | Edward J. Yoon | Edward J. Yoon | +| [HADOOP-4606](https://issues.apache.org/jira/browse/HADOOP-4606) | There is a cygpath error if log directory does not exist | Minor | scripts | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4598](https://issues.apache.org/jira/browse/HADOOP-4598) | '-setrep' command skips under-replicated blocks | Minor | . | Koji Noguchi | Hairong Kuang | +| [HADOOP-4589](https://issues.apache.org/jira/browse/HADOOP-4589) | Fix the PiEstimator output messages and code comments | Minor | documentation | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4587](https://issues.apache.org/jira/browse/HADOOP-4587) | typo in javadoc for map.input.file | Trivial | documentation | Koji Noguchi | Koji Noguchi | +| [HADOOP-4583](https://issues.apache.org/jira/browse/HADOOP-4583) | Code optimization/cleanup in HDFS | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-4579](https://issues.apache.org/jira/browse/HADOOP-4579) | StorageDirectory is not removed from the storage directories list when writing to an edits fails | Major | . | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-4571](https://issues.apache.org/jira/browse/HADOOP-4571) | chukwa conf files are not cleaned up | Minor | . | Tsz Wo Nicholas Sze | Eric Yang | +| [HADOOP-4558](https://issues.apache.org/jira/browse/HADOOP-4558) | Scheduler fails to reclaim capacity if Jobs are submitted to queue one after the other | Major | . | Karam Singh | Amar Kamat | +| [HADOOP-4530](https://issues.apache.org/jira/browse/HADOOP-4530) | In fsck, HttpServletResponse sendError fails with IllegalStateException | Major | . | Koji Noguchi | Hairong Kuang | +| [HADOOP-4513](https://issues.apache.org/jira/browse/HADOOP-4513) | Capacity scheduler should initialize tasks asynchronously | Major | . | Hemanth Yamijala | Sreekanth Ramakrishnan | +| [HADOOP-4429](https://issues.apache.org/jira/browse/HADOOP-4429) | Misconfigured UNIX Groups Break Hadoop | Minor | . | Alex Loddengaard | Alex Loddengaard | +| [HADOOP-4408](https://issues.apache.org/jira/browse/HADOOP-4408) | FsAction operations shouldn't create new Objects | Minor | fs | Chris Douglas | Chris Douglas | +| [HADOOP-4388](https://issues.apache.org/jira/browse/HADOOP-4388) | Bug in Datanode transferBlocks code | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-4377](https://issues.apache.org/jira/browse/HADOOP-4377) | Race condition creating S3 buffer directory for NativeS3FileSystem | Major | fs/s3 | David Phillips | David Phillips | +| [HADOOP-4374](https://issues.apache.org/jira/browse/HADOOP-4374) | JVM should not be killed but given an opportunity to exit gracefully | Major | . | Devaraj Das | Ravi Gummadi | +| [HADOOP-4346](https://issues.apache.org/jira/browse/HADOOP-4346) | Hadoop triggers a "soft" fd leak. | Major | io | Raghu Angadi | Raghu Angadi | +| [HADOOP-4325](https://issues.apache.org/jira/browse/HADOOP-4325) | Hadoop SocketInputStream.read() should return -1 in case of EOF. | Minor | io | Raghu Angadi | Raghu Angadi | +| [HADOOP-4262](https://issues.apache.org/jira/browse/HADOOP-4262) | message generated when the client exception has a null message is not useful | Minor | ipc | Steve Loughran | | +| [HADOOP-4245](https://issues.apache.org/jira/browse/HADOOP-4245) | KFS: Update the kfs jar file | Major | fs | Sriram Rao | Sriram Rao | +| [HADOOP-4238](https://issues.apache.org/jira/browse/HADOOP-4238) | [mapred] Unavailable schedulingInfo in "hadoop list [all]" output should be marked "N/A" | Major | . | Vinod Kumar Vavilapalli | Sreekanth Ramakrishnan | +| [HADOOP-4207](https://issues.apache.org/jira/browse/HADOOP-4207) | update derby.jar to eliminate the one time wait of 2-3 minutes while running junit tests | Minor | . | Prasad Chakka | Prasad Chakka | +| [HADOOP-4204](https://issues.apache.org/jira/browse/HADOOP-4204) | Fix warnings generated by FindBugs | Blocker | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-4141](https://issues.apache.org/jira/browse/HADOOP-4141) | ScriptBasedMapping has infinite loop with invalid jobconf parameter | Minor | . | Aaron Kimball | Aaron Kimball | +| [HADOOP-3921](https://issues.apache.org/jira/browse/HADOOP-3921) | Clover breaks nightly builds | Major | build | Tom White | Tom White | +| [HADOOP-3842](https://issues.apache.org/jira/browse/HADOOP-3842) | There is a window where the JobTracker is in the RUNNING state (i.e ready to accept jobs) and never executes them. | Major | . | Amar Kamat | Hemanth Yamijala | +| [HADOOP-3810](https://issues.apache.org/jira/browse/HADOOP-3810) | NameNode seems unstable on a cluster with little space left | Major | . | Raghu Angadi | Hairong Kuang | +| [HADOOP-3293](https://issues.apache.org/jira/browse/HADOOP-3293) | When an input split spans cross block boundary, the split location should be the host having most of bytes on it. | Major | . | Runping Qi | Jothi Padmanabhan | +| [HADOOP-2771](https://issues.apache.org/jira/browse/HADOOP-2771) | changing the number of reduces dramatically changes the time of the map time | Major | . | Owen O'Malley | | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-4857](https://issues.apache.org/jira/browse/HADOOP-4857) | TestUlimit is failing after Hadoop-4620 | Blocker | test | Sharad Agarwal | Ravi Gummadi | +| [HADOOP-4728](https://issues.apache.org/jira/browse/HADOOP-4728) | Tests for NameNode -checkpoint option with different configurations | Minor | . | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-4722](https://issues.apache.org/jira/browse/HADOOP-4722) | adding tests for quotas command line error messages | Minor | test | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-4688](https://issues.apache.org/jira/browse/HADOOP-4688) | multiple spills/reducers test case for mergeParts() needed | Major | . | Yuri Pradkin | Chris Douglas | +| [HADOOP-4505](https://issues.apache.org/jira/browse/HADOOP-4505) | Add a unit test to test faulty setup task and cleanup task killing the job | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-4458](https://issues.apache.org/jira/browse/HADOOP-4458) | Add a unit test for applications creating symlinks in wokring directory | Major | test | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-3192](https://issues.apache.org/jira/browse/HADOOP-3192) | Test behavior of very large (32GB) name nodes | Major | . | Robert Chansler | | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-4612](https://issues.apache.org/jira/browse/HADOOP-4612) | Remove RunJar's dependency on JobClient | Major | . | Sharad Agarwal | Sharad Agarwal | +| [HADOOP-4187](https://issues.apache.org/jira/browse/HADOOP-4187) | Create a MapReduce-specific ReflectionUtils that handles JobConf and JobConfigurable | Major | . | Tom White | Sharad Agarwal | +| [HADOOP-4185](https://issues.apache.org/jira/browse/HADOOP-4185) | Add setVerifyChecksum() method to FileSystem | Major | fs | Tom White | Sharad Agarwal | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + +
http://git-wip-us.apache.org/repos/asf/hadoop/blob/d759b4bd/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.0/RELEASENOTES.0.20.0.md ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.0/RELEASENOTES.0.20.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.0/RELEASENOTES.0.20.0.md new file mode 100644 index 0000000..339d7ab --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.0/RELEASENOTES.0.20.0.md @@ -0,0 +1,347 @@ + +<!--- +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +--> +# Apache Hadoop 0.20.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-5565](https://issues.apache.org/jira/browse/HADOOP-5565) | *Major* | **The job instrumentation API needs to have a method for finalizeJob,** + +Add finalizeJob & terminateJob methods to JobTrackerInstrumentation class + + +--- + +* [HADOOP-5548](https://issues.apache.org/jira/browse/HADOOP-5548) | *Blocker* | **Observed negative running maps on the job tracker** + +Adds synchronization for JobTracker methods in RecoveryManager. + + +--- + +* [HADOOP-5531](https://issues.apache.org/jira/browse/HADOOP-5531) | *Blocker* | **Remove Chukwa on branch-0.20** + +Disabled Chukwa unit tests for 0.20 branch only. + + +--- + +* [HADOOP-5521](https://issues.apache.org/jira/browse/HADOOP-5521) | *Major* | **Remove dependency of testcases on RESTART\_COUNT** + +This patch makes TestJobHistory and its dependent testcases independent of RESTART\_COUNT. + + +--- + +* [HADOOP-5468](https://issues.apache.org/jira/browse/HADOOP-5468) | *Major* | **Change Hadoop doc menu to sub-menus** + +Reformatted HTML documentation for Hadoop to use submenus at the left column. + + +--- + +* [HADOOP-5030](https://issues.apache.org/jira/browse/HADOOP-5030) | *Major* | **Chukwa RPM build improvements** + +Changed RPM install location to the value specified by build.properties file. + + +--- + +* [HADOOP-4970](https://issues.apache.org/jira/browse/HADOOP-4970) | *Major* | **Use the full path when move files to .Trash/Current** + +Changed trash facility to use absolute path of the deleted file. + + +--- + +* [HADOOP-4873](https://issues.apache.org/jira/browse/HADOOP-4873) | *Major* | **display minMaps/Reduces on advanced scheduler page** + +Changed fair scheduler UI to display minMaps and minReduces variables. + + +--- + +* [HADOOP-4843](https://issues.apache.org/jira/browse/HADOOP-4843) | *Major* | **Collect Job History log file and Job Conf file into Chukwa** + +Introduced Chuckwa collection of job history. + + +--- + +* [HADOOP-4827](https://issues.apache.org/jira/browse/HADOOP-4827) | *Major* | **Improve data aggregation in database** + +Improved framework for data aggregation in Chuckwa. + + +--- + +* [HADOOP-4826](https://issues.apache.org/jira/browse/HADOOP-4826) | *Major* | **Admin command saveNamespace.** + +Introduced new dfsadmin command saveNamespace to command the name service to do an immediate save of the file system image. + + +--- + +* [HADOOP-4789](https://issues.apache.org/jira/browse/HADOOP-4789) | *Minor* | **Change fair scheduler to share between pools by default, not between invidual jobs** + +Changed fair scheduler to divide resources equally between pools, not jobs. + + +--- + +* [HADOOP-4783](https://issues.apache.org/jira/browse/HADOOP-4783) | *Blocker* | **History files are given world readable permissions.** + +Changed history directory permissions to 750 and history file permissions to 740. + + +--- + +* [HADOOP-4749](https://issues.apache.org/jira/browse/HADOOP-4749) | *Major* | **reducer should output input data size when shuffling is done** + +Added a new counter REDUCE\_INPUT\_BYTES. + + +--- + +* [HADOOP-4661](https://issues.apache.org/jira/browse/HADOOP-4661) | *Major* | **distch: a tool for distributed ch{mod,own}** + +Introduced distch tool for parallel ch{mod, own, grp}. + + +--- + +* [HADOOP-4631](https://issues.apache.org/jira/browse/HADOOP-4631) | *Major* | **Split the default configurations into 3 parts** + +Split hadoop-default.xml into core-default.xml, hdfs-default.xml and mapreduce-default.xml. + + +--- + +* [HADOOP-4618](https://issues.apache.org/jira/browse/HADOOP-4618) | *Major* | **Move http server from FSNamesystem into NameNode.** + +Moved HTTP server from FSNameSystem to NameNode. Removed FSNamesystem.getNameNodeInfoPort(). Replaced FSNamesystem.getDFSNameNodeMachine() and FSNamesystem.getDFSNameNodePort() with new method FSNamesystem.getDFSNameNodeAddress(). Removed constructor NameNode(bindAddress, conf). + + +--- + +* [HADOOP-4576](https://issues.apache.org/jira/browse/HADOOP-4576) | *Major* | **Modify pending tasks count in the UI to pending jobs count in the UI** + +Changed capacity scheduler UI to better present number of running and pending tasks. + + +--- + +* [HADOOP-4575](https://issues.apache.org/jira/browse/HADOOP-4575) | *Major* | **An independent HTTPS proxy for HDFS** + +Introduced independent HSFTP proxy server for authenticated access to clusters. + + +--- + +* [HADOOP-4572](https://issues.apache.org/jira/browse/HADOOP-4572) | *Major* | **INode and its sub-classes should be package private** + +Moved org.apache.hadoop.hdfs.{CreateEditsLog, NNThroughputBenchmark} to org.apache.hadoop.hdfs.server.namenode. + + +--- + +* [HADOOP-4567](https://issues.apache.org/jira/browse/HADOOP-4567) | *Major* | **GetFileBlockLocations should return the NetworkTopology information of the machines that hosts those blocks** + +Changed GetFileBlockLocations to return topology information for nodes that host the block replicas. + + +--- + +* [HADOOP-4565](https://issues.apache.org/jira/browse/HADOOP-4565) | *Major* | **MultiFileInputSplit can use data locality information to create splits** + +Improved MultiFileInputFormat so that multiple blocks from the same node or same rack can be combined into a single split. + + +--- + +* [HADOOP-4454](https://issues.apache.org/jira/browse/HADOOP-4454) | *Minor* | **Support comments in 'slaves' file** + +Changed processing of conf/slaves file to allow # to begin a comment. + + +--- + +* [HADOOP-4445](https://issues.apache.org/jira/browse/HADOOP-4445) | *Major* | **Wrong number of running map/reduce tasks are displayed in queue information.** + +Changed JobTracker UI to better present the number of active tasks. + + +--- + +* [HADOOP-4435](https://issues.apache.org/jira/browse/HADOOP-4435) | *Minor* | **The JobTracker should display the amount of heap memory used** + +Changed JobTracker web status page to display the amount of heap memory in use. This changes the JobSubmissionProtocol. + + +--- + +* [HADOOP-4422](https://issues.apache.org/jira/browse/HADOOP-4422) | *Major* | **S3 file systems should not create bucket** + +Modified Hadoop file system to no longer create S3 buckets. Applications can create buckets for their S3 file systems by other means, for example, using the JetS3t API. + + +--- + +* [HADOOP-4374](https://issues.apache.org/jira/browse/HADOOP-4374) | *Major* | **JVM should not be killed but given an opportunity to exit gracefully** + +This patch (1) Adds a shutdownHook that does syncLogs sothat logs of the current task are flushed and log.index is up to date in cases like System.exit(), or killed using signals(other than SIGKILL). +(2) Changes writeToIndexFile() to write to a temporary index file first and then rename to log.index sothat updates to log.index file are atomic. + + +--- + +* [HADOOP-4305](https://issues.apache.org/jira/browse/HADOOP-4305) | *Major* | **repeatedly blacklisted tasktrackers should get declared dead** + +Improved TaskTracker blacklisting strategy to better exclude faulty tracker from executing tasks. + + +--- + +* [HADOOP-4284](https://issues.apache.org/jira/browse/HADOOP-4284) | *Major* | **Support for user configurable global filters on HttpServer** + +Introduced HttpServer method to support global filters. + + +--- + +* [HADOOP-4253](https://issues.apache.org/jira/browse/HADOOP-4253) | *Major* | **Fix warnings generated by FindBugs** + +Removed from class org.apache.hadoop.fs.RawLocalFileSystem deprecated methods public String getName(), public void lock(Path p, boolean shared) and public void release(Path p). + + +--- + +* [HADOOP-4234](https://issues.apache.org/jira/browse/HADOOP-4234) | *Minor* | **KFS: Allow KFS layer to interface with multiple KFS namenodes** + +Changed KFS glue layer to allow applications to interface with multiple KFS metaservers. + + +--- + +* [HADOOP-4210](https://issues.apache.org/jira/browse/HADOOP-4210) | *Major* | **Findbugs warnings are printed related to equals implementation of several classes** + +Changed public class org.apache.hadoop.mapreduce.ID to be an abstract class. Removed from class org.apache.hadoop.mapreduce.ID the methods public static ID read(DataInput in) and public static ID forName(String str). + + +--- + +* [HADOOP-4188](https://issues.apache.org/jira/browse/HADOOP-4188) | *Major* | **Remove Task's dependency on concrete file systems** + +Removed Task's dependency on concrete file systems by taking list from FileSystem class. Added statistics table to FileSystem class. Deprecated FileSystem method getStatistics(Class\<? extends FileSystem\> cls). + + +--- + +* [HADOOP-4179](https://issues.apache.org/jira/browse/HADOOP-4179) | *Major* | **Hadoop-Vaidya : Rule based performance diagnostic tool for Map/Reduce jobs** + +Introduced Vaidya rule based performance diagnostic tool for Map/Reduce jobs. + + +--- + +* [HADOOP-4103](https://issues.apache.org/jira/browse/HADOOP-4103) | *Major* | **Alert for missing blocks** + +Modified dfsadmin -report to report under replicated blocks. blocks with corrupt replicas, and missing blocks". + + +--- + +* [HADOOP-4035](https://issues.apache.org/jira/browse/HADOOP-4035) | *Blocker* | **Modify the capacity scheduler (HADOOP-3445) to schedule tasks based on memory requirements and task trackers free memory** + +Changed capacity scheduler policy to take note of task memory requirements and task tracker memory availability. + + +--- + +* [HADOOP-4029](https://issues.apache.org/jira/browse/HADOOP-4029) | *Major* | **NameNode should report status and performance for each replica of image and log** + +Added name node storage information to the dfshealth page, and moved data node information to a separated page. + + +--- + +* [HADOOP-3986](https://issues.apache.org/jira/browse/HADOOP-3986) | *Major* | **JobClient should not have a static configuration** + +Removed classes org.apache.hadoop.mapred.JobShell and org.apache.hadoop.mapred.TestJobShell. Removed from JobClient methods static void setCommandLineConfig(Configuration conf) and public static Configuration getCommandLineConfig(). + + +--- + +* [HADOOP-3923](https://issues.apache.org/jira/browse/HADOOP-3923) | *Minor* | **Deprecate org.apache.hadoop.mapred.StatusHttpServer** + +Moved class org.apache.hadoop.mapred.StatusHttpServer to org.apache.hadoop.http.HttpServer. + + +--- + +* [HADOOP-3750](https://issues.apache.org/jira/browse/HADOOP-3750) | *Major* | **Fix and enforce module dependencies** + +Removed deprecated method parseArgs from org.apache.hadoop.fs.FileSystem. + + +--- + +* [HADOOP-3497](https://issues.apache.org/jira/browse/HADOOP-3497) | *Major* | **File globbing with a PathFilter is too restrictive** + +Changed the semantics of file globbing with a PathFilter (using the globStatus method of FileSystem). Previously, the filtering was too restrictive, so that a glob of /*/* and a filter that only accepts /a/b would not have matched /a/b. With this change /a/b does match. + + +--- + +* [HADOOP-3422](https://issues.apache.org/jira/browse/HADOOP-3422) | *Major* | **Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen** + +Changed names of ganglia metrics to avoid conflicts and to better identify source function. + + +--- + +* [HADOOP-3344](https://issues.apache.org/jira/browse/HADOOP-3344) | *Major* | **libhdfs: always builds 32bit, even when x86\_64 Java used** + +Changed build procedure for libhdfs to build correctly for different platforms. Build instructions are in the Jira item. + + +--- + +* [HADOOP-3063](https://issues.apache.org/jira/browse/HADOOP-3063) | *Major* | **BloomMapFile - fail-fast version of MapFile for sparsely populated key space** + +Introduced BloomMapFile subclass of MapFile that creates a Bloom filter from all keys. + + +--- + +* [HADOOP-1650](https://issues.apache.org/jira/browse/HADOOP-1650) | *Major* | **Upgrade Jetty to 6.x** + +Upgraded all core servers to use Jetty 6 + + +--- + +* [HADOOP-1230](https://issues.apache.org/jira/browse/HADOOP-1230) | *Major* | **Replace parameters with context objects in Mapper, Reducer, Partitioner, InputFormat, and OutputFormat classes** + +Replaced parameters with context obejcts in Mapper, Reducer, Partitioner, InputFormat, and OutputFormat classes. + + + http://git-wip-us.apache.org/repos/asf/hadoop/blob/d759b4bd/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.1/CHANGES.0.20.1.md ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.1/CHANGES.0.20.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.1/CHANGES.0.20.1.md new file mode 100644 index 0000000..59af3f2 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.1/CHANGES.0.20.1.md @@ -0,0 +1,139 @@ + +<!--- +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +--> +# Apache Hadoop Changelog + +## Release 0.20.1 - 2009-09-01 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-5881](https://issues.apache.org/jira/browse/HADOOP-5881) | Simplify configuration related to task-memory-monitoring and memory-based scheduling | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [HADOOP-5726](https://issues.apache.org/jira/browse/HADOOP-5726) | Remove pre-emption from the capacity scheduler code base | Major | . | Hemanth Yamijala | rahul k singh | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-6080](https://issues.apache.org/jira/browse/HADOOP-6080) | Handling of Trash with quota | Major | fs | Koji Noguchi | Jakob Homan | +| [HADOOP-5714](https://issues.apache.org/jira/browse/HADOOP-5714) | Metric to show number of fs.exists (or number of getFileInfo) calls | Minor | metrics | Koji Noguchi | Jakob Homan | +| [HADOOP-3315](https://issues.apache.org/jira/browse/HADOOP-3315) | New binary file format | Major | io | Owen O'Malley | Hong Tang | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-635](https://issues.apache.org/jira/browse/HDFS-635) | HDFS Project page does not show 0.20.1 documentation/release information. | Major | documentation | Andy Sautins | | +| [HDFS-527](https://issues.apache.org/jira/browse/HDFS-527) | Refactor DFSClient constructors | Major | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-767](https://issues.apache.org/jira/browse/MAPREDUCE-767) | to remove mapreduce dependency on commons-cli2 | Major | contrib/streaming | Giridharan Kesavan | Amar Kamat | +| [MAPREDUCE-465](https://issues.apache.org/jira/browse/MAPREDUCE-465) | Deprecate org.apache.hadoop.mapred.lib.MultithreadedMapRunner | Minor | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-6215](https://issues.apache.org/jira/browse/HADOOP-6215) | fix GenericOptionParser to deal with -D with '=' in the value | Major | . | Owen O'Malley | Amar Kamat | +| [HADOOP-6145](https://issues.apache.org/jira/browse/HADOOP-6145) | No error message for deleting non-existant file or directory. | Major | fs | Suman Sehgal | Jakob Homan | +| [HADOOP-6141](https://issues.apache.org/jira/browse/HADOOP-6141) | hadoop 0.20 branch "test-patch" is broken | Major | build | Hong Tang | Hong Tang | +| [HADOOP-6139](https://issues.apache.org/jira/browse/HADOOP-6139) | Incomplete help message is displayed for rm and rmr options. | Minor | . | Suman Sehgal | Jakob Homan | +| [HADOOP-6017](https://issues.apache.org/jira/browse/HADOOP-6017) | NameNode and SecondaryNameNode fail to restart because of abnormal filenames. | Blocker | . | Raghu Angadi | Tsz Wo Nicholas Sze | +| [HADOOP-5951](https://issues.apache.org/jira/browse/HADOOP-5951) | StorageInfo needs Apache license header. | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-5937](https://issues.apache.org/jira/browse/HADOOP-5937) | Correct info message "Use hadoop dfs -safemode option" to " Use hdfs dfsadmin -safemode option" . | Minor | . | Ravi Phulari | Ravi Phulari | +| [HADOOP-5932](https://issues.apache.org/jira/browse/HADOOP-5932) | MemoryMatcher logs 0 as freeMemOnTT even though there are free slots available on TaskTraker | Major | . | Karam Singh | Vinod Kumar Vavilapalli | +| [HADOOP-5924](https://issues.apache.org/jira/browse/HADOOP-5924) | JT fails to recover the jobs after restart after HADOOP:4372 | Major | . | Ramya Sunil | Amar Kamat | +| [HADOOP-5921](https://issues.apache.org/jira/browse/HADOOP-5921) | JobTracker does not come up because of NotReplicatedYetException | Major | . | Amareshwari Sriramadasu | Amar Kamat | +| [HADOOP-5920](https://issues.apache.org/jira/browse/HADOOP-5920) | TestJobHistory fails some times. | Major | . | Amareshwari Sriramadasu | Amar Kamat | +| [HADOOP-5908](https://issues.apache.org/jira/browse/HADOOP-5908) | ArithmeticException in heartbeats with zero map jobs | Major | . | Vinod Kumar Vavilapalli | Amar Kamat | +| [HADOOP-5884](https://issues.apache.org/jira/browse/HADOOP-5884) | Capacity scheduler should account high memory jobs as using more capacity of the queue | Major | . | Hemanth Yamijala | Vinod Kumar Vavilapalli | +| [HADOOP-5883](https://issues.apache.org/jira/browse/HADOOP-5883) | TaskMemoryMonitorThread might shoot down tasks even if their processes momentarily exceed the requested memory | Major | . | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-5882](https://issues.apache.org/jira/browse/HADOOP-5882) | Progress is not updated when the New Reducer is running reduce phase | Blocker | . | Jothi Padmanabhan | Amareshwari Sriramadasu | +| [HADOOP-5850](https://issues.apache.org/jira/browse/HADOOP-5850) | map/reduce doesn't run jobs with 0 maps | Critical | . | Owen O'Malley | Vinod Kumar Vavilapalli | +| [HADOOP-5828](https://issues.apache.org/jira/browse/HADOOP-5828) | Use absolute path for JobTracker's mapred.local.dir in MiniMRCluster | Major | test | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-5746](https://issues.apache.org/jira/browse/HADOOP-5746) | Errors encountered in MROutputThread after the last map/reduce call can go undetected | Major | . | Devaraj Das | Amar Kamat | +| [HADOOP-5736](https://issues.apache.org/jira/browse/HADOOP-5736) | Update CapacityScheduler documentation to reflect latest changes | Major | . | Sreekanth Ramakrishnan | Sreekanth Ramakrishnan | +| [HADOOP-5719](https://issues.apache.org/jira/browse/HADOOP-5719) | Jobs failed during job initalization are never removed from Capacity Schedulers waiting list | Major | . | Sreekanth Ramakrishnan | Sreekanth Ramakrishnan | +| [HADOOP-5718](https://issues.apache.org/jira/browse/HADOOP-5718) | Capacity Scheduler should not check for presence of default queue while starting up. | Major | . | Sreekanth Ramakrishnan | Sreekanth Ramakrishnan | +| [HADOOP-5711](https://issues.apache.org/jira/browse/HADOOP-5711) | Change Namenode file close log to info | Minor | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-5691](https://issues.apache.org/jira/browse/HADOOP-5691) | org.apache.hadoop.mapreduce.Reducer should not be abstract. | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-5688](https://issues.apache.org/jira/browse/HADOOP-5688) | HftpFileSystem.getChecksum(..) does not work for the paths with scheme and authority | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-5655](https://issues.apache.org/jira/browse/HADOOP-5655) | TestMRServerPorts fails on java.net.BindException | Major | . | Hairong Kuang | Devaraj Das | +| [HADOOP-5654](https://issues.apache.org/jira/browse/HADOOP-5654) | TestReplicationPolicy.\<init\> fails on java.net.BindException | Major | test | Hairong Kuang | Hairong Kuang | +| [HADOOP-5648](https://issues.apache.org/jira/browse/HADOOP-5648) | Not able to generate gridmix.jar on already compiled version of hadoop | Major | benchmarks | Suman Sehgal | Giridharan Kesavan | +| [HADOOP-5646](https://issues.apache.org/jira/browse/HADOOP-5646) | TestQueueCapacities is failing Hudson tests for the last few builds | Major | . | Jothi Padmanabhan | Vinod Kumar Vavilapalli | +| [HADOOP-5641](https://issues.apache.org/jira/browse/HADOOP-5641) | Possible NPE in CapacityScheduler's MemoryMatcher | Major | . | Vinod Kumar Vavilapalli | Hemanth Yamijala | +| [HADOOP-5636](https://issues.apache.org/jira/browse/HADOOP-5636) | Job is left in Running state after a killJob | Critical | . | Amareshwari Sriramadasu | Amar Kamat | +| [HADOOP-5539](https://issues.apache.org/jira/browse/HADOOP-5539) | o.a.h.mapred.Merger not maintaining map out compression on intermediate files | Blocker | . | Billy Pearson | Jothi Padmanabhan | +| [HADOOP-5533](https://issues.apache.org/jira/browse/HADOOP-5533) | Recovery duration shown on the jobtracker webpage is inaccurate | Major | . | Amar Kamat | Amar Kamat | +| [HADOOP-5349](https://issues.apache.org/jira/browse/HADOOP-5349) | When the size required for a path is -1, LocalDirAllocator.getLocalPathForWrite fails with a DiskCheckerException when the disk it selects is bad. | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [HADOOP-5213](https://issues.apache.org/jira/browse/HADOOP-5213) | BZip2CompressionOutputStream NullPointerException | Blocker | io | Zheng Shao | Zheng Shao | +| [HADOOP-5210](https://issues.apache.org/jira/browse/HADOOP-5210) | Reduce Task Progress shows \> 100% when the total size of map outputs (for a single reducer) is high | Minor | . | Jothi Padmanabhan | Ravi Gummadi | +| [HADOOP-4674](https://issues.apache.org/jira/browse/HADOOP-4674) | hadoop fs -help should list detailed help info for the following commands: test, text, tail, stat & touchz | Trivial | fs | David NeSmith | Ravi Phulari | +| [HADOOP-4626](https://issues.apache.org/jira/browse/HADOOP-4626) | API link in forrest doc should point to the same version of hadoop. | Minor | documentation | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1022](https://issues.apache.org/jira/browse/HDFS-1022) | Merge under-10-min tests specs into one file | Major | test | Erik Steffl | Erik Steffl | +| [HDFS-525](https://issues.apache.org/jira/browse/HDFS-525) | ListPathsServlet.java uses static SimpleDateFormat that has threading issues | Major | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-438](https://issues.apache.org/jira/browse/HDFS-438) | Improve help message for quotas | Minor | . | Raghu Angadi | Raghu Angadi | +| [HDFS-167](https://issues.apache.org/jira/browse/HDFS-167) | DFSClient continues to retry indefinitely | Minor | hdfs-client | Derek Wollenstein | Bill Zeller | +| [HDFS-26](https://issues.apache.org/jira/browse/HDFS-26) | HADOOP-5862 for version .20 (Namespace quota exceeded message unclear) | Major | . | Boris Shkolnik | Boris Shkolnik | +| [MAPREDUCE-924](https://issues.apache.org/jira/browse/MAPREDUCE-924) | TestPipes must not directly invoke 'main' of pipes as an exit from main could cause the testcase to crash. | Major | pipes | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-911](https://issues.apache.org/jira/browse/MAPREDUCE-911) | TestTaskFail fail sometimes | Major | test | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-838](https://issues.apache.org/jira/browse/MAPREDUCE-838) | Task succeeds even when committer.commitTask fails with IOException | Blocker | task | Koji Noguchi | Amareshwari Sriramadasu | +| [MAPREDUCE-834](https://issues.apache.org/jira/browse/MAPREDUCE-834) | When TaskTracker config use old memory management values its memory monitoring is diabled. | Major | . | Karam Singh | Sreekanth Ramakrishnan | +| [MAPREDUCE-832](https://issues.apache.org/jira/browse/MAPREDUCE-832) | Too many WARN messages about deprecated memorty config variables in JobTacker log | Major | . | Karam Singh | rahul k singh | +| [MAPREDUCE-818](https://issues.apache.org/jira/browse/MAPREDUCE-818) | org.apache.hadoop.mapreduce.Counters.getGroup returns null if the group name doesnt exist. | Minor | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-807](https://issues.apache.org/jira/browse/MAPREDUCE-807) | Stray user files in mapred.system.dir with permissions other than 777 can prevent the jobtracker from starting up. | Blocker | jobtracker | Amar Kamat | Amar Kamat | +| [MAPREDUCE-805](https://issues.apache.org/jira/browse/MAPREDUCE-805) | Deadlock in Jobtracker | Major | . | Michael Tamm | Amar Kamat | +| [MAPREDUCE-796](https://issues.apache.org/jira/browse/MAPREDUCE-796) | Encountered "ClassCastException" on tasktracker while running wordcount with MultithreadedMapRunner | Major | examples | Suman Sehgal | Amar Kamat | +| [MAPREDUCE-745](https://issues.apache.org/jira/browse/MAPREDUCE-745) | TestRecoveryManager fails sometimes | Major | jobtracker | Amareshwari Sriramadasu | Amar Kamat | +| [MAPREDUCE-735](https://issues.apache.org/jira/browse/MAPREDUCE-735) | ArrayIndexOutOfBoundsException is thrown by KeyFieldBasedPartitioner | Major | . | Suman Sehgal | Amar Kamat | +| [MAPREDUCE-687](https://issues.apache.org/jira/browse/MAPREDUCE-687) | TestMiniMRMapRedDebugScript fails sometimes | Major | test | Amar Kamat | Amareshwari Sriramadasu | +| [MAPREDUCE-657](https://issues.apache.org/jira/browse/MAPREDUCE-657) | CompletedJobStatusStore hardcodes filesystem to hdfs | Major | jobtracker | Amar Kamat | Amar Kamat | +| [MAPREDUCE-565](https://issues.apache.org/jira/browse/MAPREDUCE-565) | Partitioner does not work with new API | Blocker | task | Jothi Padmanabhan | Owen O'Malley | +| [MAPREDUCE-430](https://issues.apache.org/jira/browse/MAPREDUCE-430) | Task stuck in cleanup with OutOfMemoryErrors | Major | . | Amareshwari Sriramadasu | Amar Kamat | +| [MAPREDUCE-421](https://issues.apache.org/jira/browse/MAPREDUCE-421) | mapred pipes might return exit code 0 even when failing | Major | pipes | Christian Kunz | Christian Kunz | +| [MAPREDUCE-383](https://issues.apache.org/jira/browse/MAPREDUCE-383) | pipes combiner does not reset properly after a spill | Major | . | Christian Kunz | Christian Kunz | +| [MAPREDUCE-179](https://issues.apache.org/jira/browse/MAPREDUCE-179) | setProgress not called for new RecordReaders | Blocker | . | Chris Douglas | Chris Douglas | +| [MAPREDUCE-130](https://issues.apache.org/jira/browse/MAPREDUCE-130) | Delete the jobconf copy from the log directory of the JobTracker when the job is retired | Major | . | Devaraj Das | Amar Kamat | +| [MAPREDUCE-124](https://issues.apache.org/jira/browse/MAPREDUCE-124) | When abortTask of OutputCommitter fails with an Exception for a map-only job, the task is marked as success | Major | . | Jothi Padmanabhan | Amareshwari Sriramadasu | +| [MAPREDUCE-40](https://issues.apache.org/jira/browse/MAPREDUCE-40) | Memory management variables need a backwards compatibility option after HADOOP-5881 | Blocker | . | Hemanth Yamijala | rahul k singh | +| [MAPREDUCE-18](https://issues.apache.org/jira/browse/MAPREDUCE-18) | Under load the shuffle sometimes gets incorrect data | Blocker | . | Owen O'Malley | Ravi Gummadi | +| [MAPREDUCE-2](https://issues.apache.org/jira/browse/MAPREDUCE-2) | ArrayOutOfIndex error in KeyFieldBasedPartitioner on empty key | Major | . | Amar Kamat | Amar Kamat | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-6213](https://issues.apache.org/jira/browse/HADOOP-6213) | Remove commons dependency on commons-cli2 | Blocker | util | Amar Kamat | Amar Kamat | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + +