http://git-wip-us.apache.org/repos/asf/hadoop/blob/d759b4bd/hadoop-common-project/hadoop-common/src/site/markdown/release/2.1.0-beta/RELEASENOTES.2.1.0-beta.md ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.1.0-beta/RELEASENOTES.2.1.0-beta.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.1.0-beta/RELEASENOTES.2.1.0-beta.md new file mode 100644 index 0000000..0dbd6bd --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.1.0-beta/RELEASENOTES.2.1.0-beta.md @@ -0,0 +1,757 @@ + +<!--- +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +--> +# Apache Hadoop 2.1.0-beta Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-9832](https://issues.apache.org/jira/browse/HADOOP-9832) | *Blocker* | **Add RPC header to client ping** + +Client ping will be sent as a RPC header with a reserved callId instead of as a sentinel RPC packet length. + + +--- + +* [HADOOP-9820](https://issues.apache.org/jira/browse/HADOOP-9820) | *Blocker* | **RPCv9 wire protocol is insufficient to support multiplexing** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-9698](https://issues.apache.org/jira/browse/HADOOP-9698) | *Blocker* | **RPCv9 client must honor server's SASL negotiate response** + +The RPC client now waits for the Server's SASL negotiate response before instantiating its SASL client. + + +--- + +* [HADOOP-9683](https://issues.apache.org/jira/browse/HADOOP-9683) | *Blocker* | **Wrap IpcConnectionContext in RPC headers** + +Connection context is now sent as a rpc header wrapped protobuf. + + +--- + +* [HADOOP-9649](https://issues.apache.org/jira/browse/HADOOP-9649) | *Blocker* | **Promote YARN service life-cycle libraries into Hadoop Common** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-9630](https://issues.apache.org/jira/browse/HADOOP-9630) | *Major* | **Remove IpcSerializationType** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-9425](https://issues.apache.org/jira/browse/HADOOP-9425) | *Major* | **Add error codes to rpc-response** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-9421](https://issues.apache.org/jira/browse/HADOOP-9421) | *Blocker* | **Convert SASL to use ProtoBuf and provide negotiation capabilities** + +Raw SASL protocol now uses protobufs wrapped with RPC headers. +The negotiation sequence incorporates the state of the exchange. +The server now has the ability to advertise its supported auth types. + + +--- + +* [HADOOP-9380](https://issues.apache.org/jira/browse/HADOOP-9380) | *Major* | **Add totalLength to rpc response** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-9228](https://issues.apache.org/jira/browse/HADOOP-9228) | *Minor* | **FileSystemContractTestBase never verifies that files are files** + +fixed in HADOOP-9258 + + +--- + +* [HADOOP-9227](https://issues.apache.org/jira/browse/HADOOP-9227) | *Trivial* | **FileSystemContractBaseTest doesn't test filesystem's mkdir/isDirectory() logic rigorously enough** + +fixed in HADOOP-9258 + + +--- + +* [HADOOP-9194](https://issues.apache.org/jira/browse/HADOOP-9194) | *Major* | **RPC Support for QoS** + +Part of the RPC version 9 change. A service class byte is added after the version byte. + + +--- + +* [HADOOP-9163](https://issues.apache.org/jira/browse/HADOOP-9163) | *Major* | **The rpc msg in ProtobufRpcEngine.proto should be moved out to avoid an extra copy** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-9151](https://issues.apache.org/jira/browse/HADOOP-9151) | *Major* | **Include RPC error info in RpcResponseHeader instead of sending it separately** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-8886](https://issues.apache.org/jira/browse/HADOOP-8886) | *Major* | **Remove KFS support** + +Kosmos FS (KFS) is no longer maintained and Hadoop support has been removed. KFS has been replaced by QFS (HADOOP-8885). + + +--- + +* [HADOOP-8562](https://issues.apache.org/jira/browse/HADOOP-8562) | *Major* | **Enhancements to support Hadoop on Windows Server and Windows Azure environments** + +This umbrella jira makes enhancements to support Hadoop natively on Windows Server and Windows Azure environments. + + +--- + +* [HADOOP-8470](https://issues.apache.org/jira/browse/HADOOP-8470) | *Major* | **Implementation of 4-layer subclass of NetworkTopology (NetworkTopologyWithNodeGroup)** + +This patch should be checked in together (or after) with JIRA Hadoop-8469: https://issues.apache.org/jira/browse/HADOOP-8469 + + +--- + +* [HDFS-5083](https://issues.apache.org/jira/browse/HDFS-5083) | *Blocker* | **Update the HDFS compatibility version range** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-4996](https://issues.apache.org/jira/browse/HDFS-4996) | *Minor* | **ClientProtocol#metaSave can be made idempotent by overwriting the output file instead of appending to it** + +The dfsadmin -metasave command has been changed to overwrite the output file. Previously, this command would append to the output file if it already existed. + + +--- + +* [HDFS-4866](https://issues.apache.org/jira/browse/HDFS-4866) | *Blocker* | **Protocol buffer support cannot compile under C** + +The Protocol Buffers definition of the inter-namenode protocol required a change for compatibility with compiled C clients. This is a backwards-incompatible change. A namenode prior to this change will not be able to communicate with a namenode after this change. + + +--- + +* [HDFS-4832](https://issues.apache.org/jira/browse/HDFS-4832) | *Critical* | **Namenode doesn't change the number of missing blocks in safemode when DNs rejoin or leave** + +This change makes name node keep its internal replication queues and data node state updated in manual safe mode. This allows metrics and UI to present up-to-date information while in safe mode. The behavior during start-up safe mode is unchanged. + + +--- + +* [HDFS-4659](https://issues.apache.org/jira/browse/HDFS-4659) | *Major* | **Support setting execution bit for regular files** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-4519](https://issues.apache.org/jira/browse/HDFS-4519) | *Major* | **Support override of jsvc binary and log file locations when launching secure datanode.** + +With this improvement the following options are available in release 1.2.0 and later on 1.x release stream: +1. jsvc location can be overridden by setting environment variable JSVC\_HOME. Defaults to jsvc binary packaged within the Hadoop distro. +2. jsvc log output is directed to the file defined by JSVC\_OUTFILE. Defaults to $HADOOP\_LOG\_DIR/jsvc.out. +3. jsvc error output is directed to the file defined by JSVC\_ERRFILE file. Defaults to $HADOOP\_LOG\_DIR/jsvc.err. + +With this improvement the following options are available in release 2.0.4 and later on 2.x release stream: +1. jsvc log output is directed to the file defined by JSVC\_OUTFILE. Defaults to $HADOOP\_LOG\_DIR/jsvc.out. +2. jsvc error output is directed to the file defined by JSVC\_ERRFILE file. Defaults to $HADOOP\_LOG\_DIR/jsvc.err. + +For overriding jsvc location on 2.x releases, here is the release notes from HDFS-2303: +To run secure Datanodes users must install jsvc for their platform and set JSVC\_HOME to point to the location of jsvc in their environment. + + +--- + +* [HDFS-4434](https://issues.apache.org/jira/browse/HDFS-4434) | *Major* | **Provide a mapping from INodeId to INode** + +This change adds support for referencing files and directories based on fileID/inodeID using a path /.reserved/.inodes/\<inodeid\>. With this change creating a file or directory /.reserved is not longer allowed. Before upgrading to a release with this change, files /.reserved needs to be renamed to another name. + + +--- + +* [HDFS-4305](https://issues.apache.org/jira/browse/HDFS-4305) | *Minor* | **Add a configurable limit on number of blocks per file, and min block size** + +This change introduces a maximum number of blocks per file, by default one million, and a minimum block size, by default 1MB. These can optionally be changed via the configuration settings "dfs.namenode.fs-limits.max-blocks-per-file" and "dfs.namenode.fs-limits.min-block-size", respectively. + + +--- + +* [HDFS-4053](https://issues.apache.org/jira/browse/HDFS-4053) | *Major* | **Increase the default block size** + +The default blocks size prior to this change was 64MB. This jira changes the default block size to 128MB. To go back to previous behavior, please configure the in hdfs-site.xml, the configuration parameter "dfs.blocksize" to 67108864. + + +--- + +* [HDFS-2802](https://issues.apache.org/jira/browse/HDFS-2802) | *Major* | **Support for RW/RO snapshots in HDFS** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-1804](https://issues.apache.org/jira/browse/HDFS-1804) | *Minor* | **Add a new block-volume device choosing policy that looks at free space** + +There is now a new option to have the DN take into account available disk space on each volume when choosing where to place a replica when performing an HDFS write. This can be enabled by setting the config "dfs.datanode.fsdataset.volume.choosing.policy" to the value "org.apache.hadoop.hdfs.server.datanode.fsdataset.AvailableSpaceVolumeChoosingPolicy". + + +--- + +* [MAPREDUCE-5399](https://issues.apache.org/jira/browse/MAPREDUCE-5399) | *Blocker* | **Unnecessary Configuration instantiation in IFileInputStream slows down merge** + +Fixes blank Configuration object creation overhead by reusing the Job configuration in InMemoryReader. + + +--- + +* [MAPREDUCE-5304](https://issues.apache.org/jira/browse/MAPREDUCE-5304) | *Blocker* | **mapreduce.Job killTask/failTask/getTaskCompletionEvents methods have incompatible signature changes** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-5300](https://issues.apache.org/jira/browse/MAPREDUCE-5300) | *Major* | **Two function signature changes in filecache.DistributedCache** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-5237](https://issues.apache.org/jira/browse/MAPREDUCE-5237) | *Major* | **ClusterStatus incompatiblity issues with MR1** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-5234](https://issues.apache.org/jira/browse/MAPREDUCE-5234) | *Major* | **Signature changes for getTaskId of TaskReport in mapred** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-5233](https://issues.apache.org/jira/browse/MAPREDUCE-5233) | *Major* | **Functions are changed or removed from Job in jobcontrol** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-5184](https://issues.apache.org/jira/browse/MAPREDUCE-5184) | *Major* | **Document MR Binary Compatibility vis-a-vis hadoop-1 and hadoop-2** + +Document MR Binary Compatibility vis-a-vis hadoop-1 and hadoop-2 for end-users. + + +--- + +* [MAPREDUCE-5176](https://issues.apache.org/jira/browse/MAPREDUCE-5176) | *Major* | **Preemptable annotations (to support preemption in MR)** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-5156](https://issues.apache.org/jira/browse/MAPREDUCE-5156) | *Blocker* | **Hadoop-examples-1.x.x.jar cannot run on Yarn** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-4942](https://issues.apache.org/jira/browse/MAPREDUCE-4942) | *Major* | **mapreduce.Job has a bunch of methods that throw InterruptedException so its incompatible with MR1** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-4737](https://issues.apache.org/jira/browse/MAPREDUCE-4737) | *Major* | ** Hadoop does not close output file / does not call Mapper.cleanup if exception in map** + +Ensure that mapreduce APIs are semantically consistent with mapred API w.r.t Mapper.cleanup and Reducer.cleanup; in the sense that cleanup is now called even if there is an error. The old mapred API already ensures that Mapper.close and Reducer.close are invoked during error handling. Note that it is an incompatible change, however end-users can override Mapper.run and Reducer.run to get the old (inconsistent) behaviour. + + +--- + +* [MAPREDUCE-4356](https://issues.apache.org/jira/browse/MAPREDUCE-4356) | *Major* | **Provide access to ParsedTask.obtainTaskAttempts()** + +Made the method ParsedTask.obtainTaskAttempts() public. + + +--- + +* [MAPREDUCE-4149](https://issues.apache.org/jira/browse/MAPREDUCE-4149) | *Major* | **Rumen fails to parse certain counter strings** + +Fixes Rumen to parse counter strings containing the special characters "{" and "}". + + +--- + +* [MAPREDUCE-4100](https://issues.apache.org/jira/browse/MAPREDUCE-4100) | *Minor* | **Sometimes gridmix emulates data larger much larger then acutal counter for map only jobs** + +Bug fixed in compression emulation feature for map only jobs. + + +--- + +* [MAPREDUCE-4083](https://issues.apache.org/jira/browse/MAPREDUCE-4083) | *Major* | **GridMix emulated job tasks.resource-usage emulator for CPU usage throws NPE when Trace contains cumulativeCpuUsage value of 0 at attempt level** + +Fixes NPE in cpu emulation in Gridmix + + +--- + +* [MAPREDUCE-4067](https://issues.apache.org/jira/browse/MAPREDUCE-4067) | *Critical* | **Replace YarnRemoteException with IOException in MRv2 APIs** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-3953](https://issues.apache.org/jira/browse/MAPREDUCE-3953) | *Major* | **Gridmix throws NPE and does not simulate a job if the trace contains null taskStatus for a task** + +Fixes NPE and makes Gridmix simulate succeeded-jobs-with-failed-tasks. All tasks of such simulated jobs(including the failed ones of original job) will succeed. + + +--- + +* [MAPREDUCE-3829](https://issues.apache.org/jira/browse/MAPREDUCE-3829) | *Major* | **[Gridmix] Gridmix should give better error message when input-data directory already exists and -generate option is given** + +Makes Gridmix emit out correct error message when the input data directory already exists and -generate option is used. Makes Gridmix exit with proper exit codes when Gridmix fails in args-processing, startup/setup. + + +--- + +* [MAPREDUCE-3787](https://issues.apache.org/jira/browse/MAPREDUCE-3787) | *Major* | **[Gridmix] Improve STRESS mode** + +JobMonitor can now deploy multiple threads for faster job-status polling. Use 'gridmix.job-monitor.thread-count' to set the number of threads. Stress mode now relies on the updates from the job monitor instead of polling for job status. Failures in job submission now get reported to the statistics module and ultimately reported to the user via summary. + + +--- + +* [MAPREDUCE-3757](https://issues.apache.org/jira/browse/MAPREDUCE-3757) | *Major* | **Rumen Folder is not adjusting the shuffleFinished and sortFinished times of reduce task attempts** + +Fixed the sortFinishTime and shuffleFinishTime adjustments in Rumen Folder. + + +--- + +* [MAPREDUCE-2722](https://issues.apache.org/jira/browse/MAPREDUCE-2722) | *Major* | **Gridmix simulated job's map's hdfsBytesRead counter is wrong when compressed input is used** + +Makes Gridmix use the uncompressed input data size while simulating map tasks in the case where compressed input data was used in original job. + + +--- + +* [YARN-1056](https://issues.apache.org/jira/browse/YARN-1056) | *Trivial* | **Fix configs yarn.resourcemanager.resourcemanager.connect.{max.wait.secs\|retry\_interval.secs}** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-926](https://issues.apache.org/jira/browse/YARN-926) | *Blocker* | **ContainerManagerProtcol APIs should take in requests for multiple containers** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-918](https://issues.apache.org/jira/browse/YARN-918) | *Blocker* | **ApplicationMasterProtocol doesn't need ApplicationAttemptId in the payload after YARN-701** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-869](https://issues.apache.org/jira/browse/YARN-869) | *Blocker* | **ResourceManagerAdministrationProtocol should neither be public(yet) nor in yarn.api** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-851](https://issues.apache.org/jira/browse/YARN-851) | *Major* | **Share NMTokens using NMTokenCache (api-based) instead of memory based approach which is used currently.** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-841](https://issues.apache.org/jira/browse/YARN-841) | *Major* | **Annotate and document AuxService APIs** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-840](https://issues.apache.org/jira/browse/YARN-840) | *Major* | **Move ProtoUtils to yarn.api.records.pb.impl** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-837](https://issues.apache.org/jira/browse/YARN-837) | *Major* | **ClusterInfo.java doesn't seem to belong to org.apache.hadoop.yarn** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-834](https://issues.apache.org/jira/browse/YARN-834) | *Blocker* | **Review/fix annotations for yarn-client module and clearly differentiate \*Async apis** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-831](https://issues.apache.org/jira/browse/YARN-831) | *Blocker* | **Remove resource min from GetNewApplicationResponse** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-829](https://issues.apache.org/jira/browse/YARN-829) | *Major* | **Rename RMTokenSelector to be RMDelegationTokenSelector** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-828](https://issues.apache.org/jira/browse/YARN-828) | *Major* | **Remove YarnVersionAnnotation** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-826](https://issues.apache.org/jira/browse/YARN-826) | *Major* | **Move Clock/SystemClock to util package** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-824](https://issues.apache.org/jira/browse/YARN-824) | *Major* | **Add static factory to yarn client lib interface and change it to abstract class** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-823](https://issues.apache.org/jira/browse/YARN-823) | *Major* | **Move RMAdmin from yarn.client to yarn.client.cli and rename as RMAdminCLI** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-822](https://issues.apache.org/jira/browse/YARN-822) | *Major* | **Rename ApplicationToken to AMRMToken** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-821](https://issues.apache.org/jira/browse/YARN-821) | *Major* | **Rename FinishApplicationMasterRequest.setFinishApplicationStatus to setFinalApplicationStatus to be consistent with getter** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-806](https://issues.apache.org/jira/browse/YARN-806) | *Major* | **Move ContainerExitStatus from yarn.api to yarn.api.records** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-792](https://issues.apache.org/jira/browse/YARN-792) | *Major* | **Move NodeHealthStatus from yarn.api.record to yarn.server.api.record** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-791](https://issues.apache.org/jira/browse/YARN-791) | *Blocker* | **Ensure that RM RPC APIs that return nodes are consistent with /nodes REST API** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-787](https://issues.apache.org/jira/browse/YARN-787) | *Blocker* | **Remove resource min from Yarn client API** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-777](https://issues.apache.org/jira/browse/YARN-777) | *Major* | **Remove unreferenced objects from proto** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-756](https://issues.apache.org/jira/browse/YARN-756) | *Major* | **Move PreemptionContainer/PremptionContract/PreemptionMessage/StrictPreemptionContract/PreemptionResourceRequest to api.records** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-755](https://issues.apache.org/jira/browse/YARN-755) | *Major* | **Rename AllocateResponse.reboot to AllocateResponse.resync** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-753](https://issues.apache.org/jira/browse/YARN-753) | *Major* | **Add individual factory method for api protocol records** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-749](https://issues.apache.org/jira/browse/YARN-749) | *Major* | **Rename ResourceRequest (get,set)HostName to (get,set)ResourceName** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-748](https://issues.apache.org/jira/browse/YARN-748) | *Major* | **Move BuilderUtils from yarn-common to yarn-server-common** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-746](https://issues.apache.org/jira/browse/YARN-746) | *Major* | **rename Service.register() and Service.unregister() to registerServiceListener() & unregisterServiceListener() respectively** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-735](https://issues.apache.org/jira/browse/YARN-735) | *Major* | **Make ApplicationAttemptID, ContainerID, NodeID immutable** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-724](https://issues.apache.org/jira/browse/YARN-724) | *Major* | **Move ProtoBase from api.records to api.records.impl.pb** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-720](https://issues.apache.org/jira/browse/YARN-720) | *Major* | **container-log4j.properties should not refer to mapreduce properties** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-716](https://issues.apache.org/jira/browse/YARN-716) | *Major* | **Make ApplicationID immutable** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-701](https://issues.apache.org/jira/browse/YARN-701) | *Blocker* | **ApplicationTokens should be used irrespective of kerberos** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-694](https://issues.apache.org/jira/browse/YARN-694) | *Major* | **Start using NMTokens to authenticate all communication with NM** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-684](https://issues.apache.org/jira/browse/YARN-684) | *Major* | **ContainerManager.startContainer needs to only have ContainerTokenIdentifier instead of the whole Container** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-642](https://issues.apache.org/jira/browse/YARN-642) | *Major* | **Fix up /nodes REST API to have 1 param and be consistent with the Java API** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-635](https://issues.apache.org/jira/browse/YARN-635) | *Major* | **Rename YarnRemoteException to YarnException** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-633](https://issues.apache.org/jira/browse/YARN-633) | *Major* | **Change RMAdminProtocol api to throw IOException and YarnRemoteException** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-632](https://issues.apache.org/jira/browse/YARN-632) | *Major* | **Change ContainerManager api to throw IOException and YarnRemoteException** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-631](https://issues.apache.org/jira/browse/YARN-631) | *Major* | **Change ClientRMProtocol api to throw IOException and YarnRemoteException** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-630](https://issues.apache.org/jira/browse/YARN-630) | *Major* | **Change AMRMProtocol api to throw IOException and YarnRemoteException** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-629](https://issues.apache.org/jira/browse/YARN-629) | *Major* | **Make YarnRemoteException not be rooted at IOException** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-615](https://issues.apache.org/jira/browse/YARN-615) | *Major* | **ContainerLaunchContext.containerTokens should simply be called tokens** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-610](https://issues.apache.org/jira/browse/YARN-610) | *Blocker* | **ClientToken (ClientToAMToken) should not be set in the environment** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-579](https://issues.apache.org/jira/browse/YARN-579) | *Major* | **Make ApplicationToken part of Container's token list to help RM-restart** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-571](https://issues.apache.org/jira/browse/YARN-571) | *Major* | **User should not be part of ContainerLaunchContext** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-561](https://issues.apache.org/jira/browse/YARN-561) | *Major* | **Nodemanager should set some key information into the environment of every container that it launches.** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-553](https://issues.apache.org/jira/browse/YARN-553) | *Minor* | **Have YarnClient generate a directly usable ApplicationSubmissionContext** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-536](https://issues.apache.org/jira/browse/YARN-536) | *Major* | **Remove ContainerStatus, ContainerState from Container api interface as they will not be called by the container object** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-530](https://issues.apache.org/jira/browse/YARN-530) | *Major* | **Define Service model strictly, implement AbstractService for robust subclassing, migrate yarn-common services** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-440](https://issues.apache.org/jira/browse/YARN-440) | *Major* | **Flatten RegisterNodeManagerResponse** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-439](https://issues.apache.org/jira/browse/YARN-439) | *Major* | **Flatten NodeHeartbeatResponse** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-396](https://issues.apache.org/jira/browse/YARN-396) | *Major* | **Rationalize AllocateResponse in RM scheduler API** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-387](https://issues.apache.org/jira/browse/YARN-387) | *Blocker* | **Fix inconsistent protocol naming** + +**WARNING: No release note provided for this incompatible change.** + + +
http://git-wip-us.apache.org/repos/asf/hadoop/blob/d759b4bd/hadoop-common-project/hadoop-common/src/site/markdown/release/2.1.1-beta/CHANGES.2.1.1-beta.md ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.1.1-beta/CHANGES.2.1.1-beta.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.1.1-beta/CHANGES.2.1.1-beta.md new file mode 100644 index 0000000..245bdd4 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.1.1-beta/CHANGES.2.1.1-beta.md @@ -0,0 +1,239 @@ + +<!--- +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +--> +# Apache Hadoop Changelog + +## Release 2.1.1-beta - 2013-09-16 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9944](https://issues.apache.org/jira/browse/HADOOP-9944) | RpcRequestHeaderProto defines callId as uint32 while ipc.Client.CONNECTION\_CONTEXT\_CALL\_ID is signed (-3) | Blocker | . | Arun C Murthy | Arun C Murthy | +| [YARN-1170](https://issues.apache.org/jira/browse/YARN-1170) | yarn proto definitions should specify package as 'hadoop.yarn' | Blocker | . | Arun C Murthy | Binglin Chang | +| [YARN-707](https://issues.apache.org/jira/browse/YARN-707) | Add user info in the YARN ClientToken | Blocker | . | Bikas Saha | Jason Lowe | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9789](https://issues.apache.org/jira/browse/HADOOP-9789) | Support server advertised kerberos principals | Critical | ipc, security | Daryn Sharp | Daryn Sharp | +| [HDFS-5118](https://issues.apache.org/jira/browse/HDFS-5118) | Provide testing support for DFSClient to drop RPC responses | Major | . | Jing Zhao | Jing Zhao | +| [HDFS-5076](https://issues.apache.org/jira/browse/HDFS-5076) | Add MXBean methods to query NN's transaction information and JournalNode's journal status | Minor | . | Jing Zhao | Jing Zhao | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9962](https://issues.apache.org/jira/browse/HADOOP-9962) | in order to avoid dependency divergence within Hadoop itself lets enable DependencyConvergence | Major | build | Roman Shaposhnik | Roman Shaposhnik | +| [HADOOP-9945](https://issues.apache.org/jira/browse/HADOOP-9945) | HAServiceState should have a state for stopped services | Minor | ha | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-9918](https://issues.apache.org/jira/browse/HADOOP-9918) | Add addIfService() to CompositeService | Minor | . | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-9886](https://issues.apache.org/jira/browse/HADOOP-9886) | Turn warning message in RetryInvocationHandler to debug | Minor | . | Arpit Gupta | Arpit Gupta | +| [HADOOP-9879](https://issues.apache.org/jira/browse/HADOOP-9879) | Move the version info of zookeeper dependencies to hadoop-project/pom | Minor | build | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-9833](https://issues.apache.org/jira/browse/HADOOP-9833) | move slf4j to version 1.7.5 | Minor | build | Steve Loughran | Kousuke Saruta | +| [HADOOP-9831](https://issues.apache.org/jira/browse/HADOOP-9831) | Make checknative shell command accessible on Windows. | Minor | bin | Chris Nauroth | Chris Nauroth | +| [HADOOP-9821](https://issues.apache.org/jira/browse/HADOOP-9821) | ClientId should have getMsb/getLsb methods | Minor | . | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [HADOOP-9803](https://issues.apache.org/jira/browse/HADOOP-9803) | Add generic type parameter to RetryInvocationHandler | Minor | ipc | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-9802](https://issues.apache.org/jira/browse/HADOOP-9802) | Support Snappy codec on Windows. | Major | io | Chris Nauroth | Chris Nauroth | +| [HADOOP-9686](https://issues.apache.org/jira/browse/HADOOP-9686) | Easy access to final parameters in Configuration | Major | conf | Jason Lowe | Jason Lowe | +| [HADOOP-9672](https://issues.apache.org/jira/browse/HADOOP-9672) | Upgrade Avro dependency to 1.7.4 | Major | . | Sandy Ryza | Sandy Ryza | +| [HADOOP-9669](https://issues.apache.org/jira/browse/HADOOP-9669) | Reduce the number of byte array creations and copies in XDR data manipulation | Major | nfs | Tsz Wo Nicholas Sze | Haohui Mai | +| [HADOOP-9446](https://issues.apache.org/jira/browse/HADOOP-9446) | Support Kerberos HTTP SPNEGO authentication for non-SUN JDK | Major | security | Yu Gao | Yu Gao | +| [HADOOP-8814](https://issues.apache.org/jira/browse/HADOOP-8814) | Inefficient comparison with the empty string. Use isEmpty() instead | Minor | conf, fs, fs/s3, ha, io, metrics, performance, record, security, util | Brandon Li | Brandon Li | +| [HDFS-5128](https://issues.apache.org/jira/browse/HDFS-5128) | Allow multiple net interfaces to be used with HA namenode RPC server | Critical | . | Kihwal Lee | Kihwal Lee | +| [HDFS-5061](https://issues.apache.org/jira/browse/HDFS-5061) | Make FSNameSystem#auditLoggers an unmodifiable list | Major | namenode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-5045](https://issues.apache.org/jira/browse/HDFS-5045) | Add more unit tests for retry cache to cover all AtMostOnce methods | Minor | . | Jing Zhao | Jing Zhao | +| [HDFS-4926](https://issues.apache.org/jira/browse/HDFS-4926) | namenode webserver's page has a tooltip that is inconsistent with the datanode HTML link | Trivial | namenode | Joseph Lorenzini | Vivek Ganesan | +| [HDFS-4905](https://issues.apache.org/jira/browse/HDFS-4905) | Add appendToFile command to "hdfs dfs" | Minor | tools | Arpit Agarwal | Arpit Agarwal | +| [HDFS-4513](https://issues.apache.org/jira/browse/HDFS-4513) | Clarify WebHDFS REST API that all JSON respsonses may contain additional properties | Minor | documentation, webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-3245](https://issues.apache.org/jira/browse/HDFS-3245) | Add metrics and web UI for cluster version summary | Major | namenode | Todd Lipcon | Ravi Prakash | +| [MAPREDUCE-5478](https://issues.apache.org/jira/browse/MAPREDUCE-5478) | TeraInputFormat unnecessarily defines its own FileSplit subclass | Minor | examples | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-5379](https://issues.apache.org/jira/browse/MAPREDUCE-5379) | Include token tracking ids in jobconf | Major | job submission, security | Sandy Ryza | Karthik Kambatla | +| [MAPREDUCE-5367](https://issues.apache.org/jira/browse/MAPREDUCE-5367) | Local jobs all use same local working directory | Major | . | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-1981](https://issues.apache.org/jira/browse/MAPREDUCE-1981) | Improve getSplits performance by using listLocatedStatus | Major | job submission | Hairong Kuang | Hairong Kuang | +| [YARN-1137](https://issues.apache.org/jira/browse/YARN-1137) | Add support whitelist for system users to Yarn container-executor.c | Major | nodemanager | Alejandro Abdelnur | Roman Shaposhnik | +| [YARN-1117](https://issues.apache.org/jira/browse/YARN-1117) | Improve help message for $ yarn applications and $yarn node | Major | client | Tassapol Athiapinya | Xuan Gong | +| [YARN-1081](https://issues.apache.org/jira/browse/YARN-1081) | Minor improvement to output header for $ yarn node -list | Minor | client | Tassapol Athiapinya | Akira AJISAKA | +| [YARN-1080](https://issues.apache.org/jira/browse/YARN-1080) | Improve help message for $ yarn logs | Major | client | Tassapol Athiapinya | Xuan Gong | +| [YARN-1074](https://issues.apache.org/jira/browse/YARN-1074) | Clean up YARN CLI app list to show only running apps. | Major | client | Tassapol Athiapinya | Xuan Gong | +| [YARN-910](https://issues.apache.org/jira/browse/YARN-910) | Allow auxiliary services to listen for container starts and completions | Major | nodemanager | Sandy Ryza | Alejandro Abdelnur | +| [YARN-758](https://issues.apache.org/jira/browse/YARN-758) | Augment MockNM to use multiple cores | Minor | . | Bikas Saha | Karthik Kambatla | +| [YARN-696](https://issues.apache.org/jira/browse/YARN-696) | Enable multiple states to to be specified in Resource Manager apps REST call | Major | resourcemanager | Trevor Lorimer | Trevor Lorimer | +| [YARN-589](https://issues.apache.org/jira/browse/YARN-589) | Expose a REST API for monitoring the fair scheduler | Major | scheduler | Sandy Ryza | Sandy Ryza | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9977](https://issues.apache.org/jira/browse/HADOOP-9977) | Hadoop services won't start with different keypass and keystorepass when https is enabled | Major | security | Yesha Vora | Chris Nauroth | +| [HADOOP-9961](https://issues.apache.org/jira/browse/HADOOP-9961) | versions of a few transitive dependencies diverged between hadoop subprojects | Minor | build | Roman Shaposhnik | Roman Shaposhnik | +| [HADOOP-9960](https://issues.apache.org/jira/browse/HADOOP-9960) | Upgrade Jersey version to 1.9 | Blocker | . | Brock Noland | Karthik Kambatla | +| [HADOOP-9958](https://issues.apache.org/jira/browse/HADOOP-9958) | Add old constructor back to DelegationTokenInformation to unbreak downstream builds | Major | . | Andrew Wang | Andrew Wang | +| [HADOOP-9932](https://issues.apache.org/jira/browse/HADOOP-9932) | Improper synchronization in RetryCache | Blocker | . | Kihwal Lee | Kihwal Lee | +| [HADOOP-9924](https://issues.apache.org/jira/browse/HADOOP-9924) | FileUtil.createJarWithClassPath() does not generate relative classpath correctly | Major | fs | shanyu zhao | shanyu zhao | +| [HADOOP-9916](https://issues.apache.org/jira/browse/HADOOP-9916) | Race condition in ipc.Client causes TestIPC timeout | Minor | . | Binglin Chang | Binglin Chang | +| [HADOOP-9910](https://issues.apache.org/jira/browse/HADOOP-9910) | proxy server start and stop documentation wrong | Minor | . | André Kelpe | | +| [HADOOP-9906](https://issues.apache.org/jira/browse/HADOOP-9906) | Move HAZKUtil to o.a.h.util.ZKUtil and make inner-classes public | Minor | ha | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-9899](https://issues.apache.org/jira/browse/HADOOP-9899) | Remove the debug message added by HADOOP-8855 | Minor | security | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-9880](https://issues.apache.org/jira/browse/HADOOP-9880) | SASL changes from HADOOP-9421 breaks Secure HA NN | Blocker | . | Kihwal Lee | Daryn Sharp | +| [HADOOP-9868](https://issues.apache.org/jira/browse/HADOOP-9868) | Server must not advertise kerberos realm | Blocker | ipc | Daryn Sharp | Daryn Sharp | +| [HADOOP-9858](https://issues.apache.org/jira/browse/HADOOP-9858) | Remove unused private RawLocalFileSystem#execCommand method from branch-2. | Trivial | fs | Chris Nauroth | Chris Nauroth | +| [HADOOP-9857](https://issues.apache.org/jira/browse/HADOOP-9857) | Tests block and sometimes timeout on Windows due to invalid entropy source. | Major | build, test | Chris Nauroth | Chris Nauroth | +| [HADOOP-9806](https://issues.apache.org/jira/browse/HADOOP-9806) | PortmapInterface should check if the procedure is out-of-range | Major | nfs | Brandon Li | Brandon Li | +| [HADOOP-9801](https://issues.apache.org/jira/browse/HADOOP-9801) | Configuration#writeXml uses platform defaulting encoding, which may mishandle multi-byte characters. | Major | conf | Chris Nauroth | Chris Nauroth | +| [HADOOP-9774](https://issues.apache.org/jira/browse/HADOOP-9774) | RawLocalFileSystem.listStatus() return absolute paths when input path is relative on Windows | Major | fs | shanyu zhao | shanyu zhao | +| [HADOOP-9768](https://issues.apache.org/jira/browse/HADOOP-9768) | chown and chgrp reject users and groups with spaces on platforms where spaces are otherwise acceptable | Major | fs | Chris Nauroth | Chris Nauroth | +| [HADOOP-9757](https://issues.apache.org/jira/browse/HADOOP-9757) | Har metadata cache can grow without limit | Major | fs | Jason Lowe | Cristina L. Abad | +| [HADOOP-9557](https://issues.apache.org/jira/browse/HADOOP-9557) | hadoop-client excludes commons-httpclient | Major | build | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-9435](https://issues.apache.org/jira/browse/HADOOP-9435) | Support building the JNI code against the IBM JVM | Major | build | Tian Hong Wang | Tian Hong Wang | +| [HADOOP-9381](https://issues.apache.org/jira/browse/HADOOP-9381) | Document dfs cp -f option | Trivial | . | Keegan Witt | Keegan Witt | +| [HADOOP-9315](https://issues.apache.org/jira/browse/HADOOP-9315) | Port HADOOP-9249 hadoop-maven-plugins Clover fix to branch-2 to fix build failures | Major | build | Dennis Y | Chris Nauroth | +| [HDFS-5231](https://issues.apache.org/jira/browse/HDFS-5231) | Fix broken links in the document of HDFS Federation | Minor | . | Haohui Mai | Haohui Mai | +| [HDFS-5219](https://issues.apache.org/jira/browse/HDFS-5219) | Add configuration keys for retry policy in WebHDFSFileSystem | Major | webhdfs | Haohui Mai | Haohui Mai | +| [HDFS-5192](https://issues.apache.org/jira/browse/HDFS-5192) | NameNode may fail to start when dfs.client.test.drop.namenode.response.number is set | Minor | . | Jing Zhao | Jing Zhao | +| [HDFS-5159](https://issues.apache.org/jira/browse/HDFS-5159) | Secondary NameNode fails to checkpoint if error occurs downloading edits on first checkpoint | Major | namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-5150](https://issues.apache.org/jira/browse/HDFS-5150) | Allow per NN SPN for internal SPNEGO. | Blocker | . | Kihwal Lee | Kihwal Lee | +| [HDFS-5140](https://issues.apache.org/jira/browse/HDFS-5140) | Too many safemode monitor threads being created in the standby namenode causing it to fail with out of memory error | Blocker | ha | Arpit Gupta | Jing Zhao | +| [HDFS-5132](https://issues.apache.org/jira/browse/HDFS-5132) | Deadlock in NameNode between SafeModeMonitor#run and DatanodeManager#handleHeartbeat | Blocker | namenode | Arpit Gupta | Kihwal Lee | +| [HDFS-5124](https://issues.apache.org/jira/browse/HDFS-5124) | DelegationTokenSecretManager#retrievePassword can cause deadlock in NameNode | Blocker | namenode | Deepesh Khandelwal | Daryn Sharp | +| [HDFS-5111](https://issues.apache.org/jira/browse/HDFS-5111) | Remove duplicated error message for snapshot commands when processing invalid arguments | Minor | snapshots | Jing Zhao | Jing Zhao | +| [HDFS-5106](https://issues.apache.org/jira/browse/HDFS-5106) | TestDatanodeBlockScanner fails on Windows due to incorrect path format | Minor | test | Chuan Liu | Chuan Liu | +| [HDFS-5105](https://issues.apache.org/jira/browse/HDFS-5105) | TestFsck fails on Windows | Minor | . | Chuan Liu | Chuan Liu | +| [HDFS-5103](https://issues.apache.org/jira/browse/HDFS-5103) | TestDirectoryScanner fails on Windows | Minor | test | Chuan Liu | Chuan Liu | +| [HDFS-5102](https://issues.apache.org/jira/browse/HDFS-5102) | Snapshot names should not be allowed to contain slash characters | Major | snapshots | Aaron T. Myers | Jing Zhao | +| [HDFS-5100](https://issues.apache.org/jira/browse/HDFS-5100) | TestNamenodeRetryCache fails on Windows due to incorrect cleanup | Minor | test | Chuan Liu | Chuan Liu | +| [HDFS-5099](https://issues.apache.org/jira/browse/HDFS-5099) | Namenode#copyEditLogSegmentsToSharedDir should close EditLogInputStreams upon finishing | Major | namenode | Chuan Liu | Chuan Liu | +| [HDFS-5091](https://issues.apache.org/jira/browse/HDFS-5091) | Support for spnego keytab separate from the JournalNode keytab for secure HA | Minor | . | Jing Zhao | Jing Zhao | +| [HDFS-5080](https://issues.apache.org/jira/browse/HDFS-5080) | BootstrapStandby not working with QJM when the existing NN is active | Major | ha, qjm | Jing Zhao | Jing Zhao | +| [HDFS-5077](https://issues.apache.org/jira/browse/HDFS-5077) | NPE in FSNamesystem.commitBlockSynchronization() | Major | namenode | Konstantin Shvachko | Plamen Jeliazkov | +| [HDFS-5055](https://issues.apache.org/jira/browse/HDFS-5055) | nn fails to download checkpointed image from snn in some setups | Blocker | namenode | Allen Wittenauer | Vinayakumar B | +| [HDFS-5047](https://issues.apache.org/jira/browse/HDFS-5047) | Supress logging of full stack trace of quota and lease exceptions | Major | namenode | Kihwal Lee | Robert Parker | +| [HDFS-5043](https://issues.apache.org/jira/browse/HDFS-5043) | For HdfsFileStatus, set default value of childrenNum to -1 instead of 0 to avoid confusing applications | Major | . | Brandon Li | Brandon Li | +| [HDFS-5028](https://issues.apache.org/jira/browse/HDFS-5028) | LeaseRenewer throw java.util.ConcurrentModificationException when timeout | Major | . | zhaoyunjiong | zhaoyunjiong | +| [HDFS-4993](https://issues.apache.org/jira/browse/HDFS-4993) | fsck can fail if a file is renamed or deleted | Major | . | Kihwal Lee | Robert Parker | +| [HDFS-4898](https://issues.apache.org/jira/browse/HDFS-4898) | BlockPlacementPolicyWithNodeGroup.chooseRemoteRack() fails to properly fallback to local rack | Minor | namenode | Eric Sirianni | Tsz Wo Nicholas Sze | +| [HDFS-4680](https://issues.apache.org/jira/browse/HDFS-4680) | Audit logging of delegation tokens for MR tracing | Major | namenode, security | Andrew Wang | Andrew Wang | +| [HDFS-4632](https://issues.apache.org/jira/browse/HDFS-4632) | globStatus using backslash for escaping does not work on Windows | Major | test | Chris Nauroth | Chuan Liu | +| [HDFS-4594](https://issues.apache.org/jira/browse/HDFS-4594) | WebHDFS open sets Content-Length header to what is specified by length parameter rather than how much data is actually returned. | Minor | webhdfs | Arpit Gupta | Chris Nauroth | +| [HDFS-2994](https://issues.apache.org/jira/browse/HDFS-2994) | If lease soft limit is recovered successfully the append can fail | Major | . | Todd Lipcon | Tao Luo | +| [MAPREDUCE-5497](https://issues.apache.org/jira/browse/MAPREDUCE-5497) | '5s sleep' in MRAppMaster.shutDownJob is only needed before stopping ClientService | Major | . | Jian He | Jian He | +| [MAPREDUCE-5493](https://issues.apache.org/jira/browse/MAPREDUCE-5493) | In-memory map outputs can be leaked after shuffle completes | Blocker | mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-5483](https://issues.apache.org/jira/browse/MAPREDUCE-5483) | revert MAPREDUCE-5357 | Major | distcp | Alejandro Abdelnur | Robert Kanter | +| [MAPREDUCE-5475](https://issues.apache.org/jira/browse/MAPREDUCE-5475) | MRClientService does not verify ACLs properly | Blocker | mr-am, mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-5470](https://issues.apache.org/jira/browse/MAPREDUCE-5470) | LocalJobRunner does not work on Windows. | Major | . | Chris Nauroth | Sandy Ryza | +| [MAPREDUCE-5468](https://issues.apache.org/jira/browse/MAPREDUCE-5468) | AM recovery does not work for map only jobs | Blocker | . | Yesha Vora | Vinod Kumar Vavilapalli | +| [MAPREDUCE-5454](https://issues.apache.org/jira/browse/MAPREDUCE-5454) | TestDFSIO fails intermittently on JDK7 | Major | test | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-5446](https://issues.apache.org/jira/browse/MAPREDUCE-5446) | TestJobHistoryEvents and TestJobHistoryParsing have race conditions | Major | mrv2, test | Jason Lowe | Jason Lowe | +| [MAPREDUCE-5440](https://issues.apache.org/jira/browse/MAPREDUCE-5440) | TestCopyCommitter Fails on JDK7 | Major | mrv2 | Robert Parker | Robert Parker | +| [MAPREDUCE-5428](https://issues.apache.org/jira/browse/MAPREDUCE-5428) | HistoryFileManager doesn't stop threads when service is stopped | Major | jobhistoryserver, mrv2 | Jason Lowe | Karthik Kambatla | +| [MAPREDUCE-5425](https://issues.apache.org/jira/browse/MAPREDUCE-5425) | Junit in TestJobHistoryServer failing in jdk 7 | Major | jobhistoryserver | Ashwin Shankar | Robert Parker | +| [MAPREDUCE-5414](https://issues.apache.org/jira/browse/MAPREDUCE-5414) | TestTaskAttempt fails jdk7 with NullPointerException | Major | test | Nemon Lou | Nemon Lou | +| [MAPREDUCE-5385](https://issues.apache.org/jira/browse/MAPREDUCE-5385) | JobContext cache files api are broken | Blocker | . | Omkar Vinit Joshi | Omkar Vinit Joshi | +| [MAPREDUCE-5358](https://issues.apache.org/jira/browse/MAPREDUCE-5358) | MRAppMaster throws invalid transitions for JobImpl | Major | mr-am | Devaraj K | Devaraj K | +| [MAPREDUCE-5317](https://issues.apache.org/jira/browse/MAPREDUCE-5317) | Stale files left behind for failed jobs | Major | mrv2 | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-5251](https://issues.apache.org/jira/browse/MAPREDUCE-5251) | Reducer should not implicate map attempt if it has insufficient space to fetch map output | Major | mrv2 | Jason Lowe | Ashwin Shankar | +| [MAPREDUCE-5164](https://issues.apache.org/jira/browse/MAPREDUCE-5164) | command "mapred job" and "mapred queue" omit HADOOP\_CLIENT\_OPTS | Major | . | Nemon Lou | Nemon Lou | +| [MAPREDUCE-5020](https://issues.apache.org/jira/browse/MAPREDUCE-5020) | Compile failure with JDK8 | Major | client | Trevor Robinson | Trevor Robinson | +| [MAPREDUCE-5001](https://issues.apache.org/jira/browse/MAPREDUCE-5001) | LocalJobRunner has race condition resulting in job failures | Major | . | Brock Noland | Sandy Ryza | +| [MAPREDUCE-3193](https://issues.apache.org/jira/browse/MAPREDUCE-3193) | FileInputFormat doesn't read files recursively in the input path dir | Major | mrv1, mrv2 | Ramgopal N | Devaraj K | +| [YARN-1194](https://issues.apache.org/jira/browse/YARN-1194) | TestContainerLogsPage fails with native builds | Minor | nodemanager | Roman Shaposhnik | Roman Shaposhnik | +| [YARN-1189](https://issues.apache.org/jira/browse/YARN-1189) | NMTokenSecretManagerInNM is not being told when applications have finished | Blocker | . | Jason Lowe | Omkar Vinit Joshi | +| [YARN-1176](https://issues.apache.org/jira/browse/YARN-1176) | RM web services ClusterMetricsInfo total nodes doesn't include unhealthy nodes | Critical | resourcemanager | Thomas Graves | Jonathan Eagles | +| [YARN-1152](https://issues.apache.org/jira/browse/YARN-1152) | Invalid key to HMAC computation error when getting application report for completed app attempt | Blocker | resourcemanager | Jason Lowe | Jason Lowe | +| [YARN-1144](https://issues.apache.org/jira/browse/YARN-1144) | Unmanaged AMs registering a tracking URI should not be proxy-fied | Critical | resourcemanager | Alejandro Abdelnur | Alejandro Abdelnur | +| [YARN-1124](https://issues.apache.org/jira/browse/YARN-1124) | By default yarn application -list should display all the applications in a state other than FINISHED / FAILED | Blocker | . | Omkar Vinit Joshi | Xuan Gong | +| [YARN-1120](https://issues.apache.org/jira/browse/YARN-1120) | Make ApplicationConstants.Environment.USER definition OS neutral | Minor | . | Chuan Liu | Chuan Liu | +| [YARN-1107](https://issues.apache.org/jira/browse/YARN-1107) | Job submitted with Delegation token in secured environment causes RM to fail during RM restart | Blocker | resourcemanager | Arpit Gupta | Omkar Vinit Joshi | +| [YARN-1101](https://issues.apache.org/jira/browse/YARN-1101) | Active nodes can be decremented below 0 | Major | resourcemanager | Robert Parker | Robert Parker | +| [YARN-1094](https://issues.apache.org/jira/browse/YARN-1094) | RM restart throws Null pointer Exception in Secure Env | Blocker | . | Yesha Vora | Vinod Kumar Vavilapalli | +| [YARN-1093](https://issues.apache.org/jira/browse/YARN-1093) | Corrections to Fair Scheduler documentation | Major | documentation | Wing Yew Poon | | +| [YARN-1083](https://issues.apache.org/jira/browse/YARN-1083) | ResourceManager should fail when yarn.nm.liveness-monitor.expiry-interval-ms is set less than heartbeat interval | Major | resourcemanager | Yesha Vora | Zhijie Shen | +| [YARN-1082](https://issues.apache.org/jira/browse/YARN-1082) | Secure RM with recovery enabled and rm state store on hdfs fails with gss exception | Blocker | resourcemanager | Arpit Gupta | Vinod Kumar Vavilapalli | +| [YARN-1078](https://issues.apache.org/jira/browse/YARN-1078) | TestNodeManagerResync, TestNodeManagerShutdown, and TestNodeStatusUpdater fail on Windows | Minor | . | Chuan Liu | Chuan Liu | +| [YARN-1077](https://issues.apache.org/jira/browse/YARN-1077) | TestContainerLaunch fails on Windows | Minor | . | Chuan Liu | Chuan Liu | +| [YARN-1025](https://issues.apache.org/jira/browse/YARN-1025) | ResourceManager and NodeManager do not load native libraries on Windows. | Major | nodemanager, resourcemanager | Chris Nauroth | Chris Nauroth | +| [YARN-1008](https://issues.apache.org/jira/browse/YARN-1008) | MiniYARNCluster with multiple nodemanagers, all nodes have same key for allocations | Major | nodemanager | Alejandro Abdelnur | Alejandro Abdelnur | +| [YARN-1006](https://issues.apache.org/jira/browse/YARN-1006) | Nodes list web page on the RM web UI is broken | Major | . | Jian He | Xuan Gong | +| [YARN-994](https://issues.apache.org/jira/browse/YARN-994) | HeartBeat thread in AMRMClientAsync does not handle runtime exception correctly | Major | . | Xuan Gong | Xuan Gong | +| [YARN-981](https://issues.apache.org/jira/browse/YARN-981) | YARN/MR2/Job-history /logs link does not have correct content | Major | . | Xuan Gong | Jian He | +| [YARN-966](https://issues.apache.org/jira/browse/YARN-966) | The thread of ContainerLaunch#call will fail without any signal if getLocalizedResources() is called when the container is not at LOCALIZED | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-957](https://issues.apache.org/jira/browse/YARN-957) | Capacity Scheduler tries to reserve the memory more than what node manager reports. | Blocker | . | Omkar Vinit Joshi | Omkar Vinit Joshi | +| [YARN-948](https://issues.apache.org/jira/browse/YARN-948) | RM should validate the release container list before actually releasing them | Major | . | Omkar Vinit Joshi | Omkar Vinit Joshi | +| [YARN-942](https://issues.apache.org/jira/browse/YARN-942) | In Fair Scheduler documentation, inconsistency on which properties have prefix | Major | scheduler | Sandy Ryza | Akira AJISAKA | +| [YARN-903](https://issues.apache.org/jira/browse/YARN-903) | DistributedShell throwing Errors in logs after successfull completion | Major | applications/distributed-shell | Abhishek Kapoor | Omkar Vinit Joshi | +| [YARN-881](https://issues.apache.org/jira/browse/YARN-881) | Priority#compareTo method seems to be wrong. | Major | . | Jian He | Jian He | +| [YARN-643](https://issues.apache.org/jira/browse/YARN-643) | WHY appToken is removed both in BaseFinalTransition and AMUnregisteredTransition AND clientToken is removed in FinalTransition and not BaseFinalTransition | Major | . | Jian He | Xuan Gong | +| [YARN-602](https://issues.apache.org/jira/browse/YARN-602) | NodeManager should mandatorily set some Environment variables into every containers that it launches | Major | . | Xuan Gong | Kenji Kikushima | +| [YARN-337](https://issues.apache.org/jira/browse/YARN-337) | RM handles killed application tracking URL poorly | Major | resourcemanager | Jason Lowe | Jason Lowe | +| [YARN-107](https://issues.apache.org/jira/browse/YARN-107) | ClientRMService.forceKillApplication() should handle the non-RUNNING applications properly | Major | resourcemanager | Devaraj K | Xuan Gong | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-5249](https://issues.apache.org/jira/browse/HDFS-5249) | Fix dumper thread which may die silently | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-5234](https://issues.apache.org/jira/browse/HDFS-5234) | Move RpcFrameDecoder out of the public API | Minor | nfs | Haohui Mai | Haohui Mai | +| [HDFS-5212](https://issues.apache.org/jira/browse/HDFS-5212) | Refactor RpcMessage and NFS3Response to support different types of authentication information | Major | nfs | Jing Zhao | Jing Zhao | +| [HDFS-5199](https://issues.apache.org/jira/browse/HDFS-5199) | Add more debug trace for NFS READ and WRITE | Trivial | nfs | Brandon Li | Brandon Li | +| [HDFS-5136](https://issues.apache.org/jira/browse/HDFS-5136) | MNT EXPORT should give the full group list which can mount the exports | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-5110](https://issues.apache.org/jira/browse/HDFS-5110) | Change FSDataOutputStream to HdfsDataOutputStream for opened streams to fix type cast error | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-5107](https://issues.apache.org/jira/browse/HDFS-5107) | Fix array copy error in Readdir and Readdirplus responses | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-5104](https://issues.apache.org/jira/browse/HDFS-5104) | Support dotdot name in NFS LOOKUP operation | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-5085](https://issues.apache.org/jira/browse/HDFS-5085) | Refactor o.a.h.nfs to support different types of authentications | Major | nfs | Brandon Li | Jing Zhao | +| [HDFS-5078](https://issues.apache.org/jira/browse/HDFS-5078) | Support file append in NFSv3 gateway to enable data streaming to HDFS | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-5071](https://issues.apache.org/jira/browse/HDFS-5071) | Change hdfs-nfs parent project to hadoop-project | Major | nfs | Kihwal Lee | Brandon Li | +| [HDFS-5069](https://issues.apache.org/jira/browse/HDFS-5069) | Include hadoop-nfs and hadoop-hdfs-nfs into hadoop dist for NFS deployment | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-5067](https://issues.apache.org/jira/browse/HDFS-5067) | Support symlink operations | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-4971](https://issues.apache.org/jira/browse/HDFS-4971) | Move IO operations out of locking in OpenFileCtx | Major | nfs | Jing Zhao | Jing Zhao | +| [HDFS-4962](https://issues.apache.org/jira/browse/HDFS-4962) | Use enum for nfs constants | Minor | nfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-4947](https://issues.apache.org/jira/browse/HDFS-4947) | Add NFS server export table to control export by hostname or IP range | Major | nfs | Brandon Li | Jing Zhao | +| [HDFS-4763](https://issues.apache.org/jira/browse/HDFS-4763) | Add script changes/utility for starting NFS gateway | Major | nfs | Brandon Li | Brandon Li | +| [MAPREDUCE-5476](https://issues.apache.org/jira/browse/MAPREDUCE-5476) | Job can fail when RM restarts after staging dir is cleaned but before MR successfully unregister with RM | Blocker | . | Jian He | Jian He | +| [MAPREDUCE-5466](https://issues.apache.org/jira/browse/MAPREDUCE-5466) | Historyserver does not refresh the result of restarted jobs after RM restart | Blocker | . | Yesha Vora | Jian He | +| [MAPREDUCE-5462](https://issues.apache.org/jira/browse/MAPREDUCE-5462) | In map-side sort, swap entire meta entries instead of indexes for better cache performance | Major | performance, task | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-5441](https://issues.apache.org/jira/browse/MAPREDUCE-5441) | JobClient exit whenever RM issue Reboot command to 1st attempt App Master. | Major | applicationmaster, client | Rohith Sharma K S | Jian He | +| [YARN-1184](https://issues.apache.org/jira/browse/YARN-1184) | ClassCastException is thrown during preemption When a huge job is submitted to a queue B whose resources is used by a job in queueA | Major | capacityscheduler, resourcemanager | J.Andreina | Chris Douglas | +| [YARN-1116](https://issues.apache.org/jira/browse/YARN-1116) | Populate AMRMTokens back to AMRMTokenSecretManager after RM restarts | Major | resourcemanager | Jian He | Jian He | +| [YARN-1085](https://issues.apache.org/jira/browse/YARN-1085) | Yarn and MRv2 should do HTTP client authentication in kerberos setup. | Blocker | nodemanager, resourcemanager | Jaimin D Jetly | Omkar Vinit Joshi | +| [YARN-1049](https://issues.apache.org/jira/browse/YARN-1049) | ContainerExistStatus should define a status for preempted containers | Blocker | api | Alejandro Abdelnur | Alejandro Abdelnur | +| [YARN-906](https://issues.apache.org/jira/browse/YARN-906) | Cancelling ContainerLaunch#call at KILLING causes that the container cannot be completed | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-771](https://issues.apache.org/jira/browse/YARN-771) | AMRMClient support for resource blacklisting | Major | . | Bikas Saha | Junping Du | +| [YARN-573](https://issues.apache.org/jira/browse/YARN-573) | Shared data structures in Public Localizer and Private Localizer are not Thread safe. | Critical | . | Omkar Vinit Joshi | Omkar Vinit Joshi | +| [YARN-540](https://issues.apache.org/jira/browse/YARN-540) | Race condition causing RM to potentially relaunch already unregistered AMs on RM restart | Major | resourcemanager | Jian He | Jian He | +| [YARN-502](https://issues.apache.org/jira/browse/YARN-502) | RM crash with NPE on NODE\_REMOVED event with FairScheduler | Major | . | Lohit Vijayarenu | Mayank Bansal | +| [YARN-292](https://issues.apache.org/jira/browse/YARN-292) | ResourceManager throws ArrayIndexOutOfBoundsException while handling CONTAINER\_ALLOCATED for application attempt | Major | resourcemanager | Devaraj K | Zhijie Shen | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [YARN-1034](https://issues.apache.org/jira/browse/YARN-1034) | Remove "experimental" in the Fair Scheduler documentation | Trivial | documentation, scheduler | Sandy Ryza | Karthik Kambatla | +| [YARN-1001](https://issues.apache.org/jira/browse/YARN-1001) | YARN should provide per application-type and state statistics | Blocker | api | Srimanth Gunturi | Zhijie Shen | + + http://git-wip-us.apache.org/repos/asf/hadoop/blob/d759b4bd/hadoop-common-project/hadoop-common/src/site/markdown/release/2.1.1-beta/RELEASENOTES.2.1.1-beta.md ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.1.1-beta/RELEASENOTES.2.1.1-beta.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.1.1-beta/RELEASENOTES.2.1.1-beta.md new file mode 100644 index 0000000..db2f170 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.1.1-beta/RELEASENOTES.2.1.1-beta.md @@ -0,0 +1,52 @@ + +<!--- +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +--> +# Apache Hadoop 2.1.1-beta Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-9944](https://issues.apache.org/jira/browse/HADOOP-9944) | *Blocker* | **RpcRequestHeaderProto defines callId as uint32 while ipc.Client.CONNECTION\_CONTEXT\_CALL\_ID is signed (-3)** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-5118](https://issues.apache.org/jira/browse/HDFS-5118) | *Major* | **Provide testing support for DFSClient to drop RPC responses** + +Used for testing when NameNode HA is enabled. Users can use a new configuration property "dfs.client.test.drop.namenode.response.number" to specify the number of responses that DFSClient will drop in each RPC call. This feature can help testing functionalities such as NameNode retry cache. + + +--- + +* [YARN-1170](https://issues.apache.org/jira/browse/YARN-1170) | *Blocker* | **yarn proto definitions should specify package as 'hadoop.yarn'** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-707](https://issues.apache.org/jira/browse/YARN-707) | *Blocker* | **Add user info in the YARN ClientToken** + +**WARNING: No release note provided for this incompatible change.** + + + http://git-wip-us.apache.org/repos/asf/hadoop/blob/d759b4bd/hadoop-common-project/hadoop-common/src/site/markdown/release/2.2.0/CHANGES.2.2.0.md ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.2.0/CHANGES.2.2.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.2.0/CHANGES.2.2.0.md new file mode 100644 index 0000000..e70a79a --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.2.0/CHANGES.2.2.0.md @@ -0,0 +1,152 @@ + +<!--- +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +--> +# Apache Hadoop Changelog + +## Release 2.2.0 - 2013-10-15 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10020](https://issues.apache.org/jira/browse/HADOOP-10020) | disable symlinks temporarily | Blocker | fs | Colin Patrick McCabe | Sanjay Radia | +| [YARN-1229](https://issues.apache.org/jira/browse/YARN-1229) | Define constraints on Auxiliary Service names. Change ShuffleHandler service name from mapreduce.shuffle to mapreduce\_shuffle. | Blocker | nodemanager | Tassapol Athiapinya | Xuan Gong | +| [YARN-1228](https://issues.apache.org/jira/browse/YARN-1228) | Clean up Fair Scheduler configuration loading | Major | scheduler | Sandy Ryza | Sandy Ryza | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9758](https://issues.apache.org/jira/browse/HADOOP-9758) | Provide configuration option for FileSystem/FileContext symlink resolution | Major | . | Andrew Wang | Andrew Wang | +| [HADOOP-8315](https://issues.apache.org/jira/browse/HADOOP-8315) | Support SASL-authenticated ZooKeeper in ActiveStandbyElector | Major | auto-failover, ha | Todd Lipcon | Todd Lipcon | +| [HDFS-5308](https://issues.apache.org/jira/browse/HDFS-5308) | Replace HttpConfig#getSchemePrefix with implicit schemes in HDFS JSP | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-5256](https://issues.apache.org/jira/browse/HDFS-5256) | Use guava LoadingCache to implement DFSClientCache | Major | nfs | Haohui Mai | Haohui Mai | +| [HDFS-5139](https://issues.apache.org/jira/browse/HDFS-5139) | Remove redundant -R option from setrep | Major | tools | Arpit Agarwal | Arpit Agarwal | +| [HDFS-4817](https://issues.apache.org/jira/browse/HDFS-4817) | make HDFS advisory caching configurable on a per-file basis | Minor | hdfs-client | Colin Patrick McCabe | Colin Patrick McCabe | +| [YARN-1246](https://issues.apache.org/jira/browse/YARN-1246) | Log application status in the rm log when app is done running | Minor | . | Arpit Gupta | Arpit Gupta | +| [YARN-1213](https://issues.apache.org/jira/browse/YARN-1213) | Restore config to ban submitting to undeclared pools in the Fair Scheduler | Major | scheduler | Sandy Ryza | Sandy Ryza | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10012](https://issues.apache.org/jira/browse/HADOOP-10012) | Secure Oozie jobs fail with delegation token renewal exception in Namenode HA setup | Blocker | ha | Arpit Gupta | Suresh Srinivas | +| [HADOOP-10003](https://issues.apache.org/jira/browse/HADOOP-10003) | HarFileSystem.listLocatedStatus() fails | Major | fs | Jason Dere | | +| [HADOOP-9976](https://issues.apache.org/jira/browse/HADOOP-9976) | Different versions of avro and avro-maven-plugin | Major | . | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-9776](https://issues.apache.org/jira/browse/HADOOP-9776) | HarFileSystem.listStatus() returns invalid authority if port number is empty | Major | fs | shanyu zhao | shanyu zhao | +| [HADOOP-9761](https://issues.apache.org/jira/browse/HADOOP-9761) | ViewFileSystem#rename fails when using DistributedFileSystem | Blocker | viewfs | Andrew Wang | Andrew Wang | +| [HDFS-5300](https://issues.apache.org/jira/browse/HDFS-5300) | FSNameSystem#deleteSnapshot() should not check owner in case of permissions disabled | Major | namenode | Vinayakumar B | Vinayakumar B | +| [HDFS-5299](https://issues.apache.org/jira/browse/HDFS-5299) | DFS client hangs in updatePipeline RPC when failover happened | Blocker | namenode | Vinayakumar B | Vinayakumar B | +| [HDFS-5289](https://issues.apache.org/jira/browse/HDFS-5289) | Race condition in TestRetryCacheWithHA#testCreateSymlink causes spurious test failure | Major | test | Aaron T. Myers | Aaron T. Myers | +| [HDFS-5279](https://issues.apache.org/jira/browse/HDFS-5279) | Guard against NullPointerException in NameNode JSP pages before initialization of FSNamesystem. | Major | namenode | Chris Nauroth | Chris Nauroth | +| [HDFS-5268](https://issues.apache.org/jira/browse/HDFS-5268) | NFS write commit verifier is not set in a few places | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-5265](https://issues.apache.org/jira/browse/HDFS-5265) | Namenode fails to start when dfs.https.port is unspecified | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-5258](https://issues.apache.org/jira/browse/HDFS-5258) | Skip tests in TestHDFSCLI that are not applicable on Windows. | Minor | test | Chris Nauroth | Chuan Liu | +| [HDFS-5255](https://issues.apache.org/jira/browse/HDFS-5255) | Distcp job fails with hsftp when https is enabled in insecure cluster | Major | . | Yesha Vora | Arpit Agarwal | +| [HDFS-5251](https://issues.apache.org/jira/browse/HDFS-5251) | Race between the initialization of NameNode and the http server | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-5228](https://issues.apache.org/jira/browse/HDFS-5228) | The RemoteIterator returned by DistributedFileSystem.listFiles(..) may throw NPE | Blocker | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-5031](https://issues.apache.org/jira/browse/HDFS-5031) | BlockScanner scans the block multiple times and on restart scans everything | Blocker | datanode | Vinayakumar B | Vinayakumar B | +| [MAPREDUCE-5554](https://issues.apache.org/jira/browse/MAPREDUCE-5554) | hdfs-site.xml included in hadoop-mapreduce-client-jobclient tests jar is breaking tests for downstream components | Minor | test | Robert Kanter | Robert Kanter | +| [MAPREDUCE-5545](https://issues.apache.org/jira/browse/MAPREDUCE-5545) | org.apache.hadoop.mapred.TestTaskAttemptListenerImpl.testCommitWindow times out | Major | . | Robert Kanter | Robert Kanter | +| [MAPREDUCE-5544](https://issues.apache.org/jira/browse/MAPREDUCE-5544) | JobClient#getJob loads job conf twice | Major | . | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-5536](https://issues.apache.org/jira/browse/MAPREDUCE-5536) | mapreduce.jobhistory.webapp.https.address property is not respected | Blocker | . | Yesha Vora | Omkar Vinit Joshi | +| [MAPREDUCE-5533](https://issues.apache.org/jira/browse/MAPREDUCE-5533) | Speculative execution does not function for reduce | Major | applicationmaster | Tassapol Athiapinya | Xuan Gong | +| [MAPREDUCE-5523](https://issues.apache.org/jira/browse/MAPREDUCE-5523) | Need to add https port related property in Job history server | Major | . | Omkar Vinit Joshi | Omkar Vinit Joshi | +| [MAPREDUCE-5515](https://issues.apache.org/jira/browse/MAPREDUCE-5515) | Application Manager UI does not appear with Https enabled | Major | . | Omkar Vinit Joshi | Omkar Vinit Joshi | +| [MAPREDUCE-5513](https://issues.apache.org/jira/browse/MAPREDUCE-5513) | ConcurrentModificationException in JobControl | Major | . | Jason Lowe | Robert Parker | +| [MAPREDUCE-5503](https://issues.apache.org/jira/browse/MAPREDUCE-5503) | TestMRJobClient.testJobClient is failing | Blocker | mrv2 | Jason Lowe | Jian He | +| [MAPREDUCE-5489](https://issues.apache.org/jira/browse/MAPREDUCE-5489) | MR jobs hangs as it does not use the node-blacklisting feature in RM requests | Critical | . | Yesha Vora | Zhijie Shen | +| [MAPREDUCE-5488](https://issues.apache.org/jira/browse/MAPREDUCE-5488) | Job recovery fails after killing all the running containers for the app | Major | . | Arpit Gupta | Jian He | +| [MAPREDUCE-5459](https://issues.apache.org/jira/browse/MAPREDUCE-5459) | Update the doc of running MRv1 examples jar on YARN | Major | . | Zhijie Shen | Zhijie Shen | +| [MAPREDUCE-5442](https://issues.apache.org/jira/browse/MAPREDUCE-5442) | $HADOOP\_MAPRED\_HOME/$HADOOP\_CONF\_DIR setting not working on Windows | Major | client | Yingda Chen | Yingda Chen | +| [MAPREDUCE-5170](https://issues.apache.org/jira/browse/MAPREDUCE-5170) | incorrect exception message if min node size \> min rack size | Trivial | mrv2 | Sangjin Lee | Sangjin Lee | +| [YARN-1278](https://issues.apache.org/jira/browse/YARN-1278) | New AM does not start after rm restart | Blocker | . | Yesha Vora | Hitesh Shah | +| [YARN-1274](https://issues.apache.org/jira/browse/YARN-1274) | LCE fails to run containers that don't have resources to localize | Blocker | nodemanager | Alejandro Abdelnur | Siddharth Seth | +| [YARN-1273](https://issues.apache.org/jira/browse/YARN-1273) | Distributed shell does not account for start container failures reported asynchronously. | Major | . | Hitesh Shah | Hitesh Shah | +| [YARN-1271](https://issues.apache.org/jira/browse/YARN-1271) | "Text file busy" errors launching containers again | Major | nodemanager | Sandy Ryza | Sandy Ryza | +| [YARN-1262](https://issues.apache.org/jira/browse/YARN-1262) | TestApplicationCleanup relies on all containers assigned in a single heartbeat | Major | . | Sandy Ryza | Karthik Kambatla | +| [YARN-1251](https://issues.apache.org/jira/browse/YARN-1251) | TestDistributedShell#TestDSShell failed with timeout | Major | applications/distributed-shell | Junping Du | Xuan Gong | +| [YARN-1247](https://issues.apache.org/jira/browse/YARN-1247) | test-container-executor has gotten out of sync with the changes to container-executor | Major | nodemanager | Roman Shaposhnik | Roman Shaposhnik | +| [YARN-1236](https://issues.apache.org/jira/browse/YARN-1236) | FairScheduler setting queue name in RMApp is not working | Major | resourcemanager | Sandy Ryza | Sandy Ryza | +| [YARN-1221](https://issues.apache.org/jira/browse/YARN-1221) | With Fair Scheduler, reserved MB reported in RM web UI increases indefinitely | Major | resourcemanager, scheduler | Sandy Ryza | Siqi Li | +| [YARN-1219](https://issues.apache.org/jira/browse/YARN-1219) | FSDownload changes file suffix making FileUtil.unTar() throw exception | Major | nodemanager | shanyu zhao | shanyu zhao | +| [YARN-1215](https://issues.apache.org/jira/browse/YARN-1215) | Yarn URL should include userinfo | Major | api | Chuan Liu | Chuan Liu | +| [YARN-1167](https://issues.apache.org/jira/browse/YARN-1167) | Submitted distributed shell application shows appMasterHost = empty | Major | applications/distributed-shell | Tassapol Athiapinya | Xuan Gong | +| [YARN-1157](https://issues.apache.org/jira/browse/YARN-1157) | ResourceManager UI has invalid tracking URL link for distributed shell application | Major | resourcemanager | Tassapol Athiapinya | Xuan Gong | +| [YARN-1149](https://issues.apache.org/jira/browse/YARN-1149) | NM throws InvalidStateTransitonException: Invalid event: APPLICATION\_LOG\_HANDLING\_FINISHED at RUNNING | Major | . | Ramya Sunil | Xuan Gong | +| [YARN-1141](https://issues.apache.org/jira/browse/YARN-1141) | Updating resource requests should be decoupled with updating blacklist | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-1128](https://issues.apache.org/jira/browse/YARN-1128) | FifoPolicy.computeShares throws NPE on empty list of Schedulables | Major | scheduler | Sandy Ryza | Karthik Kambatla | +| [YARN-1090](https://issues.apache.org/jira/browse/YARN-1090) | Job does not get into Pending State | Major | . | Yesha Vora | Jian He | +| [YARN-1032](https://issues.apache.org/jira/browse/YARN-1032) | NPE in RackResolve | Critical | . | Lohit Vijayarenu | Lohit Vijayarenu | +| [YARN-890](https://issues.apache.org/jira/browse/YARN-890) | The roundup for memory values on resource manager UI is misleading | Major | resourcemanager | Trupti Dhavle | Xuan Gong | +| [YARN-876](https://issues.apache.org/jira/browse/YARN-876) | Node resource is added twice when node comes back from unhealthy to healthy | Major | resourcemanager | Peng Zhang | Peng Zhang | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9948](https://issues.apache.org/jira/browse/HADOOP-9948) | Add a config value to CLITestHelper to skip tests on Windows | Minor | test | Chuan Liu | Chuan Liu | +| [HDFS-5186](https://issues.apache.org/jira/browse/HDFS-5186) | TestFileJournalManager fails on Windows due to file handle leaks | Minor | namenode, test | Chuan Liu | Chuan Liu | +| [MAPREDUCE-5525](https://issues.apache.org/jira/browse/MAPREDUCE-5525) | Increase timeout of TestDFSIO.testAppend and TestMRJobsWithHistoryService.testJobHistoryData | Minor | mrv2, test | Chuan Liu | Chuan Liu | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10017](https://issues.apache.org/jira/browse/HADOOP-10017) | Fix NPE in DFSClient#getDelegationToken when doing Distcp from a secured cluster to an insecured cluster | Major | . | Jing Zhao | Haohui Mai | +| [HDFS-5306](https://issues.apache.org/jira/browse/HDFS-5306) | Datanode https port is not available at the namenode | Major | datanode, namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-5259](https://issues.apache.org/jira/browse/HDFS-5259) | Support client which combines appended data with old data before sends it to NFS server | Major | nfs | Yesha Vora | Brandon Li | +| [HDFS-5246](https://issues.apache.org/jira/browse/HDFS-5246) | Make Hadoop nfs server port and mount daemon port configurable | Major | nfs | Jinghui Wang | Jinghui Wang | +| [HDFS-5230](https://issues.apache.org/jira/browse/HDFS-5230) | Introduce RpcInfo to decouple XDR classes from the RPC API | Major | nfs | Haohui Mai | Haohui Mai | +| [MAPREDUCE-5562](https://issues.apache.org/jira/browse/MAPREDUCE-5562) | MR AM should exit when unregister() throws exception | Major | . | Zhijie Shen | Zhijie Shen | +| [MAPREDUCE-5551](https://issues.apache.org/jira/browse/MAPREDUCE-5551) | Binary Incompatibility of O.A.H.U.mapred.SequenceFileAsBinaryOutputFormat.WritableValueBytes | Blocker | . | Zhijie Shen | Zhijie Shen | +| [MAPREDUCE-5538](https://issues.apache.org/jira/browse/MAPREDUCE-5538) | MRAppMaster#shutDownJob shouldn't send job end notification before checking isLastRetry | Blocker | . | Zhijie Shen | Zhijie Shen | +| [MAPREDUCE-5531](https://issues.apache.org/jira/browse/MAPREDUCE-5531) | Binary and source incompatibility in mapreduce.TaskID and mapreduce.TaskAttemptID between branch-1 and branch-2 | Blocker | mrv1, mrv2 | Robert Kanter | Robert Kanter | +| [MAPREDUCE-5530](https://issues.apache.org/jira/browse/MAPREDUCE-5530) | Binary and source incompatibility in mapred.lib.CombineFileInputFormat between branch-1 and branch-2 | Blocker | mrv1, mrv2 | Robert Kanter | Robert Kanter | +| [MAPREDUCE-5529](https://issues.apache.org/jira/browse/MAPREDUCE-5529) | Binary incompatibilities in mapred.lib.TotalOrderPartitioner between branch-1 and branch-2 | Blocker | mrv1, mrv2 | Robert Kanter | Robert Kanter | +| [MAPREDUCE-5505](https://issues.apache.org/jira/browse/MAPREDUCE-5505) | Clients should be notified job finished only after job successfully unregistered | Critical | . | Jian He | Zhijie Shen | +| [YARN-1277](https://issues.apache.org/jira/browse/YARN-1277) | Add http policy support for YARN daemons | Major | . | Suresh Srinivas | Omkar Vinit Joshi | +| [YARN-1260](https://issues.apache.org/jira/browse/YARN-1260) | RM\_HOME link breaks when webapp.https.address related properties are not specified | Major | . | Yesha Vora | Omkar Vinit Joshi | +| [YARN-1256](https://issues.apache.org/jira/browse/YARN-1256) | NM silently ignores non-existent service in StartContainerRequest | Critical | . | Bikas Saha | Xuan Gong | +| [YARN-1254](https://issues.apache.org/jira/browse/YARN-1254) | NM is polluting container's credentials | Major | . | Vinod Kumar Vavilapalli | Omkar Vinit Joshi | +| [YARN-1214](https://issues.apache.org/jira/browse/YARN-1214) | Register ClientToken MasterKey in SecretManager after it is saved | Critical | resourcemanager | Jian He | Jian He | +| [YARN-1204](https://issues.apache.org/jira/browse/YARN-1204) | Need to add https port related property in Yarn | Major | . | Yesha Vora | Omkar Vinit Joshi | +| [YARN-1203](https://issues.apache.org/jira/browse/YARN-1203) | Application Manager UI does not appear with Https enabled | Major | . | Yesha Vora | Omkar Vinit Joshi | +| [YARN-1131](https://issues.apache.org/jira/browse/YARN-1131) | $yarn logs command should return an appropriate error message if YARN application is still running | Minor | client | Tassapol Athiapinya | Siddharth Seth | +| [YARN-1070](https://issues.apache.org/jira/browse/YARN-1070) | ContainerImpl State Machine: Invalid event: CONTAINER\_KILLED\_ON\_REQUEST at CONTAINER\_CLEANEDUP\_AFTER\_KILL | Major | nodemanager | Hitesh Shah | Zhijie Shen | +| [YARN-899](https://issues.apache.org/jira/browse/YARN-899) | Get queue administration ACLs working | Major | scheduler | Sandy Ryza | Xuan Gong | +| [YARN-621](https://issues.apache.org/jira/browse/YARN-621) | RM triggers web auth failure before first job | Critical | resourcemanager | Allen Wittenauer | Omkar Vinit Joshi | +| [YARN-49](https://issues.apache.org/jira/browse/YARN-49) | Improve distributed shell application to work on a secure cluster | Major | applications/distributed-shell | Hitesh Shah | Vinod Kumar Vavilapalli | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + http://git-wip-us.apache.org/repos/asf/hadoop/blob/d759b4bd/hadoop-common-project/hadoop-common/src/site/markdown/release/2.2.0/RELEASENOTES.2.2.0.md ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.2.0/RELEASENOTES.2.2.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.2.0/RELEASENOTES.2.2.0.md new file mode 100644 index 0000000..301e0c6 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.2.0/RELEASENOTES.2.2.0.md @@ -0,0 +1,45 @@ + +<!--- +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +--> +# Apache Hadoop 2.2.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-10020](https://issues.apache.org/jira/browse/HADOOP-10020) | *Blocker* | **disable symlinks temporarily** + +During review of symbolic links, many issues were found related to the impact on semantics of existing APIs such FileSystem#listStatus, FileSystem#globStatus etc. There were also many issues brought up about symbolic links, and the impact on security and functionality of HDFS. All these issues will be addressed in the upcoming release 2.3. Until then the feature is temporarily disabled. + + +--- + +* [YARN-1229](https://issues.apache.org/jira/browse/YARN-1229) | *Blocker* | **Define constraints on Auxiliary Service names. Change ShuffleHandler service name from mapreduce.shuffle to mapreduce\_shuffle.** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-1228](https://issues.apache.org/jira/browse/YARN-1228) | *Major* | **Clean up Fair Scheduler configuration loading** + +**WARNING: No release note provided for this incompatible change.** + + +
