http://git-wip-us.apache.org/repos/asf/hadoop/blob/19041008/hadoop-common-project/hadoop-common/src/site/markdown/release/2.4.0/RELEASENOTES.2.4.0.md ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.4.0/RELEASENOTES.2.4.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.4.0/RELEASENOTES.2.4.0.md index a86f1e0..ea93496 100644 --- a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.4.0/RELEASENOTES.2.4.0.md +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.4.0/RELEASENOTES.2.4.0.md @@ -23,102 +23,102 @@ These release notes cover new developer and user-facing incompatibilities, impor --- -* [HADOOP-10295](https://issues.apache.org/jira/browse/HADOOP-10295) | *Major* | **Allow distcp to automatically identify the checksum type of source files and use it for the target** +* [HDFS-5790](https://issues.apache.org/jira/browse/HDFS-5790) | *Major* | **LeaseManager.findPath is very slow when many leases need recovery** -Add option for distcp to preserve the checksum type of the source files. Users can use "-pc" as distcp command option to preserve the checksum type. +Committed to branch-2 and trunk. --- -* [HADOOP-10221](https://issues.apache.org/jira/browse/HADOOP-10221) | *Major* | **Add a plugin to specify SaslProperties for RPC protocol based on connection properties** - -SaslPropertiesResolver or its subclass is used to resolve the QOP used for a connection. The subclass can be specified via "hadoop.security.saslproperties.resolver.class" configuration property. If not specified, the full set of values specified in hadoop.rpc.protection is used while determining the QOP used for the connection. If a class is specified, then the QOP values returned by the class will be used while determining the QOP used for the connection. +* [HADOOP-10295](https://issues.apache.org/jira/browse/HADOOP-10295) | *Major* | **Allow distcp to automatically identify the checksum type of source files and use it for the target** -Note that this change, effectively removes SaslRpcServer.SASL\_PROPS which was a public field. Any use of this variable should be replaced with the following code: -SaslPropertiesResolver saslPropsResolver = SaslPropertiesResolver.getInstance(conf); -Map\<String, String\> sasl\_props = saslPropsResolver.getDefaultProperties(); +Add option for distcp to preserve the checksum type of the source files. Users can use "-pc" as distcp command option to preserve the checksum type. --- -* [HADOOP-10211](https://issues.apache.org/jira/browse/HADOOP-10211) | *Major* | **Enable RPC protocol to negotiate SASL-QOP values between clients and servers** +* [HDFS-5804](https://issues.apache.org/jira/browse/HDFS-5804) | *Major* | **HDFS NFS Gateway fails to mount and proxy when using Kerberos** -The hadoop.rpc.protection configuration property previously supported specifying a single value: one of authentication, integrity or privacy. An unrecognized value was silently assumed to mean authentication. This configuration property now accepts a comma-separated list of any of the 3 values, and unrecognized values are rejected with an error. Existing configurations containing an invalid value must be corrected. If the property is empty or not specified, authentication is assumed. +Fixes NFS on Kerberized cluster. --- -* [HADOOP-8691](https://issues.apache.org/jira/browse/HADOOP-8691) | *Minor* | **FsShell can print "Found xxx items" unnecessarily often** +* [HDFS-5698](https://issues.apache.org/jira/browse/HDFS-5698) | *Major* | **Use protobuf to serialize / deserialize FSImage** -The `ls` command only prints "Found foo items" once when listing the directories recursively. +Use protobuf to serialize/deserialize the FSImage. --- -* [HDFS-6102](https://issues.apache.org/jira/browse/HDFS-6102) | *Blocker* | **Lower the default maximum items per directory to fix PB fsimage loading** +* [HDFS-4370](https://issues.apache.org/jira/browse/HDFS-4370) | *Major* | **Fix typo Blanacer in DataNode** -**WARNING: No release note provided for this incompatible change.** +I just committed this. Thank you Chu. --- -* [HDFS-6055](https://issues.apache.org/jira/browse/HDFS-6055) | *Major* | **Change default configuration to limit file name length in HDFS** +* [HDFS-5776](https://issues.apache.org/jira/browse/HDFS-5776) | *Major* | **Support 'hedged' reads in DFSClient** -The default configuration of HDFS now sets dfs.namenode.fs-limits.max-component-length to 255 for improved interoperability with other file system implementations. This limits each component of a file system path to a maximum of 255 bytes in UTF-8 encoding. Attempts to create new files that violate this rule will fail with an error. Existing files that violate the rule are not effected. Previously, dfs.namenode.fs-limits.max-component-length was set to 0 (ignored). If necessary, it is possible to set the value back to 0 in the cluster's configuration to restore the old behavior. +If a read from a block is slow, start up another parallel, 'hedged' read against a different block replica. We then take the result of which ever read returns first (the outstanding read is cancelled). This 'hedged' read feature will help rein in the outliers, the odd read that takes a long time because it hit a bad patch on the disc, etc. +This feature is off by default. To enable this feature, set \<code\>dfs.client.hedged.read.threadpool.size\</code\> to a positive number. The threadpool size is how many threads to dedicate to the running of these 'hedged', concurrent reads in your client. ---- +Then set \<code\>dfs.client.hedged.read.threshold.millis\</code\> to the number of milliseconds to wait before starting up a 'hedged' read. For example, if you set this property to 10, then if a read has not returned within 10 milliseconds, we will start up a new read against a different block replica. -* [HDFS-5804](https://issues.apache.org/jira/browse/HDFS-5804) | *Major* | **HDFS NFS Gateway fails to mount and proxy when using Kerberos** +This feature emits new metrics: -Fixes NFS on Kerberized cluster. ++ hedgedReadOps ++ hedgeReadOpsWin -- how many times the hedged read 'beat' the original read ++ hedgedReadOpsInCurThread -- how many times we went to do a hedged read but we had to run it in the current thread because dfs.client.hedged.read.threadpool.size was at a maximum. --- -* [HDFS-5790](https://issues.apache.org/jira/browse/HDFS-5790) | *Major* | **LeaseManager.findPath is very slow when many leases need recovery** +* [HADOOP-8691](https://issues.apache.org/jira/browse/HADOOP-8691) | *Minor* | **FsShell can print "Found xxx items" unnecessarily often** -Committed to branch-2 and trunk. +The \`ls\` command only prints "Found foo items" once when listing the directories recursively. --- -* [HDFS-5776](https://issues.apache.org/jira/browse/HDFS-5776) | *Major* | **Support 'hedged' reads in DFSClient** +* [HDFS-5321](https://issues.apache.org/jira/browse/HDFS-5321) | *Major* | **Clean up the HTTP-related configuration in HDFS** -If a read from a block is slow, start up another parallel, 'hedged' read against a different block replica. We then take the result of which ever read returns first (the outstanding read is cancelled). This 'hedged' read feature will help rein in the outliers, the odd read that takes a long time because it hit a bad patch on the disc, etc. +dfs.http.port and dfs.https.port are removed. Filesystem clients, such as WebHdfsFileSystem, now have fixed instead of configurable default ports (i.e., 50070 for http and 50470 for https). -This feature is off by default. To enable this feature, set \<code\>dfs.client.hedged.read.threadpool.size\</code\> to a positive number. The threadpool size is how many threads to dedicate to the running of these 'hedged', concurrent reads in your client. +Users can explicitly specify the port in the URI to access the file system which runs on non-default ports. -Then set \<code\>dfs.client.hedged.read.threshold.millis\</code\> to the number of milliseconds to wait before starting up a 'hedged' read. For example, if you set this property to 10, then if a read has not returned within 10 milliseconds, we will start up a new read against a different block replica. -This feature emits new metrics: +--- -+ hedgedReadOps -+ hedgeReadOpsWin -- how many times the hedged read 'beat' the original read -+ hedgedReadOpsInCurThread -- how many times we went to do a hedged read but we had to run it in the current thread because dfs.client.hedged.read.threadpool.size was at a maximum. +* [HADOOP-10211](https://issues.apache.org/jira/browse/HADOOP-10211) | *Major* | **Enable RPC protocol to negotiate SASL-QOP values between clients and servers** + +The hadoop.rpc.protection configuration property previously supported specifying a single value: one of authentication, integrity or privacy. An unrecognized value was silently assumed to mean authentication. This configuration property now accepts a comma-separated list of any of the 3 values, and unrecognized values are rejected with an error. Existing configurations containing an invalid value must be corrected. If the property is empty or not specified, authentication is assumed. --- -* [HDFS-5698](https://issues.apache.org/jira/browse/HDFS-5698) | *Major* | **Use protobuf to serialize / deserialize FSImage** +* [HDFS-6055](https://issues.apache.org/jira/browse/HDFS-6055) | *Major* | **Change default configuration to limit file name length in HDFS** -Use protobuf to serialize/deserialize the FSImage. +The default configuration of HDFS now sets dfs.namenode.fs-limits.max-component-length to 255 for improved interoperability with other file system implementations. This limits each component of a file system path to a maximum of 255 bytes in UTF-8 encoding. Attempts to create new files that violate this rule will fail with an error. Existing files that violate the rule are not effected. Previously, dfs.namenode.fs-limits.max-component-length was set to 0 (ignored). If necessary, it is possible to set the value back to 0 in the cluster's configuration to restore the old behavior. --- -* [HDFS-5321](https://issues.apache.org/jira/browse/HDFS-5321) | *Major* | **Clean up the HTTP-related configuration in HDFS** - -dfs.http.port and dfs.https.port are removed. Filesystem clients, such as WebHdfsFileSystem, now have fixed instead of configurable default ports (i.e., 50070 for http and 50470 for https). +* [HDFS-6102](https://issues.apache.org/jira/browse/HDFS-6102) | *Blocker* | **Lower the default maximum items per directory to fix PB fsimage loading** -Users can explicitly specify the port in the URI to access the file system which runs on non-default ports. +**WARNING: No release note provided for this change.** --- -* [HDFS-5138](https://issues.apache.org/jira/browse/HDFS-5138) | *Blocker* | **Support HDFS upgrade in HA** +* [HADOOP-10221](https://issues.apache.org/jira/browse/HADOOP-10221) | *Major* | **Add a plugin to specify SaslProperties for RPC protocol based on connection properties** -**WARNING: No release note provided for this incompatible change.** +SaslPropertiesResolver or its subclass is used to resolve the QOP used for a connection. The subclass can be specified via "hadoop.security.saslproperties.resolver.class" configuration property. If not specified, the full set of values specified in hadoop.rpc.protection is used while determining the QOP used for the connection. If a class is specified, then the QOP values returned by the class will be used while determining the QOP used for the connection. + +Note that this change, effectively removes SaslRpcServer.SASL\_PROPS which was a public field. Any use of this variable should be replaced with the following code: +SaslPropertiesResolver saslPropsResolver = SaslPropertiesResolver.getInstance(conf); +Map\<String, String\> sasl\_props = saslPropsResolver.getDefaultProperties(); --- @@ -130,16 +130,16 @@ HDFS now supports ACLs (Access Control Lists). ACLs can specify fine-grained fi --- -* [HDFS-4370](https://issues.apache.org/jira/browse/HDFS-4370) | *Major* | **Fix typo Blanacer in DataNode** +* [HDFS-5138](https://issues.apache.org/jira/browse/HDFS-5138) | *Blocker* | **Support HDFS upgrade in HA** -I just committed this. Thank you Chu. +**WARNING: No release note provided for this change.** --- * [MAPREDUCE-5036](https://issues.apache.org/jira/browse/MAPREDUCE-5036) | *Major* | **Default shuffle handler port should not be 8080** -**WARNING: No release note provided for this incompatible change.** +**WARNING: No release note provided for this change.**
http://git-wip-us.apache.org/repos/asf/hadoop/blob/19041008/hadoop-common-project/hadoop-common/src/site/markdown/release/2.4.1/CHANGES.2.4.1.md ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.4.1/CHANGES.2.4.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.4.1/CHANGES.2.4.1.md index 9ad1697..c5cd5d7 100644 --- a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.4.1/CHANGES.2.4.1.md +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.4.1/CHANGES.2.4.1.md @@ -20,30 +20,14 @@ ## Release 2.4.1 - 2014-06-30 -### INCOMPATIBLE CHANGES: - -| JIRA | Summary | Priority | Component | Reporter | Contributor | -|:---- |:---- | :--- |:---- |:---- |:---- | - - -### IMPORTANT ISSUES: - -| JIRA | Summary | Priority | Component | Reporter | Contributor | -|:---- |:---- | :--- |:---- |:---- |:---- | - - -### NEW FEATURES: - -| JIRA | Summary | Priority | Component | Reporter | Contributor | -|:---- |:---- | :--- |:---- |:---- |:---- | ### IMPROVEMENTS: | JIRA | Summary | Priority | Component | Reporter | Contributor | |:---- |:---- | :--- |:---- |:---- |:---- | -| [HADOOP-10466](https://issues.apache.org/jira/browse/HADOOP-10466) | Lower the log level in UserGroupInformation | Minor | security | Nicolas Liochon | Nicolas Liochon | | [HDFS-4052](https://issues.apache.org/jira/browse/HDFS-4052) | BlockManager#invalidateWork should print logs outside the lock | Minor | . | Jing Zhao | Jing Zhao | +| [HADOOP-10466](https://issues.apache.org/jira/browse/HADOOP-10466) | Lower the log level in UserGroupInformation | Minor | security | Nicolas Liochon | Nicolas Liochon | | [YARN-1892](https://issues.apache.org/jira/browse/YARN-1892) | Excessive logging in RM | Minor | scheduler | Siddharth Seth | Jian He | @@ -51,77 +35,77 @@ | JIRA | Summary | Priority | Component | Reporter | Contributor | |:---- |:---- | :--- |:---- |:---- |:---- | -| [HADOOP-11273](https://issues.apache.org/jira/browse/HADOOP-11273) | TestMiniKdc failure: login options not compatible with IBM JDK | Major | test | Gao Zhong Liang | Gao Zhong Liang | -| [HADOOP-10612](https://issues.apache.org/jira/browse/HADOOP-10612) | NFS failed to refresh the user group id mapping table | Major | nfs | Brandon Li | Brandon Li | -| [HADOOP-10562](https://issues.apache.org/jira/browse/HADOOP-10562) | Namenode exits on exception without printing stack trace in AbstractDelegationTokenSecretManager | Critical | . | Suresh Srinivas | Suresh Srinivas | -| [HADOOP-10527](https://issues.apache.org/jira/browse/HADOOP-10527) | Fix incorrect return code and allow more retries on EINTR | Major | . | Kihwal Lee | Kihwal Lee | -| [HADOOP-10522](https://issues.apache.org/jira/browse/HADOOP-10522) | JniBasedUnixGroupMapping mishandles errors | Critical | . | Kihwal Lee | Kihwal Lee | -| [HADOOP-10490](https://issues.apache.org/jira/browse/HADOOP-10490) | TestMapFile and TestBloomMapFile leak file descriptors. | Minor | test | Chris Nauroth | Chris Nauroth | -| [HADOOP-10473](https://issues.apache.org/jira/browse/HADOOP-10473) | TestCallQueueManager is still flaky | Minor | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | -| [HADOOP-10456](https://issues.apache.org/jira/browse/HADOOP-10456) | Bug in Configuration.java exposed by Spark (ConcurrentModificationException) | Major | conf | Nishkam Ravi | Nishkam Ravi | +| [YARN-1883](https://issues.apache.org/jira/browse/YARN-1883) | TestRMAdminService fails due to inconsistent entries in UserGroups | Major | . | Mit Desai | Mit Desai | | [HADOOP-10455](https://issues.apache.org/jira/browse/HADOOP-10455) | When there is an exception, ipc.Server should first check whether it is an terse exception | Major | ipc | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | -| [HADOOP-8826](https://issues.apache.org/jira/browse/HADOOP-8826) | Docs still refer to 0.20.205 as stable line | Minor | . | Robert Joseph Evans | Mit Desai | -| [HDFS-6411](https://issues.apache.org/jira/browse/HDFS-6411) | nfs-hdfs-gateway mount raises I/O error and hangs when a unauthorized user attempts to access it | Major | nfs | Zhongyi Xie | Brandon Li | -| [HDFS-6402](https://issues.apache.org/jira/browse/HDFS-6402) | Suppress findbugs warning for failure to override equals and hashCode in FsAclPermission. | Trivial | namenode | Chris Nauroth | Chris Nauroth | -| [HDFS-6397](https://issues.apache.org/jira/browse/HDFS-6397) | NN shows inconsistent value in deadnode count | Critical | . | Mohammad Kamrul Islam | Mohammad Kamrul Islam | -| [HDFS-6362](https://issues.apache.org/jira/browse/HDFS-6362) | InvalidateBlocks is inconsistent in usage of DatanodeUuid and StorageID | Blocker | namenode | Arpit Agarwal | Arpit Agarwal | -| [HDFS-6361](https://issues.apache.org/jira/browse/HDFS-6361) | TestIdUserGroup.testUserUpdateSetting failed due to out of range nfsnobody Id | Major | nfs | Yongjun Zhang | Yongjun Zhang | -| [HDFS-6340](https://issues.apache.org/jira/browse/HDFS-6340) | DN can't finalize upgrade | Blocker | datanode | Rahul Singhal | Rahul Singhal | -| [HDFS-6329](https://issues.apache.org/jira/browse/HDFS-6329) | WebHdfs does not work if HA is enabled on NN but logical URI is not configured. | Blocker | . | Kihwal Lee | Kihwal Lee | -| [HDFS-6326](https://issues.apache.org/jira/browse/HDFS-6326) | WebHdfs ACL compatibility is broken | Blocker | webhdfs | Daryn Sharp | Chris Nauroth | -| [HDFS-6325](https://issues.apache.org/jira/browse/HDFS-6325) | Append should fail if the last block has insufficient number of replicas | Major | namenode | Konstantin Shvachko | Keith Pak | -| [HDFS-6313](https://issues.apache.org/jira/browse/HDFS-6313) | WebHdfs may use the wrong NN when configured for multiple HA NNs | Blocker | webhdfs | Daryn Sharp | Kihwal Lee | -| [HDFS-6245](https://issues.apache.org/jira/browse/HDFS-6245) | datanode fails to start with a bad disk even when failed volumes is set | Major | . | Arpit Gupta | Arpit Agarwal | -| [HDFS-6236](https://issues.apache.org/jira/browse/HDFS-6236) | ImageServlet should use Time#monotonicNow to measure latency. | Minor | namenode | Chris Nauroth | Chris Nauroth | -| [HDFS-6235](https://issues.apache.org/jira/browse/HDFS-6235) | TestFileJournalManager can fail on Windows due to file locking if tests run out of order. | Trivial | namenode, test | Chris Nauroth | Chris Nauroth | -| [HDFS-6234](https://issues.apache.org/jira/browse/HDFS-6234) | TestDatanodeConfig#testMemlockLimit fails on Windows due to invalid file path. | Trivial | datanode, test | Chris Nauroth | Chris Nauroth | -| [HDFS-6232](https://issues.apache.org/jira/browse/HDFS-6232) | OfflineEditsViewer throws a NPE on edits containing ACL modifications | Major | tools | Stephen Chu | Akira AJISAKA | -| [HDFS-6231](https://issues.apache.org/jira/browse/HDFS-6231) | DFSClient hangs infinitely if using hedged reads and all eligible datanodes die. | Major | hdfs-client | Chris Nauroth | Chris Nauroth | -| [HDFS-6229](https://issues.apache.org/jira/browse/HDFS-6229) | Race condition in failover can cause RetryCache fail to work | Major | ha | Jing Zhao | Jing Zhao | +| [MAPREDUCE-5818](https://issues.apache.org/jira/browse/MAPREDUCE-5818) | hsadmin cmd is missing in mapred.cmd | Major | . | Jian He | Jian He | +| [HADOOP-10456](https://issues.apache.org/jira/browse/HADOOP-10456) | Bug in Configuration.java exposed by Spark (ConcurrentModificationException) | Major | conf | Nishkam Ravi | Nishkam Ravi | +| [YARN-1837](https://issues.apache.org/jira/browse/YARN-1837) | TestMoveApplication.testMoveRejectedByScheduler randomly fails | Major | . | Tsuyoshi Ozawa | Hong Zhiguo | +| [HDFS-6197](https://issues.apache.org/jira/browse/HDFS-6197) | Rolling upgrade rollback on Windows can fail attempting to rename edit log segment files to a destination that already exists. | Minor | namenode | Chris Nauroth | Chris Nauroth | +| [HDFS-6198](https://issues.apache.org/jira/browse/HDFS-6198) | DataNode rolling upgrade does not correctly identify current block pool directory and replace with trash on Windows. | Major | datanode | Chris Nauroth | Chris Nauroth | +| [YARN-1908](https://issues.apache.org/jira/browse/YARN-1908) | Distributed shell with custom script has permission error. | Major | applications/distributed-shell | Tassapol Athiapinya | Vinod Kumar Vavilapalli | +| [YARN-1907](https://issues.apache.org/jira/browse/YARN-1907) | TestRMApplicationHistoryWriter#testRMWritingMassiveHistory runs slow and intermittently fails | Major | . | Mit Desai | Mit Desai | | [HDFS-6215](https://issues.apache.org/jira/browse/HDFS-6215) | Wrong error message for upgrade | Minor | . | Kihwal Lee | Kihwal Lee | -| [HDFS-6209](https://issues.apache.org/jira/browse/HDFS-6209) | Fix flaky test TestValidateConfigurationSettings.testThatDifferentRPCandHttpPortsAreOK | Minor | test | Arpit Agarwal | Arpit Agarwal | -| [HDFS-6208](https://issues.apache.org/jira/browse/HDFS-6208) | DataNode caching can leak file descriptors. | Major | datanode | Chris Nauroth | Chris Nauroth | +| [HADOOP-10473](https://issues.apache.org/jira/browse/HADOOP-10473) | TestCallQueueManager is still flaky | Minor | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | [HDFS-6206](https://issues.apache.org/jira/browse/HDFS-6206) | DFSUtil.substituteForWildcardAddress may throw NPE | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | | [HDFS-6204](https://issues.apache.org/jira/browse/HDFS-6204) | TestRBWBlockInvalidation may fail | Minor | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | -| [HDFS-6198](https://issues.apache.org/jira/browse/HDFS-6198) | DataNode rolling upgrade does not correctly identify current block pool directory and replace with trash on Windows. | Major | datanode | Chris Nauroth | Chris Nauroth | -| [HDFS-6197](https://issues.apache.org/jira/browse/HDFS-6197) | Rolling upgrade rollback on Windows can fail attempting to rename edit log segment files to a destination that already exists. | Minor | namenode | Chris Nauroth | Chris Nauroth | -| [HDFS-2882](https://issues.apache.org/jira/browse/HDFS-2882) | DN continues to start up, even if block pool fails to initialize | Major | datanode | Todd Lipcon | Vinayakumar B | -| [MAPREDUCE-5841](https://issues.apache.org/jira/browse/MAPREDUCE-5841) | uber job doesn't terminate on getting mapred job kill | Major | mrv2 | Sangjin Lee | Sangjin Lee | -| [MAPREDUCE-5835](https://issues.apache.org/jira/browse/MAPREDUCE-5835) | Killing Task might cause the job to go to ERROR state | Critical | . | Ming Ma | Ming Ma | -| [MAPREDUCE-5832](https://issues.apache.org/jira/browse/MAPREDUCE-5832) | Few tests in TestJobClient fail on Windows | Major | . | Jian He | Vinod Kumar Vavilapalli | -| [MAPREDUCE-5830](https://issues.apache.org/jira/browse/MAPREDUCE-5830) | HostUtil.getTaskLogUrl is not backwards binary compatible with 2.3 | Blocker | . | Jason Lowe | Akira AJISAKA | -| [MAPREDUCE-5828](https://issues.apache.org/jira/browse/MAPREDUCE-5828) | TestMapReduceJobControl fails on JDK 7 + Windows | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | -| [MAPREDUCE-5827](https://issues.apache.org/jira/browse/MAPREDUCE-5827) | TestSpeculativeExecutionWithMRApp fails | Major | . | Zhijie Shen | Zhijie Shen | -| [MAPREDUCE-5826](https://issues.apache.org/jira/browse/MAPREDUCE-5826) | TestHistoryServerFileSystemStateStoreService.testTokenStore fails in windows | Major | . | Varun Vasudev | Varun Vasudev | +| [HDFS-6209](https://issues.apache.org/jira/browse/HDFS-6209) | Fix flaky test TestValidateConfigurationSettings.testThatDifferentRPCandHttpPortsAreOK | Minor | test | Arpit Agarwal | Arpit Agarwal | +| [HDFS-6208](https://issues.apache.org/jira/browse/HDFS-6208) | DataNode caching can leak file descriptors. | Major | datanode | Chris Nauroth | Chris Nauroth | +| [YARN-1910](https://issues.apache.org/jira/browse/YARN-1910) | TestAMRMTokens fails on windows | Major | . | Xuan Gong | Xuan Gong | | [MAPREDUCE-5824](https://issues.apache.org/jira/browse/MAPREDUCE-5824) | TestPipesNonJavaInputFormat.testFormat fails in windows | Major | . | Xuan Gong | Xuan Gong | -| [MAPREDUCE-5821](https://issues.apache.org/jira/browse/MAPREDUCE-5821) | IFile merge allocates new byte array for every value | Major | performance, task | Todd Lipcon | Todd Lipcon | -| [MAPREDUCE-5818](https://issues.apache.org/jira/browse/MAPREDUCE-5818) | hsadmin cmd is missing in mapred.cmd | Major | . | Jian He | Jian He | -| [MAPREDUCE-5815](https://issues.apache.org/jira/browse/MAPREDUCE-5815) | Fix NPE in TestMRAppMaster | Blocker | client, mrv2 | Gera Shegalov | Akira AJISAKA | -| [MAPREDUCE-5714](https://issues.apache.org/jira/browse/MAPREDUCE-5714) | TestMRAppComponentDependencies causes surefire to exit without saying proper goodbye | Major | test | Jinghui Wang | Jinghui Wang | -| [MAPREDUCE-3191](https://issues.apache.org/jira/browse/MAPREDUCE-3191) | docs for map output compression incorrectly reference SequenceFile | Trivial | . | Todd Lipcon | Chen He | -| [YARN-2081](https://issues.apache.org/jira/browse/YARN-2081) | TestDistributedShell fails after YARN-1962 | Minor | applications/distributed-shell | Hong Zhiguo | Hong Zhiguo | -| [YARN-2066](https://issues.apache.org/jira/browse/YARN-2066) | Wrong field is referenced in GetApplicationsRequestPBImpl#mergeLocalToBuilder() | Minor | . | Ted Yu | Hong Zhiguo | -| [YARN-2016](https://issues.apache.org/jira/browse/YARN-2016) | Yarn getApplicationRequest start time range is not honored | Major | resourcemanager | Venkat Ranganathan | Junping Du | -| [YARN-1986](https://issues.apache.org/jira/browse/YARN-1986) | In Fifo Scheduler, node heartbeat in between creating app and attempt causes NPE | Critical | . | Jon Bringhurst | Hong Zhiguo | -| [YARN-1976](https://issues.apache.org/jira/browse/YARN-1976) | Tracking url missing http protocol for FAILED application | Major | . | Yesha Vora | Junping Du | -| [YARN-1975](https://issues.apache.org/jira/browse/YARN-1975) | Used resources shows escaped html in CapacityScheduler and FairScheduler page | Major | resourcemanager | Nathan Roberts | Mit Desai | -| [YARN-1934](https://issues.apache.org/jira/browse/YARN-1934) | Potential NPE in ZKRMStateStore caused by handling Disconnected event from ZK. | Blocker | resourcemanager | Rohith Sharma K S | Karthik Kambatla | -| [YARN-1933](https://issues.apache.org/jira/browse/YARN-1933) | TestAMRestart and TestNodeHealthService failing sometimes on Windows | Major | . | Jian He | Jian He | -| [YARN-1932](https://issues.apache.org/jira/browse/YARN-1932) | Javascript injection on the job status page | Blocker | . | Mit Desai | Mit Desai | -| [YARN-1931](https://issues.apache.org/jira/browse/YARN-1931) | Private API change in YARN-1824 in 2.4 broke compatibility with previous releases | Blocker | applications | Thomas Graves | Sandy Ryza | -| [YARN-1929](https://issues.apache.org/jira/browse/YARN-1929) | DeadLock in RM when automatic failover is enabled. | Blocker | resourcemanager | Rohith Sharma K S | Karthik Kambatla | -| [YARN-1928](https://issues.apache.org/jira/browse/YARN-1928) | TestAMRMRPCNodeUpdates fails ocassionally | Major | . | Zhijie Shen | Zhijie Shen | -| [YARN-1926](https://issues.apache.org/jira/browse/YARN-1926) | DistributedShell unit tests fail on Windows | Major | . | Varun Vasudev | Varun Vasudev | -| [YARN-1924](https://issues.apache.org/jira/browse/YARN-1924) | STATE\_STORE\_OP\_FAILED happens when ZKRMStateStore tries to update app(attempt) before storing it | Critical | . | Arpit Gupta | Jian He | | [YARN-1920](https://issues.apache.org/jira/browse/YARN-1920) | TestFileSystemApplicationHistoryStore.testMissingApplicationAttemptHistoryData fails in windows | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | | [YARN-1914](https://issues.apache.org/jira/browse/YARN-1914) | Test TestFSDownload.testDownloadPublicWithStatCache fails on Windows | Major | . | Varun Vasudev | Varun Vasudev | -| [YARN-1910](https://issues.apache.org/jira/browse/YARN-1910) | TestAMRMTokens fails on windows | Major | . | Xuan Gong | Xuan Gong | -| [YARN-1908](https://issues.apache.org/jira/browse/YARN-1908) | Distributed shell with custom script has permission error. | Major | applications/distributed-shell | Tassapol Athiapinya | Vinod Kumar Vavilapalli | -| [YARN-1907](https://issues.apache.org/jira/browse/YARN-1907) | TestRMApplicationHistoryWriter#testRMWritingMassiveHistory runs slow and intermittently fails | Major | . | Mit Desai | Mit Desai | | [YARN-1903](https://issues.apache.org/jira/browse/YARN-1903) | Killing Container on NEW and LOCALIZING will result in exitCode and diagnostics not set | Major | . | Zhijie Shen | Zhijie Shen | -| [YARN-1883](https://issues.apache.org/jira/browse/YARN-1883) | TestRMAdminService fails due to inconsistent entries in UserGroups | Major | . | Mit Desai | Mit Desai | -| [YARN-1837](https://issues.apache.org/jira/browse/YARN-1837) | TestMoveApplication.testMoveRejectedByScheduler randomly fails | Major | . | Tsuyoshi Ozawa | Hong Zhiguo | +| [YARN-1924](https://issues.apache.org/jira/browse/YARN-1924) | STATE\_STORE\_OP\_FAILED happens when ZKRMStateStore tries to update app(attempt) before storing it | Critical | . | Arpit Gupta | Jian He | +| [MAPREDUCE-5815](https://issues.apache.org/jira/browse/MAPREDUCE-5815) | Fix NPE in TestMRAppMaster | Blocker | client, mrv2 | Gera Shegalov | Akira Ajisaka | +| [YARN-1926](https://issues.apache.org/jira/browse/YARN-1926) | DistributedShell unit tests fail on Windows | Major | . | Varun Vasudev | Varun Vasudev | +| [HDFS-6231](https://issues.apache.org/jira/browse/HDFS-6231) | DFSClient hangs infinitely if using hedged reads and all eligible datanodes die. | Major | hdfs-client | Chris Nauroth | Chris Nauroth | +| [MAPREDUCE-5826](https://issues.apache.org/jira/browse/MAPREDUCE-5826) | TestHistoryServerFileSystemStateStoreService.testTokenStore fails in windows | Major | . | Varun Vasudev | Varun Vasudev | +| [HADOOP-10490](https://issues.apache.org/jira/browse/HADOOP-10490) | TestMapFile and TestBloomMapFile leak file descriptors. | Minor | test | Chris Nauroth | Chris Nauroth | +| [HDFS-6234](https://issues.apache.org/jira/browse/HDFS-6234) | TestDatanodeConfig#testMemlockLimit fails on Windows due to invalid file path. | Trivial | datanode, test | Chris Nauroth | Chris Nauroth | +| [HADOOP-8826](https://issues.apache.org/jira/browse/HADOOP-8826) | Docs still refer to 0.20.205 as stable line | Minor | . | Robert Joseph Evans | Mit Desai | +| [HDFS-6235](https://issues.apache.org/jira/browse/HDFS-6235) | TestFileJournalManager can fail on Windows due to file locking if tests run out of order. | Trivial | namenode, test | Chris Nauroth | Chris Nauroth | +| [HDFS-6229](https://issues.apache.org/jira/browse/HDFS-6229) | Race condition in failover can cause RetryCache fail to work | Major | ha | Jing Zhao | Jing Zhao | +| [HDFS-6232](https://issues.apache.org/jira/browse/HDFS-6232) | OfflineEditsViewer throws a NPE on edits containing ACL modifications | Major | tools | Stephen Chu | Akira Ajisaka | +| [HDFS-6236](https://issues.apache.org/jira/browse/HDFS-6236) | ImageServlet should use Time#monotonicNow to measure latency. | Minor | namenode | Chris Nauroth | Chris Nauroth | +| [MAPREDUCE-5828](https://issues.apache.org/jira/browse/MAPREDUCE-5828) | TestMapReduceJobControl fails on JDK 7 + Windows | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [YARN-1933](https://issues.apache.org/jira/browse/YARN-1933) | TestAMRestart and TestNodeHealthService failing sometimes on Windows | Major | . | Jian He | Jian He | +| [YARN-1928](https://issues.apache.org/jira/browse/YARN-1928) | TestAMRMRPCNodeUpdates fails ocassionally | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-1934](https://issues.apache.org/jira/browse/YARN-1934) | Potential NPE in ZKRMStateStore caused by handling Disconnected event from ZK. | Blocker | resourcemanager | Rohith Sharma K S | Karthik Kambatla | +| [MAPREDUCE-3191](https://issues.apache.org/jira/browse/MAPREDUCE-3191) | docs for map output compression incorrectly reference SequenceFile | Trivial | . | Todd Lipcon | Chen He | +| [YARN-1931](https://issues.apache.org/jira/browse/YARN-1931) | Private API change in YARN-1824 in 2.4 broke compatibility with previous releases | Blocker | applications | Thomas Graves | Sandy Ryza | +| [YARN-1932](https://issues.apache.org/jira/browse/YARN-1932) | Javascript injection on the job status page | Blocker | . | Mit Desai | Mit Desai | +| [HDFS-6245](https://issues.apache.org/jira/browse/HDFS-6245) | datanode fails to start with a bad disk even when failed volumes is set | Major | . | Arpit Gupta | Arpit Agarwal | +| [HADOOP-10522](https://issues.apache.org/jira/browse/HADOOP-10522) | JniBasedUnixGroupMapping mishandles errors | Critical | . | Kihwal Lee | Kihwal Lee | +| [MAPREDUCE-5827](https://issues.apache.org/jira/browse/MAPREDUCE-5827) | TestSpeculativeExecutionWithMRApp fails | Major | . | Zhijie Shen | Zhijie Shen | +| [MAPREDUCE-5832](https://issues.apache.org/jira/browse/MAPREDUCE-5832) | Few tests in TestJobClient fail on Windows | Major | . | Jian He | Vinod Kumar Vavilapalli | +| [HADOOP-10527](https://issues.apache.org/jira/browse/HADOOP-10527) | Fix incorrect return code and allow more retries on EINTR | Major | . | Kihwal Lee | Kihwal Lee | +| [MAPREDUCE-5841](https://issues.apache.org/jira/browse/MAPREDUCE-5841) | uber job doesn't terminate on getting mapred job kill | Major | mrv2 | Sangjin Lee | Sangjin Lee | +| [YARN-1975](https://issues.apache.org/jira/browse/YARN-1975) | Used resources shows escaped html in CapacityScheduler and FairScheduler page | Major | resourcemanager | Nathan Roberts | Mit Desai | +| [MAPREDUCE-5835](https://issues.apache.org/jira/browse/MAPREDUCE-5835) | Killing Task might cause the job to go to ERROR state | Critical | . | Ming Ma | Ming Ma | +| [MAPREDUCE-5714](https://issues.apache.org/jira/browse/MAPREDUCE-5714) | TestMRAppComponentDependencies causes surefire to exit without saying proper goodbye | Major | test | Jinghui Wang | Jinghui Wang | +| [HDFS-2882](https://issues.apache.org/jira/browse/HDFS-2882) | DN continues to start up, even if block pool fails to initialize | Major | datanode | Todd Lipcon | Vinayakumar B | +| [YARN-1929](https://issues.apache.org/jira/browse/YARN-1929) | DeadLock in RM when automatic failover is enabled. | Blocker | resourcemanager | Rohith Sharma K S | Karthik Kambatla | | [YARN-1201](https://issues.apache.org/jira/browse/YARN-1201) | TestAMAuthorization fails with local hostname cannot be resolved | Minor | resourcemanager | Nemon Lou | Wangda Tan | +| [HADOOP-10562](https://issues.apache.org/jira/browse/HADOOP-10562) | Namenode exits on exception without printing stack trace in AbstractDelegationTokenSecretManager | Critical | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-6340](https://issues.apache.org/jira/browse/HDFS-6340) | DN can't finalize upgrade | Blocker | datanode | Rahul Singhal | Rahul Singhal | +| [HDFS-6329](https://issues.apache.org/jira/browse/HDFS-6329) | WebHdfs does not work if HA is enabled on NN but logical URI is not configured. | Blocker | . | Kihwal Lee | Kihwal Lee | +| [HDFS-6313](https://issues.apache.org/jira/browse/HDFS-6313) | WebHdfs may use the wrong NN when configured for multiple HA NNs | Blocker | webhdfs | Daryn Sharp | Kihwal Lee | +| [YARN-1976](https://issues.apache.org/jira/browse/YARN-1976) | Tracking url missing http protocol for FAILED application | Major | . | Yesha Vora | Junping Du | +| [YARN-2016](https://issues.apache.org/jira/browse/YARN-2016) | Yarn getApplicationRequest start time range is not honored | Major | resourcemanager | Venkat Ranganathan | Junping Du | +| [YARN-1986](https://issues.apache.org/jira/browse/YARN-1986) | In Fifo Scheduler, node heartbeat in between creating app and attempt causes NPE | Critical | . | Jon Bringhurst | Hong Zhiguo | +| [MAPREDUCE-5821](https://issues.apache.org/jira/browse/MAPREDUCE-5821) | IFile merge allocates new byte array for every value | Major | performance, task | Todd Lipcon | Todd Lipcon | +| [HDFS-6326](https://issues.apache.org/jira/browse/HDFS-6326) | WebHdfs ACL compatibility is broken | Blocker | webhdfs | Daryn Sharp | Chris Nauroth | +| [HDFS-6361](https://issues.apache.org/jira/browse/HDFS-6361) | TestIdUserGroup.testUserUpdateSetting failed due to out of range nfsnobody Id | Major | nfs | Yongjun Zhang | Yongjun Zhang | +| [HDFS-6362](https://issues.apache.org/jira/browse/HDFS-6362) | InvalidateBlocks is inconsistent in usage of DatanodeUuid and StorageID | Blocker | namenode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-6402](https://issues.apache.org/jira/browse/HDFS-6402) | Suppress findbugs warning for failure to override equals and hashCode in FsAclPermission. | Trivial | namenode | Chris Nauroth | Chris Nauroth | +| [YARN-2066](https://issues.apache.org/jira/browse/YARN-2066) | Wrong field is referenced in GetApplicationsRequestPBImpl#mergeLocalToBuilder() | Minor | . | Ted Yu | Hong Zhiguo | +| [HDFS-6325](https://issues.apache.org/jira/browse/HDFS-6325) | Append should fail if the last block has insufficient number of replicas | Major | namenode | Konstantin Shvachko | Keith Pak | +| [HDFS-6397](https://issues.apache.org/jira/browse/HDFS-6397) | NN shows inconsistent value in deadnode count | Critical | . | Mohammad Kamrul Islam | Mohammad Kamrul Islam | +| [HADOOP-10612](https://issues.apache.org/jira/browse/HADOOP-10612) | NFS failed to refresh the user group id mapping table | Major | nfs | Brandon Li | Brandon Li | +| [YARN-2081](https://issues.apache.org/jira/browse/YARN-2081) | TestDistributedShell fails after YARN-1962 | Minor | applications/distributed-shell | Hong Zhiguo | Hong Zhiguo | +| [HDFS-6411](https://issues.apache.org/jira/browse/HDFS-6411) | nfs-hdfs-gateway mount raises I/O error and hangs when a unauthorized user attempts to access it | Major | nfs | Zhongyi Xie | Brandon Li | +| [MAPREDUCE-5830](https://issues.apache.org/jira/browse/MAPREDUCE-5830) | HostUtil.getTaskLogUrl is not backwards binary compatible with 2.3 | Blocker | . | Jason Lowe | Akira Ajisaka | +| [HADOOP-11273](https://issues.apache.org/jira/browse/HADOOP-11273) | TestMiniKdc failure: login options not compatible with IBM JDK | Major | test | Gao Zhong Liang | Gao Zhong Liang | ### TESTS: @@ -129,30 +113,24 @@ | JIRA | Summary | Priority | Component | Reporter | Contributor | |:---- |:---- | :--- |:---- |:---- |:---- | | [HDFS-6189](https://issues.apache.org/jira/browse/HDFS-6189) | Multiple HDFS tests fail on Windows attempting to use a test root path containing a colon. | Major | test | Chris Nauroth | Chris Nauroth | -| [MAPREDUCE-5843](https://issues.apache.org/jira/browse/MAPREDUCE-5843) | TestMRKeyValueTextInputFormat failing on Windows | Major | . | Varun Vasudev | Varun Vasudev | -| [MAPREDUCE-5833](https://issues.apache.org/jira/browse/MAPREDUCE-5833) | TestRMContainerAllocator fails ocassionally | Major | . | Zhijie Shen | Zhijie Shen | -| [YARN-1947](https://issues.apache.org/jira/browse/YARN-1947) | TestRMDelegationTokens#testRMDTMasterKeyStateOnRollingMasterKey is failing intermittently | Major | . | Jian He | Jian He | | [YARN-1905](https://issues.apache.org/jira/browse/YARN-1905) | TestProcfsBasedProcessTree must only run on Linux. | Trivial | nodemanager | Chris Nauroth | Chris Nauroth | | [YARN-1750](https://issues.apache.org/jira/browse/YARN-1750) | TestNodeStatusUpdater#testNMRegistration is incorrect in test case | Major | nodemanager | Ming Ma | Wangda Tan | +| [YARN-1947](https://issues.apache.org/jira/browse/YARN-1947) | TestRMDelegationTokens#testRMDTMasterKeyStateOnRollingMasterKey is failing intermittently | Major | . | Jian He | Jian He | | [YARN-1281](https://issues.apache.org/jira/browse/YARN-1281) | TestZKRMStateStoreZKClientConnections fails intermittently | Major | resourcemanager | Karthik Kambatla | Tsuyoshi Ozawa | +| [MAPREDUCE-5833](https://issues.apache.org/jira/browse/MAPREDUCE-5833) | TestRMContainerAllocator fails ocassionally | Major | . | Zhijie Shen | Zhijie Shen | +| [MAPREDUCE-5843](https://issues.apache.org/jira/browse/MAPREDUCE-5843) | TestMRKeyValueTextInputFormat failing on Windows | Major | . | Varun Vasudev | Varun Vasudev | ### SUB-TASKS: | JIRA | Summary | Priority | Component | Reporter | Contributor | |:---- |:---- | :--- |:---- |:---- |:---- | -| [YARN-2053](https://issues.apache.org/jira/browse/YARN-2053) | Slider AM fails to restart: NPE in RegisterApplicationMasterResponseProto$Builder.addAllNmTokensFromPreviousAttempts | Major | resourcemanager | Sumit Mohanty | Wangda Tan | +| [YARN-1696](https://issues.apache.org/jira/browse/YARN-1696) | Document RM HA | Blocker | resourcemanager | Karthik Kambatla | Tsuyoshi Ozawa | +| [YARN-1701](https://issues.apache.org/jira/browse/YARN-1701) | Improve default paths of timeline store and generic history store | Major | . | Gera Shegalov | Tsuyoshi Ozawa | | [YARN-1962](https://issues.apache.org/jira/browse/YARN-1962) | Timeline server is enabled by default | Major | . | Mohammad Kamrul Islam | Mohammad Kamrul Islam | +| [YARN-1861](https://issues.apache.org/jira/browse/YARN-1861) | Both RM stuck in standby mode when automatic failover is enabled | Blocker | resourcemanager | Arpit Gupta | Karthik Kambatla | | [YARN-1957](https://issues.apache.org/jira/browse/YARN-1957) | ProportionalCapacitPreemptionPolicy handling of corner cases... | Major | resourcemanager | Carlo Curino | Carlo Curino | +| [YARN-2053](https://issues.apache.org/jira/browse/YARN-2053) | Slider AM fails to restart: NPE in RegisterApplicationMasterResponseProto$Builder.addAllNmTokensFromPreviousAttempts | Major | resourcemanager | Sumit Mohanty | Wangda Tan | | [YARN-1898](https://issues.apache.org/jira/browse/YARN-1898) | Standby RM's conf, stacks, logLevel, metrics, jmx and logs links are redirecting to Active RM | Major | resourcemanager | Yesha Vora | Xuan Gong | -| [YARN-1861](https://issues.apache.org/jira/browse/YARN-1861) | Both RM stuck in standby mode when automatic failover is enabled | Blocker | resourcemanager | Arpit Gupta | Karthik Kambatla | -| [YARN-1701](https://issues.apache.org/jira/browse/YARN-1701) | Improve default paths of timeline store and generic history store | Major | . | Gera Shegalov | Tsuyoshi Ozawa | -| [YARN-1696](https://issues.apache.org/jira/browse/YARN-1696) | Document RM HA | Blocker | resourcemanager | Karthik Kambatla | Tsuyoshi Ozawa | - - -### OTHER: - -| JIRA | Summary | Priority | Component | Reporter | Contributor | -|:---- |:---- | :--- |:---- |:---- |:---- | --------------------------------------------------------------------- To unsubscribe, e-mail: common-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-commits-h...@hadoop.apache.org
