Author: mattf
Date: Tue Apr 16 09:35:51 2013
New Revision: 1468336
URL: http://svn.apache.org/r1468336
Log:
Release notes for 1.2.0
Modified:
hadoop/common/branches/branch-1.2/src/docs/releasenotes.html
Modified: hadoop/common/branches/branch-1.2/src/docs/releasenotes.html
URL:
http://svn.apache.org/viewvc/hadoop/common/branches/branch-1.2/src/docs/releasenotes.html?rev=1468336&r1=1468335&r2=1468336&view=diff
==============================================================================
--- hadoop/common/branches/branch-1.2/src/docs/releasenotes.html (original)
+++ hadoop/common/branches/branch-1.2/src/docs/releasenotes.html Tue Apr 16
09:35:51 2013
@@ -194,6 +194,11 @@ To run secure Datanodes users must insta
<b>Allow setting of end-of-record delimiter for TextInputFormat</b><br>
<blockquote>The patch for
https://issues.apache.org/jira/browse/MAPREDUCE-2254 required minor changes to
the LineReader class to allow extensions (see attached 2.patch). Description
copied below:<br><br>It will be useful to allow setting the end-of-record
delimiter for TextInputFormat. The current implementation hardcodes
'\n', '\r' or '\r\n' as the only possible record
delimiters. This is a problem if users have embedded newlines in their data
fields (which is pretty common). This is also a problem for other
...</blockquote></li>
+<li> <a
href="https://issues.apache.org/jira/browse/HADOOP-7101";>HADOOP-7101</a>.
+ Blocker bug reported by tlipcon and fixed by tlipcon (security)<br>
+ <b>UserGroupInformation.getCurrentUser() fails when called from
non-Hadoop JAAS context</b><br>
+ <blockquote>If a Hadoop client is run from inside a container like
Tomcat, and the current AccessControlContext has a Subject associated with it
that is not created by Hadoop, then UserGroupInformation.getCurrentUser() will
throw NoSuchElementException, since it assumes that any Subject will have a
hadoop User principal.</blockquote></li>
+
<li> <a
href="https://issues.apache.org/jira/browse/HADOOP-7688";>HADOOP-7688</a>.
Major improvement reported by szetszwo and fixed by umamaheswararao <br>
<b>When a servlet filter throws an exception in init(..), the Jetty
server failed silently. </b><br>
@@ -369,6 +374,11 @@ To run secure Datanodes users must insta
<b>TestSinkQueue.testConcurrentConsumers fails intermittently (Backports
HADOOP-7292)</b><br>
<blockquote>org.apache.hadoop.metrics2.impl.TestSinkQueue.testConcurrentConsumers<br>
<br><br>Error Message<br><br>should've
thrown<br>Stacktrace<br><br>junit.framework.AssertionFailedError:
should've thrown<br> at
org.apache.hadoop.metrics2.impl.TestSinkQueue.shouldThrowCME(TestSinkQueue.java:229)<br>
at
org.apache.hadoop.metrics2.impl.TestSinkQueue.testConcurrentConsumers(TestSinkQueue.java:195)<br>Standard
Output<br><br>2012-10-03 16:51:31,694 INFO impl.TestSinkQueue
(TestSinkQueue.java:consume(243)) - sleeping<br></blockquote></li>
+<li> <a
href="https://issues.apache.org/jira/browse/HADOOP-9071";>HADOOP-9071</a>.
+ Major improvement reported by gkesavan and fixed by gkesavan (build)<br>
+ <b>configure ivy log levels for resolve/retrieve</b><br>
+ <blockquote></blockquote></li>
+
<li> <a
href="https://issues.apache.org/jira/browse/HADOOP-9090";>HADOOP-9090</a>.
Minor new feature reported by mostafae and fixed by mostafae (metrics)<br>
<b>Support on-demand publish of metrics</b><br>
@@ -439,6 +449,31 @@ To run secure Datanodes users must insta
<b>Port HADOOP-7290 to branch-1 to fix TestUserGroupInformation
failure</b><br>
<blockquote>Unit test failure in
TestUserGroupInformation.testGetServerSideGroups. port HADOOP-7290 to
branch-1.1 </blockquote></li>
+<li> <a
href="https://issues.apache.org/jira/browse/HADOOP-9379";>HADOOP-9379</a>.
+ Trivial improvement reported by arpitgupta and fixed by arpitgupta <br>
+ <b>capture the ulimit info after printing the log to the console</b><br>
+ <blockquote>Based on the discussions in HADOOP-9253 people prefer if we
dont print the ulimit info to the console but still have it in the
logs.<br><br>Just need to move the head statement to before the capture of
ulimit code.</blockquote></li>
+
+<li> <a
href="https://issues.apache.org/jira/browse/HADOOP-9434";>HADOOP-9434</a>.
+ Minor improvement reported by carp84 and fixed by carp84 (bin)<br>
+ <b>Backport HADOOP-9267 to branch-1</b><br>
+ <blockquote>Currently in hadoop 1.1.2, if user issue "bin/hadoop
help" in command line, it will throw below exception. We can improve this
to print the usage
message.<br>===============================================<br>Exception in
thread "main" java.lang.NoClassDefFoundError:
help<br>===============================================<br><br>This issue is
already resolved in HADOOP-9267 in trunk, so we only need to backport it into
branch-1</blockquote></li>
+
+<li> <a
href="https://issues.apache.org/jira/browse/HADOOP-9451";>HADOOP-9451</a>.
+ Major bug reported by djp and fixed by djp (net)<br>
+ <b>Node with one topology layer should be handled as fault topology when
NodeGroup layer is enabled</b><br>
+ <blockquote>Currently, nodes with one layer topology are allowed to join
in the cluster that with enabling NodeGroup layer which cause some exception
cases. <br>When NodeGroup layer is enabled, the cluster should assumes that at
least two layer (Rack/NodeGroup) is valid topology for each nodes, so should
throw exceptions for one layer node in joining.</blockquote></li>
+
+<li> <a
href="https://issues.apache.org/jira/browse/HADOOP-9467";>HADOOP-9467</a>.
+ Major bug reported by cnauroth and fixed by cnauroth (metrics)<br>
+ <b>Metrics2 record filtering (.record.filter.include/exclude) does not
filter by name</b><br>
+ <blockquote>Filtering by record considers only the record's tag for
filtering and not the record's name.</blockquote></li>
+
+<li> <a
href="https://issues.apache.org/jira/browse/HADOOP-9473";>HADOOP-9473</a>.
+ Trivial bug reported by gmazza and fixed by (fs)<br>
+ <b>typo in FileUtil copy() method</b><br>
+ <blockquote>typo:<br>{code}<br>Index:
src/core/org/apache/hadoop/fs/FileUtil.java<br>===================================================================<br>---
src/core/org/apache/hadoop/fs/FileUtil.java (revision 1467295)<br>+++
src/core/org/apache/hadoop/fs/FileUtil.java (working copy)<br>@@ -178,7
+178,7 @@<br> // Check if dest is directory<br> if (!dstFS.exists(dst))
{<br> throw new IOException("`" + dst +"': specified
destination directory " +<br>- "doest not
exist");<br>+ ...</blockquote></li>
+
<li> <a href="https://issues.apache.org/jira/browse/HDFS-1957";>HDFS-1957</a>.
Minor improvement reported by asrabkin and fixed by asrabkin
(documentation)<br>
<b>Documentation for HFTP</b><br>
@@ -606,7 +641,7 @@ To run secure Datanodes users must insta
<li> <a href="https://issues.apache.org/jira/browse/HDFS-4222";>HDFS-4222</a>.
Minor bug reported by teledriver and fixed by teledriver (namenode)<br>
- <b>NN is unresponsive and lose heartbeats of DNs when Hadoop is
configured to use LDAP and LDAP has issues</b><br>
+ <b>NN is unresponsive and loses heartbeats of DNs when Hadoop is
configured to use LDAP and LDAP has issues</b><br>
<blockquote>For Hadoop clusters configured to access directory
information by LDAP, the FSNamesystem calls on behave of DFS clients might hang
due to LDAP issues (including LDAP access issues caused by networking issues)
while holding the single lock of FSNamesystem. That will result in the NN
unresponsive and loss of the heartbeats from DNs.<br><br>The places LDAP got
accessed by FSNamesystem calls are the instantiation of FSPermissionChecker,
which could be moved out of the lock scope since the
instantiation...</blockquote></li>
<li> <a href="https://issues.apache.org/jira/browse/HDFS-4256";>HDFS-4256</a>.
@@ -629,6 +664,11 @@ To run secure Datanodes users must insta
<b>TestCheckpoint failure with JDK7</b><br>
<blockquote>testMultipleSecondaryNameNodes doesn't shutdown the
SecondaryNameNode which causes testCheckpoint to fail.<br><br>Testcase:
testCheckpoint took 2.736 sec<br> Caused an ERROR<br>Cannot lock storage
C:\hdp1-2\build\test\data\dfs\namesecondary1. The directory is already
locked.<br>java.io.IOException: Cannot lock storage
C:\hdp1-2\build\test\data\dfs\namesecondary1. The directory is already
locked.<br> at
org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.lock(Storage.java:602)<br>
at org.apache.hadoop.hd...</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4413";>HDFS-4413</a>.
+ Major bug reported by mostafae and fixed by mostafae (namenode)<br>
+ <b>Secondary namenode won't start if HDFS isn't the default
file system</b><br>
+ <blockquote>If HDFS is not the default file system (fs.default.name is
something other than hdfs://...), then secondary namenode throws early on in
its initialization. This is a needless check as far as I can tell, and blocks
scenarios where HDFS services are up but HDFS is not the default file
system.</blockquote></li>
+
<li> <a href="https://issues.apache.org/jira/browse/HDFS-4444";>HDFS-4444</a>.
Trivial bug reported by schu and fixed by schu <br>
<b>Add space between total transaction time and number of transactions in
FSEditLog#printStatistics</b><br>
@@ -642,7 +682,7 @@ To run secure Datanodes users must insta
<li> <a href="https://issues.apache.org/jira/browse/HDFS-4479";>HDFS-4479</a>.
Major bug reported by jingzhao and fixed by jingzhao <br>
<b>logSync() with the FSNamesystem lock held in
commitBlockSynchronization</b><br>
- <blockquote>In FSNamesystem#commitBlockSynchronization of branch-1,
logSync() may be called when the FSNamesystem lock is held. Similar with
HDFS-4186, this may cause some performance issue.<br><br>Since logSync is
called right after the synchronization section, we can simply remove the
logSync call.</blockquote></li>
+ <blockquote>In FSNamesystem#commitBlockSynchronization of branch-1,
logSync() may be called when the FSNamesystem lock is held. Similar to
HDFS-4186, this may cause some performance issue.<br><br>The following issue
was observed in a cluster that was running a Hive job and was writing to
100,000 temporary files (each task is writing to 1000s of files). When this job
is killed, a large number of files are left open for write. Eventually when the
lease for open files expires, lease recovery is started for all
th...</blockquote></li>
<li> <a href="https://issues.apache.org/jira/browse/HDFS-4518";>HDFS-4518</a>.
Major bug reported by arpitagarwal and fixed by arpitagarwal <br>
@@ -664,6 +704,16 @@ To run secure Datanodes users must insta
<b>start balancer failed with NPE</b><br>
<blockquote>start balancer failed with NPE<br> File this issue to track
for QE and dev take a look<br><br>balancer.log:<br> 2013-03-06 00:19:55,174
ERROR org.apache.hadoop.hdfs.server.balancer.Balancer:
java.lang.NullPointerException<br> at
org.apache.hadoop.hdfs.server.namenode.BlockPlacementPolicy.getInstance(BlockPlacementPolicy.java:165)<br>
at
org.apache.hadoop.hdfs.server.balancer.Balancer.checkReplicationPolicyCompatibility(Balancer.java:799)<br>
at
org.apache.hadoop.hdfs.server.balancer.Balancer.<init>(Balancer.java:...</blockquote></li>
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4597";>HDFS-4597</a>.
+ Major new feature reported by szetszwo and fixed by szetszwo (webhdfs)<br>
+ <b>Backport WebHDFS concat to branch-1</b><br>
+ <blockquote>HDFS-3598 adds cancat to WebHDFS. Let's also add it to
branch-1.</blockquote></li>
+
+<li> <a href="https://issues.apache.org/jira/browse/HDFS-4651";>HDFS-4651</a>.
+ Major improvement reported by cnauroth and fixed by cnauroth (tools)<br>
+ <b>Offline Image Viewer backport to branch-1</b><br>
+ <blockquote>This issue tracks backporting the Offline Image Viewer tool
to branch-1.</blockquote></li>
+
<li> <a
href="https://issues.apache.org/jira/browse/MAPREDUCE-461";>MAPREDUCE-461</a>.
Minor new feature reported by fhedberg and fixed by fhedberg <br>
<b>Enable ServicePlugins for the JobTracker</b><br>
@@ -769,6 +819,11 @@ To run secure Datanodes users must insta
<b>Backport MR-2779 (JobSplitWriter.java can't handle large
job.split file) to branch-1</b><br>
<blockquote></blockquote></li>
+<li> <a
href="https://issues.apache.org/jira/browse/MAPREDUCE-4463";>MAPREDUCE-4463</a>.
+ Blocker bug reported by tomwhite and fixed by tomwhite (mrv1)<br>
+ <b>JobTracker recovery fails with HDFS permission issue</b><br>
+ <blockquote>Recovery fails when the job user is different to the JT owner
(i.e. on anything bigger than a pseudo-distributed cluster).</blockquote></li>
+
<li> <a
href="https://issues.apache.org/jira/browse/MAPREDUCE-4464";>MAPREDUCE-4464</a>.
Minor improvement reported by heathcd and fixed by heathcd (task)<br>
<b>Reduce tasks failing with NullPointerException in
ConcurrentHashMap.get()</b><br>
@@ -849,6 +904,11 @@ To run secure Datanodes users must insta
<b>Cleanup: Some (5) private methods in JobTracker.RecoveryManager are
not used anymore after MAPREDUCE-3837</b><br>
<blockquote>MAPREDUCE-3837 re-organized the job recovery code, moving out
the code that was using the methods in RecoveryManager.<br><br>Now, the
following methods in {{JobTracker.RecoveryManager}}seem to be unused:<br>#
{{updateJob()}}<br># {{updateTip()}}<br># {{createTaskAttempt()}}<br>#
{{addSuccessfulAttempt()}}<br># {{addUnsuccessfulAttempt()}}</blockquote></li>
+<li> <a
href="https://issues.apache.org/jira/browse/MAPREDUCE-4824";>MAPREDUCE-4824</a>.
+ Major new feature reported by tomwhite and fixed by tomwhite (mrv1)<br>
+ <b>Provide a mechanism for jobs to indicate they should not be recovered
on restart</b><br>
+ <blockquote>Some jobs (like Sqoop or HBase jobs) are not idempotent, so
should not be recovered on jobtracker restart. MAPREDUCE-2702 solves this
problem for MR2, however the approach there is not applicable for MR1, since
even if we only use the job-level part of the patch and add a
isRecoverySupported method to OutputCommitter, there is no way to use that
information from the JT (which initiates recovery), since the JT does not
instantiate OutputCommitters - and it shouldn't since they are user-level
c...</blockquote></li>
+
<li> <a
href="https://issues.apache.org/jira/browse/MAPREDUCE-4837";>MAPREDUCE-4837</a>.
Major improvement reported by acmurthy and fixed by acmurthy <br>
<b>Add webservices for jobtracker</b><br>
@@ -974,16 +1034,26 @@ To run secure Datanodes users must insta
<b>Update MR1 memory configuration docs</b><br>
<blockquote>The pmem/vmem settings in the docs
(http://hadoop.apache.org/docs/r1.1.1/cluster_setup.html#Memory+monitoring)
have not been supported for a long time. The docs should be updated to reflect
the new settings (mapred.cluster.map.memory.mb etc).</blockquote></li>
-<li> <a
href="https://issues.apache.org/jira/browse/MAPREDUCE-5038";>MAPREDUCE-5038</a>.
- Major bug reported by sandyr and fixed by sandyr <br>
- <b>old API CombineFileInputFormat missing fixes that are in new API
</b><br>
- <blockquote>The following changes patched the CombineFileInputFormat in
mapreduce, but neglected the one in mapred<br>MAPREDUCE-1597 enabled the
CombineFileInputFormat to work on splittable files<br>MAPREDUCE-2021 solved
returning duplicate hostnames in split locations<br>MAPREDUCE-1806
CombineFileInputFormat does not work with paths not on default FS<br><br>In
trunk this is not an issue as the one in mapred extends the one in
mapreduce.</blockquote></li>
-
<li> <a
href="https://issues.apache.org/jira/browse/MAPREDUCE-5049";>MAPREDUCE-5049</a>.
Major bug reported by sandyr and fixed by sandyr <br>
<b>CombineFileInputFormat counts all compressed files
non-splitable</b><br>
<blockquote>In branch-1, CombineFileInputFormat doesn't take
SplittableCompressionCodec into account and thinks that all compressible input
files aren't splittable. This is a regression from when handling for
non-splitable compression codecs was originally added in MAPREDUCE-1597, and
seems to have somehow gotten in when the code was pulled from 0.22 to
branch-1.<br></blockquote></li>
+<li> <a
href="https://issues.apache.org/jira/browse/MAPREDUCE-5081";>MAPREDUCE-5081</a>.
+ Major new feature reported by szetszwo and fixed by szetszwo (distcp)<br>
+ <b>Backport DistCpV2 and the related JIRAs to branch-1</b><br>
+ <blockquote>Here is a list of DistCpV2 JIRAs:<br>- MAPREDUCE-2765:
DistCpV2 main jira<br>- HADOOP-8703: turn CRC checking off for 0 byte size
<br>- HDFS-3054: distcp -skipcrccheck has no effect.<br>- HADOOP-8431: Running
distcp without args throws IllegalArgumentException<br>- HADOOP-8775:
non-positive value to -bandwidth<br>- MAPREDUCE-4654: TestDistCp is
ignored<br>- HADOOP-9022: distcp fails to copy file if -m 0 specified<br>-
HADOOP-9025: TestCopyListing failing<br>- MAPREDUCE-5075: DistCp leaks input
file handles<br>- distcp par...</blockquote></li>
+
+<li> <a
href="https://issues.apache.org/jira/browse/MAPREDUCE-5129";>MAPREDUCE-5129</a>.
+ Minor new feature reported by billie.rinaldi and fixed by billie.rinaldi
<br>
+ <b>Add tag info to JH files</b><br>
+ <blockquote>It will be useful to add tags to the existing workflow info
logged by JH. This will allow jobs to be filtered/grouped for analysis more
easily.</blockquote></li>
+
+<li> <a
href="https://issues.apache.org/jira/browse/MAPREDUCE-5131";>MAPREDUCE-5131</a>.
+ Major bug reported by acmurthy and fixed by acmurthy <br>
+ <b>Provide better handling of job status related apis during JT
restart</b><br>
+ <blockquote>I've seen pig/hive applications bork during JT restart
since they get NPEs - this is due to fact that jobs are not really inited, but
are submitted.</blockquote></li>
+
</ul>
