svn commit: r1411102 - /hadoop/common/branches/branch-1/CHANGES.txt
Author: mattf Date: Mon Nov 19 08:54:03 2012 New Revision: 1411102 URL: http://svn.apache.org/viewvc?rev=1411102view=rev Log: fixed CHANGES.txt to correctly show five patches applied in release 1.1.1 instead of release 1.2.0. Modified: hadoop/common/branches/branch-1/CHANGES.txt Modified: hadoop/common/branches/branch-1/CHANGES.txt URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-1/CHANGES.txt?rev=1411102r1=1411101r2=1411102view=diff == --- hadoop/common/branches/branch-1/CHANGES.txt (original) +++ hadoop/common/branches/branch-1/CHANGES.txt Mon Nov 19 08:54:03 2012 @@ -13,9 +13,6 @@ Release 1.2.0 - unreleased MAPREDUCE-987. Exposing MiniDFS and MiniMR clusters as a single process command-line (philip and ahmed via tucu) -HDFS-528. Backport: Add ability for safemode to wait for a minimum number -of live datanodes. (szetszwo) - HDFS-385. Backport: Add support for an experimental API that allows a module external to HDFS to specify how HDFS blocks should be placed. (Sumadhur Reddy Bolli via szetszwo) @@ -86,8 +83,6 @@ Release 1.2.0 - unreleased MAPREDUCE-4499. Looking for speculative tasks is very expensive in 1.x (Koji Noguchi via tgraves) -HDFS-1108 Log newly allocated blocks (hdfs-1108-hadoop-1-v5.patch) (sanjay) - HADOOP-8832. Port generic service plugin mechanism from HADOOP-5257 to branch-1. (backported by Brandon Li via suresh) @@ -213,16 +208,12 @@ Release 1.2.0 - unreleased HDFS-3754. BlockSender doesn't shutdown ReadaheadPool threads. (eli) -HDFS-3658. Fix bugs in TestDFSClientRetries and add more tests. (szetszwo) - HADOOP-8611. Allow fall-back to the shell-based implementation when JNI-based users-group mapping fails (Robert Parker via bobby) MAPREDUCE-2374. Text File Busy errors launching MR tasks. (Andy Isaacson via atm) -HDFS-3846. Namenode deadlock in branch-1. (Brandon Li via jitendra) - HADOOP-4572. Can not access user logs - Jetty is not configured by default to serve aliases/symlinks (ahmed via tucu) @@ -234,9 +225,6 @@ Release 1.2.0 - unreleased MAPREDUCE-4629. Remove JobHistory.DEBUG_MODE (Karthik Kambatla via bobby) -HADOOP-8745. Incorrect version numbers in hadoop-core POM. -(Matthias Friedrich via eli) - HDFS-2757. Cannot read a local block that's being written to when using the local read short circuit. (Jean-Daniel Cryans via eli) @@ -324,8 +312,13 @@ Release 1.1.1 - Unreleased NEW FEATURES +HDFS-528. Backport: Add ability for safemode to wait for a minimum number +of live datanodes. (szetszwo) + IMPROVEMENTS +HDFS-1108 Log newly allocated blocks (hdfs-1108-hadoop-1-v5.patch) (sanjay) + HADOOP-8823. ant package target should not depend on cn-docs. (szetszwo) HADOOP-8995. Remove unnecessary bogus exception from Configuration.java. @@ -338,7 +331,7 @@ Release 1.1.1 - Unreleased HDFS-1539. A config option for the datanode to fsycn a block file when block is completely written. (dhruba via szetszwo) -HDFS-4174. Add abilit to list the corrupted files in WebUI (backport of +HDFS-4174. Add ability to list the corrupted files in WebUI (backport of HDFS-1031). (Jing Zhao via suresh) BUG FIXES @@ -375,6 +368,13 @@ Release 1.1.1 - Unreleased MAPREDUCE-4792. Unit Test TestJobTrackerRestartWithLostTracker fails with ant-1.8.4. (Amir Sanjar via mattf) +HDFS-3658. Fix bugs in TestDFSClientRetries and add more tests. (szetszwo) + +HDFS-3846. Namenode deadlock in branch-1. (Brandon Li via jitendra) + +HADOOP-8745. Incorrect version numbers in hadoop-core POM. +(Matthias Friedrich via eli) + Release 1.1.0 - 2012.09.28 INCOMPATIBLE CHANGES
svn commit: r1411109 - /hadoop/common/tags/release-1.1.1-rc0/
Author: mattf Date: Mon Nov 19 09:21:20 2012 New Revision: 1411109 URL: http://svn.apache.org/viewvc?rev=1411109view=rev Log: Hadoop 1.1.1-rc0 release. Added: hadoop/common/tags/release-1.1.1-rc0/ (props changed) - copied from r1411108, hadoop/common/branches/branch-1.1/ Propchange: hadoop/common/tags/release-1.1.1-rc0/ -- --- svn:ignore (added) +++ svn:ignore Mon Nov 19 09:21:20 2012 @@ -0,0 +1,9 @@ +build +build.properties +logs +.classpath +.git +.project +.settings +.launches +.externalToolBuilders Propchange: hadoop/common/tags/release-1.1.1-rc0/ -- --- svn:mergeinfo (added) +++ svn:mergeinfo Mon Nov 19 09:21:20 2012 @@ -0,0 +1,9 @@ +/hadoop/common/branches/branch-0.20:826138,826568,829987,831184,833001,880632,898713,909245,909723,960946,990003,1044225 +/hadoop/common/branches/branch-0.20-append:955380,955398,955448,956329 +/hadoop/common/branches/branch-0.20-security-203:1096071,1097011,1097249,1097269,1097281,1097966,1098816,1098819,1098823,1098827,1098832,1098839,1098854,1098863,1099088,1099191,1099324,1099330,1099333,1102071,1128115 +/hadoop/common/branches/branch-0.20-security-204:1128390,1147228,1148069,1149316,1154413 +/hadoop/common/branches/branch-0.20-security-205:1174370,1174917,1176042,1176248,1176638,1176645,1202378 +/hadoop/common/branches/branch-1:1343738,1343992,1346167,1353893,1353928,1354390,1354773,1354798,1357994,1360011,1360940,1360944,1366295,1374605,1379653,1384742,1392637,1399786,1407625,1408450 +/hadoop/common/branches/branch-1.0:1214410,1291091,1330040,1334877 +/hadoop/core/branches/branch-0.19:713112 +/hadoop/core/trunk:727001,727117,727191,727212,727217,727228,727255,727869,728187,729052,729987,732385,732572,732613,732777,732838,732869,733887,734870,734916,736426,738328,738697,740077,740157,741703,741762,743745,743816,743892,744894,745180,746010,746206,746227,746233,746274,746338,746902-746903,746925,746944,746968,746970,747279,747289,747802,748084,748090,748783,749262,749318,749863,750533,752073,752609,752834,752836,752913,752932,753112-753113,753346,754645,754847,754927,755035,755226,755348,755370,755418,755426,755790,755905,755938,755960,755986,755998,756352,757448,757624,757849,758156,758180,759398,759932,760502,760783,761046,761482,761632,762216,762879,763107,763502,764967,765016,765809,765951,771607,771661,772844,772876,772884,772920,773889,776638,778962,778966,779893,781720,784661,785046,785569
svn commit: r1411235 - /hadoop/common/trunk/hadoop-project/pom.xml
Author: tomwhite Date: Mon Nov 19 15:12:22 2012 New Revision: 1411235 URL: http://svn.apache.org/viewvc?rev=1411235view=rev Log: YARN-129. Simplify classpath construction for mini YARN tests. Modified: hadoop/common/trunk/hadoop-project/pom.xml Modified: hadoop/common/trunk/hadoop-project/pom.xml URL: http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-project/pom.xml?rev=1411235r1=1411234r2=1411235view=diff == --- hadoop/common/trunk/hadoop-project/pom.xml (original) +++ hadoop/common/trunk/hadoop-project/pom.xml Mon Nov 19 15:12:22 2012 @@ -703,11 +703,6 @@ groupIdorg.apache.maven.plugins/groupId artifactIdmaven-jar-plugin/artifactId version2.3.1/version - configuration -excludes - excludemrapp-generated-classpath/exclude -/excludes - /configuration /plugin plugin groupIdorg.apache.maven.plugins/groupId @@ -803,21 +798,6 @@ /executions /plugin plugin -artifactIdmaven-dependency-plugin/artifactId -executions - execution -idbuild-classpath/id -phasegenerate-sources/phase -goals - goalbuild-classpath/goal -/goals -configuration - outputFiletarget/classes/mrapp-generated-classpath/outputFile -/configuration - /execution -/executions - /plugin - plugin groupIdorg.apache.maven.plugins/groupId artifactIdmaven-surefire-plugin/artifactId configuration
svn commit: r1411239 - /hadoop/common/branches/branch-2/hadoop-project/pom.xml
Author: tomwhite Date: Mon Nov 19 15:16:57 2012 New Revision: 1411239 URL: http://svn.apache.org/viewvc?rev=1411239view=rev Log: Merge -r 1411234:1411235 from trunk to branch-2. Fixes: YARN-129. Simplify classpath construction for mini YARN tests. Modified: hadoop/common/branches/branch-2/hadoop-project/pom.xml Modified: hadoop/common/branches/branch-2/hadoop-project/pom.xml URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-project/pom.xml?rev=1411239r1=1411238r2=1411239view=diff == --- hadoop/common/branches/branch-2/hadoop-project/pom.xml (original) +++ hadoop/common/branches/branch-2/hadoop-project/pom.xml Mon Nov 19 15:16:57 2012 @@ -705,11 +705,6 @@ groupIdorg.apache.maven.plugins/groupId artifactIdmaven-jar-plugin/artifactId version2.3.1/version - configuration -excludes - excludemrapp-generated-classpath/exclude -/excludes - /configuration /plugin plugin groupIdorg.apache.maven.plugins/groupId @@ -805,21 +800,6 @@ /executions /plugin plugin -artifactIdmaven-dependency-plugin/artifactId -executions - execution -idbuild-classpath/id -phasegenerate-sources/phase -goals - goalbuild-classpath/goal -/goals -configuration - outputFiletarget/classes/mrapp-generated-classpath/outputFile -/configuration - /execution -/executions - /plugin - plugin groupIdorg.apache.maven.plugins/groupId artifactIdmaven-enforcer-plugin/artifactId inheritedfalse/inherited
svn commit: r1411359 - in /hadoop/common/trunk/hadoop-common-project/hadoop-common: CHANGES.txt src/main/docs/src/documentation/content/xdocs/cluster_setup.xml
Author: jghoman Date: Mon Nov 19 19:27:00 2012 New Revision: 1411359 URL: http://svn.apache.org/viewvc?rev=1411359view=rev Log: HADOOP-6616. Improve documentation for rack awareness. Contributed by Adam Faris. Modified: hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/docs/src/documentation/content/xdocs/cluster_setup.xml Modified: hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt URL: http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt?rev=1411359r1=1411358r2=1411359view=diff == --- hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt (original) +++ hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt Mon Nov 19 19:27:00 2012 @@ -132,6 +132,8 @@ Trunk (Unreleased) HADOOP-9004. Allow security unit tests to use external KDC. (Stephen Chu via suresh) +HADOOP-6616. Improve documentation for rack awareness. (Adam Faris via jghoman) + BUG FIXES HADOOP-8177. MBeans shouldn't try to register when it fails to create MBeanName. Modified: hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/docs/src/documentation/content/xdocs/cluster_setup.xml URL: http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/docs/src/documentation/content/xdocs/cluster_setup.xml?rev=1411359r1=1411358r2=1411359view=diff == --- hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/docs/src/documentation/content/xdocs/cluster_setup.xml (original) +++ hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/docs/src/documentation/content/xdocs/cluster_setup.xml Mon Nov 19 19:27:00 2012 @@ -1292,23 +1292,139 @@ section titleHadoop Rack Awareness/title - pThe HDFS and the Map/Reduce components are rack-aware./p - pThe codeNameNode/code and the codeJobTracker/code obtains the - coderack id/code of the slaves in the cluster by invoking an API - a href=ext:api/org/apache/hadoop/net/dnstoswitchmapping/resolve - resolve/a in an administrator configured - module. The API resolves the slave's DNS name (also IP address) to a - rack id. What module to use can be configured using the configuration - item codenet.topology.node.switch.mapping.impl/code. The default - implementation of the same runs a script/command configured using - codenet.topology.script.file.name/code. If topology.script.file.name is - not set, the rack id code/default-rack/code is returned for any - passed IP address. The additional configuration in the Map/Reduce - part is codemapred.cache.task.levels/code which determines the number - of levels (in the network topology) of caches. So, for example, if it is - the default value of 2, two levels of caches will be constructed - - one for hosts (host - task mapping) and another for racks - (rack - task mapping). + p + Both HDFS and Map/Reduce components are rack-aware. HDFS block placement will use rack + awareness for fault tolerance by placing one block replica on a different rack. This provides + data availability in the event of a network switch failure within the cluster. The jobtracker uses rack + awareness to reduce network transfers of HDFS data blocks by attempting to schedule tasks on datanodes with a local + copy of needed HDFS blocks. If the tasks cannot be scheduled on the datanodes + containing the needed HDFS blocks, then the tasks will be scheduled on the same rack to reduce network transfers if possible. + /p + pThe NameNode and the JobTracker obtain the rack id of the cluster slaves by invoking either + an external script or java class as specified by configuration files. Using either the + java class or external script for topology, output must adhere to the java + a href=ext:api/org/apache/hadoop/net/dnstoswitchmapping/resolveDNSToSwitchMapping/a + interface. The interface expects a one-to-one correspondence to be maintained + and the topology information in the format of '/myrack/myhost', where '/' is the topology + delimiter, 'myrack' is the rack identifier, and 'myhost' is the individual host. Assuming + a single /24 subnet per rack, one could use the format of '/192.168.100.0/192.168.100.5' as a + unique rack-host topology mapping. + /p + p + To use the java class for topology mapping, the class name is specified by the + code'topology.node.switch.mapping.impl'/code parameter in the configuration file. + An example, NetworkTopology.java, is included with the hadoop
svn commit: r1411429 - /hadoop/common/branches/branch-0.23.5/
Author: tgraves Date: Mon Nov 19 21:33:35 2012 New Revision: 1411429 URL: http://svn.apache.org/viewvc?rev=1411429view=rev Log: Remove branch-0.23.5. It is going to be re-branched to pick up a couple more jira. Removed: hadoop/common/branches/branch-0.23.5/
svn commit: r1411435 - /hadoop/common/branches/branch-0.23.5/
Author: tgraves Date: Mon Nov 19 21:38:34 2012 New Revision: 1411435 URL: http://svn.apache.org/viewvc?rev=1411435view=rev Log: Branching 0.23.5 release Added: hadoop/common/branches/branch-0.23.5/ (props changed) - copied from r1411434, hadoop/common/branches/branch-0.23/ Propchange: hadoop/common/branches/branch-0.23.5/ -- --- svn:ignore (added) +++ svn:ignore Mon Nov 19 21:38:34 2012 @@ -0,0 +1,5 @@ +.classpath +.git +.project +.settings +target Propchange: hadoop/common/branches/branch-0.23.5/ -- --- svn:mergeinfo (added) +++ svn:mergeinfo Mon Nov 19 21:38:34 2012 @@ -0,0 +1 @@ +/hadoop/common/trunk:1161777,1161781,1162188,1162421,1162491,1162499,1162613,1162928,1162954,1162979,1163050,1163069,1163490,1163768,1163852,1163858,1163981,1164255,1164301,1164339,1166009,1166402,1167001,1167383,1167662,1170085,1170379,1170459,1171297,1172916,1173402,1176550,1177487,1177531,1177859,1177864,1182189,1182205,1182214,1189613,1189932,1189982,1195575,1196113,1196129,1204114,1204117,1204122,1204124,1204129,1204131,1204177,1204370,1204376,1204388,1205260,1205697,1206786,1206830,1207694,1208153,1208313,1212021,1212062,1212073,1212084,1213537,1213586,1213592-1213593,1213954,1214046,1220510,1221348,1225114,1225192,1225456,1225489,1225591,1226211,1226239,1226350,1227091,1227165,1227423,1227964,1229347,1230398,1231569,1231572,1231627,1231640,1233605,1234555,1235135,1235137,1235956,1236456,1239752,1240897,1240928,1243065,1243104,1244766,1245751,1245762,1293419,1304099,1351818,1373683
svn commit: r1411441 - in /hadoop/common/branches/branch-0.23.5: ./ hadoop-assemblies/ hadoop-client/ hadoop-dist/ hadoop-minicluster/ hadoop-project-dist/ hadoop-project/ hadoop-tools/ hadoop-tools/h
Author: tgraves Date: Mon Nov 19 21:53:12 2012 New Revision: 1411441 URL: http://svn.apache.org/viewvc?rev=1411441view=rev Log: Preparing 0.23.5 release Modified: hadoop/common/branches/branch-0.23.5/hadoop-assemblies/pom.xml hadoop/common/branches/branch-0.23.5/hadoop-client/pom.xml hadoop/common/branches/branch-0.23.5/hadoop-dist/pom.xml hadoop/common/branches/branch-0.23.5/hadoop-minicluster/pom.xml hadoop/common/branches/branch-0.23.5/hadoop-project-dist/pom.xml hadoop/common/branches/branch-0.23.5/hadoop-project/pom.xml hadoop/common/branches/branch-0.23.5/hadoop-tools/hadoop-archives/pom.xml hadoop/common/branches/branch-0.23.5/hadoop-tools/hadoop-datajoin/pom.xml hadoop/common/branches/branch-0.23.5/hadoop-tools/hadoop-distcp/pom.xml hadoop/common/branches/branch-0.23.5/hadoop-tools/hadoop-extras/pom.xml hadoop/common/branches/branch-0.23.5/hadoop-tools/hadoop-gridmix/pom.xml hadoop/common/branches/branch-0.23.5/hadoop-tools/hadoop-pipes/pom.xml hadoop/common/branches/branch-0.23.5/hadoop-tools/hadoop-rumen/pom.xml hadoop/common/branches/branch-0.23.5/hadoop-tools/hadoop-streaming/pom.xml hadoop/common/branches/branch-0.23.5/hadoop-tools/hadoop-tools-dist/pom.xml hadoop/common/branches/branch-0.23.5/hadoop-tools/pom.xml hadoop/common/branches/branch-0.23.5/pom.xml Modified: hadoop/common/branches/branch-0.23.5/hadoop-assemblies/pom.xml URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.23.5/hadoop-assemblies/pom.xml?rev=1411441r1=1411440r2=1411441view=diff == --- hadoop/common/branches/branch-0.23.5/hadoop-assemblies/pom.xml (original) +++ hadoop/common/branches/branch-0.23.5/hadoop-assemblies/pom.xml Mon Nov 19 21:53:12 2012 @@ -23,12 +23,12 @@ parent groupIdorg.apache.hadoop/groupId artifactIdhadoop-project/artifactId -version0.23.6-SNAPSHOT/version +version0.23.5/version relativePath../hadoop-project/relativePath /parent groupIdorg.apache.hadoop/groupId artifactIdhadoop-assemblies/artifactId - version0.23.6-SNAPSHOT/version + version0.23.5/version nameApache Hadoop Assemblies/name descriptionApache Hadoop Assemblies/description Modified: hadoop/common/branches/branch-0.23.5/hadoop-client/pom.xml URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.23.5/hadoop-client/pom.xml?rev=1411441r1=1411440r2=1411441view=diff == --- hadoop/common/branches/branch-0.23.5/hadoop-client/pom.xml (original) +++ hadoop/common/branches/branch-0.23.5/hadoop-client/pom.xml Mon Nov 19 21:53:12 2012 @@ -18,12 +18,12 @@ parent groupIdorg.apache.hadoop/groupId artifactIdhadoop-project/artifactId -version0.23.6-SNAPSHOT/version +version0.23.5/version relativePath../hadoop-project/relativePath /parent groupIdorg.apache.hadoop/groupId artifactIdhadoop-client/artifactId - version0.23.6-SNAPSHOT/version + version0.23.5/version packagingjar/packaging descriptionApache Hadoop Client/description Modified: hadoop/common/branches/branch-0.23.5/hadoop-dist/pom.xml URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.23.5/hadoop-dist/pom.xml?rev=1411441r1=1411440r2=1411441view=diff == --- hadoop/common/branches/branch-0.23.5/hadoop-dist/pom.xml (original) +++ hadoop/common/branches/branch-0.23.5/hadoop-dist/pom.xml Mon Nov 19 21:53:12 2012 @@ -20,12 +20,12 @@ parent groupIdorg.apache.hadoop/groupId artifactIdhadoop-project/artifactId -version0.23.6-SNAPSHOT/version +version0.23.5/version relativePath../hadoop-project/relativePath /parent groupIdorg.apache.hadoop/groupId artifactIdhadoop-dist/artifactId - version0.23.6-SNAPSHOT/version + version0.23.5/version descriptionApache Hadoop Distribution/description nameApache Hadoop Distribution/name packagingjar/packaging Modified: hadoop/common/branches/branch-0.23.5/hadoop-minicluster/pom.xml URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.23.5/hadoop-minicluster/pom.xml?rev=1411441r1=1411440r2=1411441view=diff == --- hadoop/common/branches/branch-0.23.5/hadoop-minicluster/pom.xml (original) +++ hadoop/common/branches/branch-0.23.5/hadoop-minicluster/pom.xml Mon Nov 19 21:53:12 2012 @@ -18,12 +18,12 @@ parent groupIdorg.apache.hadoop/groupId artifactIdhadoop-project/artifactId -version0.23.6-SNAPSHOT/version +version0.23.5/version relativePath../hadoop-project/relativePath /parent groupIdorg.apache.hadoop/groupId artifactIdhadoop-minicluster/artifactId - version0.23.6-SNAPSHOT/version + version0.23.5/version packagingjar/packaging descriptionApache Hadoop
svn commit: r1411470 [1/2] - /hadoop/common/branches/branch-0.23.5/hadoop-common-project/hadoop-common/src/main/docs/releasenotes.html
Author: tgraves Date: Mon Nov 19 22:41:55 2012 New Revision: 1411470 URL: http://svn.apache.org/viewvc?rev=1411470view=rev Log: Preparing 0.23.5 release Modified: hadoop/common/branches/branch-0.23.5/hadoop-common-project/hadoop-common/src/main/docs/releasenotes.html
