Re: [VOTE] Release Apache Hadoop 2.5.0 RC2
+1 (non-binding), also pending doc issue Akira found - Brought up pseudo-distributed cluster from downloaded tarball - Ran basic XAttr, ACL, snapshot commands successfully Thanks, Stephen On Sun, Aug 10, 2014 at 11:52 PM, Yongjun Zhang wrote: > +1 (non-binding), pending the document issue Akira found out > > + downloaded src tarball and did build > + deployed two pseudo-distributed clusters > + ran some sample mapreduce job on one cluster > + ran basic distcp test between the two clusters > > Thanks. > > --Yongjun > > On Sun, Aug 10, 2014 at 9:35 PM, Akira AJISAKA > > wrote: > > > -1 (non-binding) > > > > + downloaded tarball and deployed HDFS-HA cluster > > + confirmed HDFS Rolling Upgrade from 2.4.0 > > + ran some sample jobs > > - documents are missing in tarball > > > > In 2.4.1 release, the documents were deployed at share/doc/hadoop/ . > > Would you please re-create a tarball with adding -Pdoc option? > > > > Thanks, > > Akira > > > > (2014/08/11 9:12), Chen He wrote: > > > >> +1 (non-binding) > >> > >> download source code and build from it (OS is Fedora); > >> deploy to single server; > >> run word count and sleep job succeed. > >> > >> > >> > >> > >> On Sun, Aug 10, 2014 at 6:00 PM, Jason Lowe > > >> wrote: > >> > >> +1 (binding) > >>> > >>> - verified signatures and digests > >>> - built from source > >>> - deployed a single-node cluster > >>> - ran some sample jobs > >>> > >>> Jason > >>> > >>> > >>> On 08/06/2014 03:59 PM, Karthik Kambatla wrote: > >>> > >>> Hi folks, > > I have put together a release candidate (rc2) for Hadoop 2.5.0. > > The RC is available at: http://people.apache.org/~ > kasha/hadoop-2.5.0-RC2/ > The RC tag in svn is here: > > https://svn.apache.org/repos/asf/hadoop/common/tags/release-2.5.0-rc2/ > The maven artifacts are staged at: > https://repository.apache.org/content/repositories/ > orgapachehadoop-1009/ > > You can find my public key at: > http://svn.apache.org/repos/asf/hadoop/common/dist/KEYS > > Please try the release and vote. The vote will run for the now usual 5 > days. > > Thanks > > > > >>> > >> > > >
Re: [VOTE] Migration from subversion to git for version control
+1 (binding)
as this gets rolled out, I think we may also want define a branch naming
policy like feature/${JIRA}-text for features, as well as a policy for
deleting retired branches (tag the last commit then rm the branch)...then
we can cull some branches that are in SVN.
-steve
On 9 August 2014 03:57, Karthik Kambatla wrote:
> I have put together this proposal based on recent discussion on this topic.
>
> Please vote on the proposal. The vote runs for 7 days.
>
>1. Migrate from subversion to git for version control.
>2. Force-push to be disabled on trunk and branch-* branches. Applying
>changes from any of trunk/branch-* to any of branch-* should be through
>"git cherry-pick -x".
>3. Force-push on feature-branches is allowed. Before pulling in a
>feature, the feature-branch should be rebased on latest trunk and the
>changes applied to trunk through "git rebase --onto" or "git cherry-pick
>".
>4. Every time a feature branch is rebased on trunk, a tag that
>identifies the state before the rebase needs to be created (e.g.
>tag_feature_JIRA-2454_2014-08-07_rebase). These tags can be deleted once
>the feature is pulled into trunk and the tags are no longer useful.
>5. The relevance/use of tags stay the same after the migration.
>
> Thanks
> Karthik
>
> PS: Per Andrew Wang, this should be a "Adoption of New Codebase" kind of
> vote and will be Lazy 2/3 majority of PMC members.
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Hadoop small changes on hdfs
Hello, i am a german IF student. For my first studywork i need work with hdfs. For this reason i need your API for server site hdfs. this: org.apache.hadoop.hdfs.server. ... with best regards Eric Clausnitzer.
Re: [VOTE] Release Apache Hadoop 2.5.0 RC2
-1 (non-binding). * Downloaded source and verified signature. * Built from source. * Ran tests and some MR jobs. * Ran RM-HA with manual failover mode. - Documents are missing. Good catch, Akira. As he mentioned, the documents including javadocs are missing from the binary tar ball. We should include them. It only includes following docs: $ find . -name *.html ./share/doc/hadoop/httpfs/dependency-analysis.html ./share/doc/hadoop/httpfs/UsingHttpTools.html ./share/doc/hadoop/httpfs/project-reports.html ./share/doc/hadoop/httpfs/ServerSetup.html ./share/doc/hadoop/httpfs/index.html ./share/hadoop/httpfs/tomcat/webapps/ROOT/index.html ./share/hadoop/hdfs/webapps/secondary/status.html ./share/hadoop/hdfs/webapps/secondary/index.html ./share/hadoop/hdfs/webapps/hdfs/dfshealth.html ./share/hadoop/hdfs/webapps/hdfs/explorer.html ./share/hadoop/hdfs/webapps/hdfs/index.html ./share/hadoop/hdfs/webapps/journal/index.html ./share/hadoop/hdfs/webapps/datanode/index.html ./share/hadoop/tools/sls/html/showSimulationTrace.html Karthik, could you create new tar ball with the documentations? Thanks, - Tsuyoshi On Thu, Aug 7, 2014 at 5:59 AM, Karthik Kambatla wrote: > Hi folks, > > I have put together a release candidate (rc2) for Hadoop 2.5.0. > > The RC is available at: http://people.apache.org/~kasha/hadoop-2.5.0-RC2/ > The RC tag in svn is here: > https://svn.apache.org/repos/asf/hadoop/common/tags/release-2.5.0-rc2/ > The maven artifacts are staged at: > https://repository.apache.org/content/repositories/orgapachehadoop-1009/ > > You can find my public key at: > http://svn.apache.org/repos/asf/hadoop/common/dist/KEYS > > Please try the release and vote. The vote will run for the now usual 5 > days. > > Thanks -- - Tsuyoshi
Re: [VOTE] Migration from subversion to git for version control
+1(non-binding) Thanks, - Tsuyoshi On Sat, Aug 9, 2014 at 11:57 AM, Karthik Kambatla wrote: > I have put together this proposal based on recent discussion on this topic. > > Please vote on the proposal. The vote runs for 7 days. > >1. Migrate from subversion to git for version control. >2. Force-push to be disabled on trunk and branch-* branches. Applying >changes from any of trunk/branch-* to any of branch-* should be through >"git cherry-pick -x". >3. Force-push on feature-branches is allowed. Before pulling in a >feature, the feature-branch should be rebased on latest trunk and the >changes applied to trunk through "git rebase --onto" or "git cherry-pick >". >4. Every time a feature branch is rebased on trunk, a tag that >identifies the state before the rebase needs to be created (e.g. >tag_feature_JIRA-2454_2014-08-07_rebase). These tags can be deleted once >the feature is pulled into trunk and the tags are no longer useful. >5. The relevance/use of tags stay the same after the migration. > > Thanks > Karthik > > PS: Per Andrew Wang, this should be a "Adoption of New Codebase" kind of > vote and will be Lazy 2/3 majority of PMC members. -- - Tsuyoshi
Re: [VOTE] Migration from subversion to git for version control
+1. Thanks, Junping On Sat, Aug 9, 2014 at 10:57 AM, Karthik Kambatla wrote: > I have put together this proposal based on recent discussion on this topic. > > Please vote on the proposal. The vote runs for 7 days. > >1. Migrate from subversion to git for version control. >2. Force-push to be disabled on trunk and branch-* branches. Applying >changes from any of trunk/branch-* to any of branch-* should be through >"git cherry-pick -x". >3. Force-push on feature-branches is allowed. Before pulling in a >feature, the feature-branch should be rebased on latest trunk and the >changes applied to trunk through "git rebase --onto" or "git cherry-pick >". >4. Every time a feature branch is rebased on trunk, a tag that >identifies the state before the rebase needs to be created (e.g. >tag_feature_JIRA-2454_2014-08-07_rebase). These tags can be deleted once >the feature is pulled into trunk and the tags are no longer useful. >5. The relevance/use of tags stay the same after the migration. > > Thanks > Karthik > > PS: Per Andrew Wang, this should be a "Adoption of New Codebase" kind of > vote and will be Lazy 2/3 majority of PMC members. > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [VOTE] Migration from subversion to git for version control
+1 (binding) Thanks for driving this, Karthik. -- Aaron T. Myers Software Engineer, Cloudera On Fri, Aug 8, 2014 at 7:57 PM, Karthik Kambatla wrote: > I have put together this proposal based on recent discussion on this topic. > > Please vote on the proposal. The vote runs for 7 days. > >1. Migrate from subversion to git for version control. >2. Force-push to be disabled on trunk and branch-* branches. Applying >changes from any of trunk/branch-* to any of branch-* should be through >"git cherry-pick -x". >3. Force-push on feature-branches is allowed. Before pulling in a >feature, the feature-branch should be rebased on latest trunk and the >changes applied to trunk through "git rebase --onto" or "git cherry-pick >". >4. Every time a feature branch is rebased on trunk, a tag that >identifies the state before the rebase needs to be created (e.g. >tag_feature_JIRA-2454_2014-08-07_rebase). These tags can be deleted once >the feature is pulled into trunk and the tags are no longer useful. >5. The relevance/use of tags stay the same after the migration. > > Thanks > Karthik > > PS: Per Andrew Wang, this should be a "Adoption of New Codebase" kind of > vote and will be Lazy 2/3 majority of PMC members. >
[jira] [Created] (HDFS-6838) Code cleanup for unnecessary INode replacement
Jing Zhao created HDFS-6838: --- Summary: Code cleanup for unnecessary INode replacement Key: HDFS-6838 URL: https://issues.apache.org/jira/browse/HDFS-6838 Project: Hadoop HDFS Issue Type: Bug Components: namenode Reporter: Jing Zhao Assignee: Jing Zhao With INode features we now no longer have INode replacement when converting a file to an under-construction/with-snapshot file or converting a directory to a snapshottable/with-snapshot directory. This jira plans to remove some unnecessary code that is only useful for INode replacement. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: [VOTE] Release Apache Hadoop 2.5.0 RC2
Thanks Akira for catching the missing docs. Let me work on *updating* the binary tarball to include the docs *without* modifying the source. Will send an update as soon as I sort that out. The vote itself is on the source and the binary is just a convenience. I hope we can let the vote continue so long as we don't change the source. Thanks Karthik On Mon, Aug 11, 2014 at 5:51 AM, Tsuyoshi OZAWA wrote: > -1 (non-binding). > > * Downloaded source and verified signature. > * Built from source. > * Ran tests and some MR jobs. > * Ran RM-HA with manual failover mode. > - Documents are missing. > > Good catch, Akira. As he mentioned, the documents including javadocs > are missing from the binary tar ball. We should include them. It only > includes following docs: > $ find . -name *.html > ./share/doc/hadoop/httpfs/dependency-analysis.html > ./share/doc/hadoop/httpfs/UsingHttpTools.html > ./share/doc/hadoop/httpfs/project-reports.html > ./share/doc/hadoop/httpfs/ServerSetup.html > ./share/doc/hadoop/httpfs/index.html > ./share/hadoop/httpfs/tomcat/webapps/ROOT/index.html > ./share/hadoop/hdfs/webapps/secondary/status.html > ./share/hadoop/hdfs/webapps/secondary/index.html > ./share/hadoop/hdfs/webapps/hdfs/dfshealth.html > ./share/hadoop/hdfs/webapps/hdfs/explorer.html > ./share/hadoop/hdfs/webapps/hdfs/index.html > ./share/hadoop/hdfs/webapps/journal/index.html > ./share/hadoop/hdfs/webapps/datanode/index.html > ./share/hadoop/tools/sls/html/showSimulationTrace.html > > Karthik, could you create new tar ball with the documentations? > > Thanks, > - Tsuyoshi > > On Thu, Aug 7, 2014 at 5:59 AM, Karthik Kambatla > wrote: > > Hi folks, > > > > I have put together a release candidate (rc2) for Hadoop 2.5.0. > > > > The RC is available at: > http://people.apache.org/~kasha/hadoop-2.5.0-RC2/ > > The RC tag in svn is here: > > https://svn.apache.org/repos/asf/hadoop/common/tags/release-2.5.0-rc2/ > > The maven artifacts are staged at: > > https://repository.apache.org/content/repositories/orgapachehadoop-1009/ > > > > You can find my public key at: > > http://svn.apache.org/repos/asf/hadoop/common/dist/KEYS > > > > Please try the release and vote. The vote will run for the now usual 5 > > days. > > > > Thanks > > > > -- > - Tsuyoshi >
[jira] [Created] (HDFS-6839) Fix TestCLI to expect new output
Charles Lamb created HDFS-6839: -- Summary: Fix TestCLI to expect new output Key: HDFS-6839 URL: https://issues.apache.org/jira/browse/HDFS-6839 Project: Hadoop HDFS Issue Type: Sub-task Affects Versions: fs-encryption (HADOOP-10150 and HDFS-6134) Reporter: Charles Lamb Assignee: Charles Lamb TestCLI is failing because HADOOP-10919 changed the output of the cp usage command. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: [VOTE] Release Apache Hadoop 2.5.0 RC2
Thank you for dealing with this problem, Karthik. It sounds reasonable to me to update binary tarball without modifying source code. +1(non-binding) to continue the voting. Thanks, - Tsuyoshi On Tue, Aug 12, 2014 at 3:47 AM, Karthik Kambatla wrote: > Thanks Akira for catching the missing docs. Let me work on *updating* the > binary tarball to include the docs *without* modifying the source. Will > send an update as soon as I sort that out. > > The vote itself is on the source and the binary is just a convenience. I > hope we can let the vote continue so long as we don't change the source. > > Thanks > Karthik > > > On Mon, Aug 11, 2014 at 5:51 AM, Tsuyoshi OZAWA > wrote: > >> -1 (non-binding). >> >> * Downloaded source and verified signature. >> * Built from source. >> * Ran tests and some MR jobs. >> * Ran RM-HA with manual failover mode. >> - Documents are missing. >> >> Good catch, Akira. As he mentioned, the documents including javadocs >> are missing from the binary tar ball. We should include them. It only >> includes following docs: >> $ find . -name *.html >> ./share/doc/hadoop/httpfs/dependency-analysis.html >> ./share/doc/hadoop/httpfs/UsingHttpTools.html >> ./share/doc/hadoop/httpfs/project-reports.html >> ./share/doc/hadoop/httpfs/ServerSetup.html >> ./share/doc/hadoop/httpfs/index.html >> ./share/hadoop/httpfs/tomcat/webapps/ROOT/index.html >> ./share/hadoop/hdfs/webapps/secondary/status.html >> ./share/hadoop/hdfs/webapps/secondary/index.html >> ./share/hadoop/hdfs/webapps/hdfs/dfshealth.html >> ./share/hadoop/hdfs/webapps/hdfs/explorer.html >> ./share/hadoop/hdfs/webapps/hdfs/index.html >> ./share/hadoop/hdfs/webapps/journal/index.html >> ./share/hadoop/hdfs/webapps/datanode/index.html >> ./share/hadoop/tools/sls/html/showSimulationTrace.html >> >> Karthik, could you create new tar ball with the documentations? >> >> Thanks, >> - Tsuyoshi >> >> On Thu, Aug 7, 2014 at 5:59 AM, Karthik Kambatla >> wrote: >> > Hi folks, >> > >> > I have put together a release candidate (rc2) for Hadoop 2.5.0. >> > >> > The RC is available at: >> http://people.apache.org/~kasha/hadoop-2.5.0-RC2/ >> > The RC tag in svn is here: >> > https://svn.apache.org/repos/asf/hadoop/common/tags/release-2.5.0-rc2/ >> > The maven artifacts are staged at: >> > https://repository.apache.org/content/repositories/orgapachehadoop-1009/ >> > >> > You can find my public key at: >> > http://svn.apache.org/repos/asf/hadoop/common/dist/KEYS >> > >> > Please try the release and vote. The vote will run for the now usual 5 >> > days. >> > >> > Thanks >> >> >> >> -- >> - Tsuyoshi >> -- - Tsuyoshi
Re: [VOTE] Migration from subversion to git for version control
+1 (binding) Kihwal On 8/8/14, 9:57 PM, "Karthik Kambatla" wrote: >I have put together this proposal based on recent discussion on this >topic. > >Please vote on the proposal. The vote runs for 7 days. > > 1. Migrate from subversion to git for version control. > 2. Force-push to be disabled on trunk and branch-* branches. Applying > changes from any of trunk/branch-* to any of branch-* should be through > "git cherry-pick -x". > 3. Force-push on feature-branches is allowed. Before pulling in a > feature, the feature-branch should be rebased on latest trunk and the > changes applied to trunk through "git rebase --onto" or "git >cherry-pick > ". > 4. Every time a feature branch is rebased on trunk, a tag that > identifies the state before the rebase needs to be created (e.g. > tag_feature_JIRA-2454_2014-08-07_rebase). These tags can be deleted >once > the feature is pulled into trunk and the tags are no longer useful. > 5. The relevance/use of tags stay the same after the migration. > >Thanks >Karthik > >PS: Per Andrew Wang, this should be a "Adoption of New Codebase" kind of >vote and will be Lazy 2/3 majority of PMC members.
Re: [VOTE] Migration from subversion to git for version control
+1 On Mon, Aug 11, 2014 at 1:43 PM, Kihwal Lee wrote: > +1 (binding) > > Kihwal > > On 8/8/14, 9:57 PM, "Karthik Kambatla" wrote: > > >I have put together this proposal based on recent discussion on this > >topic. > > > >Please vote on the proposal. The vote runs for 7 days. > > > > 1. Migrate from subversion to git for version control. > > 2. Force-push to be disabled on trunk and branch-* branches. Applying > > changes from any of trunk/branch-* to any of branch-* should be through > > "git cherry-pick -x". > > 3. Force-push on feature-branches is allowed. Before pulling in a > > feature, the feature-branch should be rebased on latest trunk and the > > changes applied to trunk through "git rebase --onto" or "git > >cherry-pick > > ". > > 4. Every time a feature branch is rebased on trunk, a tag that > > identifies the state before the rebase needs to be created (e.g. > > tag_feature_JIRA-2454_2014-08-07_rebase). These tags can be deleted > >once > > the feature is pulled into trunk and the tags are no longer useful. > > 5. The relevance/use of tags stay the same after the migration. > > > >Thanks > >Karthik > > > >PS: Per Andrew Wang, this should be a "Adoption of New Codebase" kind of > >vote and will be Lazy 2/3 majority of PMC members. > > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [VOTE] Migration from subversion to git for version control
+1. best, Colin On Fri, Aug 8, 2014 at 7:57 PM, Karthik Kambatla wrote: > I have put together this proposal based on recent discussion on this topic. > > Please vote on the proposal. The vote runs for 7 days. > >1. Migrate from subversion to git for version control. >2. Force-push to be disabled on trunk and branch-* branches. Applying >changes from any of trunk/branch-* to any of branch-* should be through >"git cherry-pick -x". >3. Force-push on feature-branches is allowed. Before pulling in a >feature, the feature-branch should be rebased on latest trunk and the >changes applied to trunk through "git rebase --onto" or "git cherry-pick >". >4. Every time a feature branch is rebased on trunk, a tag that >identifies the state before the rebase needs to be created (e.g. >tag_feature_JIRA-2454_2014-08-07_rebase). These tags can be deleted once >the feature is pulled into trunk and the tags are no longer useful. >5. The relevance/use of tags stay the same after the migration. > > Thanks > Karthik > > PS: Per Andrew Wang, this should be a "Adoption of New Codebase" kind of > vote and will be Lazy 2/3 majority of PMC members.
Re: [VOTE] Migration from subversion to git for version control
+1 From: Colin McCabe Sent: Monday, August 11, 2014 3:40 PM To: Hadoop Common Cc: hdfs-dev@hadoop.apache.org, yarn-...@hadoop.apache.org, mapreduce-...@hadoop.apache.org +1. best, Colin On Fri, Aug 8, 2014 at 7:57 PM, Karthik Kambatla wrote: > I have put together this proposal based on recent discussion on this topic. > > Please vote on the proposal. The vote runs for 7 days. > >1. Migrate from subversion to git for version control. >2. Force-push to be disabled on trunk and branch-* branches. Applying >changes from any of trunk/branch-* to any of branch-* should be through >"git cherry-pick -x". >3. Force-push on feature-branches is allowed. Before pulling in a >feature, the feature-branch should be rebased on latest trunk and the >changes applied to trunk through "git rebase --onto" or "git cherry-pick >". >4. Every time a feature branch is rebased on trunk, a tag that >identifies the state before the rebase needs to be created (e.g. >tag_feature_JIRA-2454_2014-08-07_rebase). These tags can be deleted once >the feature is pulled into trunk and the tags are no longer useful. >5. The relevance/use of tags stay the same after the migration. > > Thanks > Karthik > > PS: Per Andrew Wang, this should be a "Adoption of New Codebase" kind of > vote and will be Lazy 2/3 majority of PMC members.
Re: [VOTE] Migration from subversion to git for version control
+1 non-binding On Mon, Aug 11, 2014 at 4:16 PM, wrote: > +1 > > > > > > > From: Colin McCabe > Sent: Monday, August 11, 2014 3:40 PM > To: Hadoop Common > Cc: hdfs-dev@hadoop.apache.org, yarn-...@hadoop.apache.org, > mapreduce-...@hadoop.apache.org > > > > > > +1. > > best, > Colin > > On Fri, Aug 8, 2014 at 7:57 PM, Karthik Kambatla > wrote: > > I have put together this proposal based on recent discussion on this > topic. > > > > Please vote on the proposal. The vote runs for 7 days. > > > >1. Migrate from subversion to git for version control. > >2. Force-push to be disabled on trunk and branch-* branches. Applying > >changes from any of trunk/branch-* to any of branch-* should be > through > >"git cherry-pick -x". > >3. Force-push on feature-branches is allowed. Before pulling in a > >feature, the feature-branch should be rebased on latest trunk and the > >changes applied to trunk through "git rebase --onto" or "git > cherry-pick > >". > >4. Every time a feature branch is rebased on trunk, a tag that > >identifies the state before the rebase needs to be created (e.g. > >tag_feature_JIRA-2454_2014-08-07_rebase). These tags can be deleted > once > >the feature is pulled into trunk and the tags are no longer useful. > >5. The relevance/use of tags stay the same after the migration. > > > > Thanks > > Karthik > > > > PS: Per Andrew Wang, this should be a "Adoption of New Codebase" kind of > > vote and will be Lazy 2/3 majority of PMC members. > -- Zhijie Shen Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Created] (HDFS-6840) Clients are always sent to the same datanode when read is off rack
Jason Lowe created HDFS-6840: Summary: Clients are always sent to the same datanode when read is off rack Key: HDFS-6840 URL: https://issues.apache.org/jira/browse/HDFS-6840 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 2.5.0 Reporter: Jason Lowe Priority: Critical After HDFS-6268 the sorting order of block locations is deterministic for a given block and locality level (e.g.: local, rack. off-rack), so off-rack clients all see the same datanode for the same block. This leads to very poor behavior in distributed cache localization and other scenarios where many clients all want the same block data at approximately the same time. The one datanode is crushed by the load while the other replicas only handle local and rack-local requests. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: [VOTE] Release Apache Hadoop 2.5.0 RC2
I have updated the binary tar ball to include the docs, by building the docs locally and copying them over. Filed HADOOP-10956 to fix the create-release script to handle this. The RC is here: http://people.apache.org/~kasha/hadoop-2.5.0-RC2/ Please note that the binary tar ball is signed by a new gpg key, so please re-import keys. (I lost my original private key). The source tar ball, signature and checksum are untouched. On Mon, Aug 11, 2014 at 1:03 PM, Tsuyoshi OZAWA wrote: > Thank you for dealing with this problem, Karthik. It sounds reasonable > to me to update binary tarball without modifying source code. > +1(non-binding) to continue the voting. > > Thanks, > - Tsuyoshi > > On Tue, Aug 12, 2014 at 3:47 AM, Karthik Kambatla > wrote: > > Thanks Akira for catching the missing docs. Let me work on *updating* the > > binary tarball to include the docs *without* modifying the source. Will > > send an update as soon as I sort that out. > > > > The vote itself is on the source and the binary is just a convenience. I > > hope we can let the vote continue so long as we don't change the source. > > > > Thanks > > Karthik > > > > > > On Mon, Aug 11, 2014 at 5:51 AM, Tsuyoshi OZAWA < > ozawa.tsuyo...@gmail.com> > > wrote: > > > >> -1 (non-binding). > >> > >> * Downloaded source and verified signature. > >> * Built from source. > >> * Ran tests and some MR jobs. > >> * Ran RM-HA with manual failover mode. > >> - Documents are missing. > >> > >> Good catch, Akira. As he mentioned, the documents including javadocs > >> are missing from the binary tar ball. We should include them. It only > >> includes following docs: > >> $ find . -name *.html > >> ./share/doc/hadoop/httpfs/dependency-analysis.html > >> ./share/doc/hadoop/httpfs/UsingHttpTools.html > >> ./share/doc/hadoop/httpfs/project-reports.html > >> ./share/doc/hadoop/httpfs/ServerSetup.html > >> ./share/doc/hadoop/httpfs/index.html > >> ./share/hadoop/httpfs/tomcat/webapps/ROOT/index.html > >> ./share/hadoop/hdfs/webapps/secondary/status.html > >> ./share/hadoop/hdfs/webapps/secondary/index.html > >> ./share/hadoop/hdfs/webapps/hdfs/dfshealth.html > >> ./share/hadoop/hdfs/webapps/hdfs/explorer.html > >> ./share/hadoop/hdfs/webapps/hdfs/index.html > >> ./share/hadoop/hdfs/webapps/journal/index.html > >> ./share/hadoop/hdfs/webapps/datanode/index.html > >> ./share/hadoop/tools/sls/html/showSimulationTrace.html > >> > >> Karthik, could you create new tar ball with the documentations? > >> > >> Thanks, > >> - Tsuyoshi > >> > >> On Thu, Aug 7, 2014 at 5:59 AM, Karthik Kambatla > >> wrote: > >> > Hi folks, > >> > > >> > I have put together a release candidate (rc2) for Hadoop 2.5.0. > >> > > >> > The RC is available at: > >> http://people.apache.org/~kasha/hadoop-2.5.0-RC2/ > >> > The RC tag in svn is here: > >> > > https://svn.apache.org/repos/asf/hadoop/common/tags/release-2.5.0-rc2/ > >> > The maven artifacts are staged at: > >> > > https://repository.apache.org/content/repositories/orgapachehadoop-1009/ > >> > > >> > You can find my public key at: > >> > http://svn.apache.org/repos/asf/hadoop/common/dist/KEYS > >> > > >> > Please try the release and vote. The vote will run for the now usual 5 > >> > days. > >> > > >> > Thanks > >> > >> > >> > >> -- > >> - Tsuyoshi > >> > > > > -- > - Tsuyoshi >
Re: [VOTE] Release Apache Hadoop 2.5.0 RC2
Can someone please verify the signatures on the new binary and the old source tarballs to make sure it is all good? If it is, I believe we can go ahead and close the vote. On Mon, Aug 11, 2014 at 9:49 PM, Karthik Kambatla wrote: > I have updated the binary tar ball to include the docs, by building the > docs locally and copying them over. Filed HADOOP-10956 to fix the > create-release script to handle this. > > The RC is here: http://people.apache.org/~kasha/hadoop-2.5.0-RC2/ > > Please note that the binary tar ball is signed by a new gpg key, so please > re-import keys. (I lost my original private key). The source tar ball, > signature and checksum are untouched. > > > > On Mon, Aug 11, 2014 at 1:03 PM, Tsuyoshi OZAWA > wrote: > >> Thank you for dealing with this problem, Karthik. It sounds reasonable >> to me to update binary tarball without modifying source code. >> +1(non-binding) to continue the voting. >> >> Thanks, >> - Tsuyoshi >> >> On Tue, Aug 12, 2014 at 3:47 AM, Karthik Kambatla >> wrote: >> > Thanks Akira for catching the missing docs. Let me work on *updating* >> the >> > binary tarball to include the docs *without* modifying the source. Will >> > send an update as soon as I sort that out. >> > >> > The vote itself is on the source and the binary is just a convenience. I >> > hope we can let the vote continue so long as we don't change the source. >> > >> > Thanks >> > Karthik >> > >> > >> > On Mon, Aug 11, 2014 at 5:51 AM, Tsuyoshi OZAWA < >> ozawa.tsuyo...@gmail.com> >> > wrote: >> > >> >> -1 (non-binding). >> >> >> >> * Downloaded source and verified signature. >> >> * Built from source. >> >> * Ran tests and some MR jobs. >> >> * Ran RM-HA with manual failover mode. >> >> - Documents are missing. >> >> >> >> Good catch, Akira. As he mentioned, the documents including javadocs >> >> are missing from the binary tar ball. We should include them. It only >> >> includes following docs: >> >> $ find . -name *.html >> >> ./share/doc/hadoop/httpfs/dependency-analysis.html >> >> ./share/doc/hadoop/httpfs/UsingHttpTools.html >> >> ./share/doc/hadoop/httpfs/project-reports.html >> >> ./share/doc/hadoop/httpfs/ServerSetup.html >> >> ./share/doc/hadoop/httpfs/index.html >> >> ./share/hadoop/httpfs/tomcat/webapps/ROOT/index.html >> >> ./share/hadoop/hdfs/webapps/secondary/status.html >> >> ./share/hadoop/hdfs/webapps/secondary/index.html >> >> ./share/hadoop/hdfs/webapps/hdfs/dfshealth.html >> >> ./share/hadoop/hdfs/webapps/hdfs/explorer.html >> >> ./share/hadoop/hdfs/webapps/hdfs/index.html >> >> ./share/hadoop/hdfs/webapps/journal/index.html >> >> ./share/hadoop/hdfs/webapps/datanode/index.html >> >> ./share/hadoop/tools/sls/html/showSimulationTrace.html >> >> >> >> Karthik, could you create new tar ball with the documentations? >> >> >> >> Thanks, >> >> - Tsuyoshi >> >> >> >> On Thu, Aug 7, 2014 at 5:59 AM, Karthik Kambatla >> >> wrote: >> >> > Hi folks, >> >> > >> >> > I have put together a release candidate (rc2) for Hadoop 2.5.0. >> >> > >> >> > The RC is available at: >> >> http://people.apache.org/~kasha/hadoop-2.5.0-RC2/ >> >> > The RC tag in svn is here: >> >> > >> https://svn.apache.org/repos/asf/hadoop/common/tags/release-2.5.0-rc2/ >> >> > The maven artifacts are staged at: >> >> > >> https://repository.apache.org/content/repositories/orgapachehadoop-1009/ >> >> > >> >> > You can find my public key at: >> >> > http://svn.apache.org/repos/asf/hadoop/common/dist/KEYS >> >> > >> >> > Please try the release and vote. The vote will run for the now usual >> 5 >> >> > days. >> >> > >> >> > Thanks >> >> >> >> >> >> >> >> -- >> >> - Tsuyoshi >> >> >> >> >> >> -- >> - Tsuyoshi >> > >
Re: [VOTE] Release Apache Hadoop 2.5.0 RC2
Thanks Karthik for the update, but the some documents are not included. It looks to me that *.apt.vm files are not compiled. Would you please generate documents with 'mvn site site:stage /path/to/deploy'? I'm +1 (non-binding) if the documents are included. Thanks, Akira (2014/08/12 13:49), Karthik Kambatla wrote: I have updated the binary tar ball to include the docs, by building the docs locally and copying them over. Filed HADOOP-10956 to fix the create-release script to handle this. The RC is here: http://people.apache.org/~kasha/hadoop-2.5.0-RC2/ Please note that the binary tar ball is signed by a new gpg key, so please re-import keys. (I lost my original private key). The source tar ball, signature and checksum are untouched. On Mon, Aug 11, 2014 at 1:03 PM, Tsuyoshi OZAWA wrote: Thank you for dealing with this problem, Karthik. It sounds reasonable to me to update binary tarball without modifying source code. +1(non-binding) to continue the voting. Thanks, - Tsuyoshi On Tue, Aug 12, 2014 at 3:47 AM, Karthik Kambatla wrote: Thanks Akira for catching the missing docs. Let me work on *updating* the binary tarball to include the docs *without* modifying the source. Will send an update as soon as I sort that out. The vote itself is on the source and the binary is just a convenience. I hope we can let the vote continue so long as we don't change the source. Thanks Karthik On Mon, Aug 11, 2014 at 5:51 AM, Tsuyoshi OZAWA < ozawa.tsuyo...@gmail.com> wrote: -1 (non-binding). * Downloaded source and verified signature. * Built from source. * Ran tests and some MR jobs. * Ran RM-HA with manual failover mode. - Documents are missing. Good catch, Akira. As he mentioned, the documents including javadocs are missing from the binary tar ball. We should include them. It only includes following docs: $ find . -name *.html ./share/doc/hadoop/httpfs/dependency-analysis.html ./share/doc/hadoop/httpfs/UsingHttpTools.html ./share/doc/hadoop/httpfs/project-reports.html ./share/doc/hadoop/httpfs/ServerSetup.html ./share/doc/hadoop/httpfs/index.html ./share/hadoop/httpfs/tomcat/webapps/ROOT/index.html ./share/hadoop/hdfs/webapps/secondary/status.html ./share/hadoop/hdfs/webapps/secondary/index.html ./share/hadoop/hdfs/webapps/hdfs/dfshealth.html ./share/hadoop/hdfs/webapps/hdfs/explorer.html ./share/hadoop/hdfs/webapps/hdfs/index.html ./share/hadoop/hdfs/webapps/journal/index.html ./share/hadoop/hdfs/webapps/datanode/index.html ./share/hadoop/tools/sls/html/showSimulationTrace.html Karthik, could you create new tar ball with the documentations? Thanks, - Tsuyoshi On Thu, Aug 7, 2014 at 5:59 AM, Karthik Kambatla wrote: Hi folks, I have put together a release candidate (rc2) for Hadoop 2.5.0. The RC is available at: http://people.apache.org/~kasha/hadoop-2.5.0-RC2/ The RC tag in svn is here: https://svn.apache.org/repos/asf/hadoop/common/tags/release-2.5.0-rc2/ The maven artifacts are staged at: https://repository.apache.org/content/repositories/orgapachehadoop-1009/ You can find my public key at: http://svn.apache.org/repos/asf/hadoop/common/dist/KEYS Please try the release and vote. The vote will run for the now usual 5 days. Thanks -- - Tsuyoshi -- - Tsuyoshi
