Re: [VOTE] Release Apache Hadoop 2.6.2

2015-10-27 Thread Hitesh Shah 
+1 (binding)

sha512 checksum is missing in the staging artifacts.

Compiled locally and ran a few example jobs. 

thanks
— Hitesh

On Oct 22, 2015, at 2:14 PM, Sangjin Lee  wrote:

> Hi all,
> 
> I have created a release candidate (RC0) for Hadoop 2.6.2.
> 
> The RC is available at: http://people.apache.org/~sjlee/hadoop-2.6.2-RC0/
> 
> The RC tag in git is: release-2.6.2-RC0
> 
> The list of JIRAs committed for 2.6.2:
> https://issues.apache.org/jira/browse/YARN-4101?jql=project%20in%20(HADOOP%2C%20HDFS%2C%20YARN%2C%20MAPREDUCE)%20AND%20fixVersion%20%3D%202.6.2
> 
> The maven artifacts are staged at
> https://repository.apache.org/content/repositories/orgapachehadoop-1022/
> 
> Please try out the release candidate and vote. The vote will run for 5 days.
> 
> Thanks,
> Sangjin

Re: Github integration for Hadoop

2015-10-29 Thread Hitesh Shah 
+1 on supporting patch contributions through github pull requests.

— Hitesh

On Oct 29, 2015, at 10:47 AM, Owen O'Malley  wrote:

> All,
>   For code & patch review, many of the newer projects are using the Github
> pull request integration. You can read about it here:
> 
> https://blogs.apache.org/infra/entry/improved_integration_between_apache_and
> 
> It basically lets you:
> * have mirroring between comments on pull requests and jira
> * lets you close pull requests
> * have mirroring between pull request comments and the Apache mail lists
> 
> Thoughts?
> .. Owen

Re: [VOTE] Release Apache Hadoop 2.7.2 RC2

2016-01-21 Thread Hitesh Shah 
+1 (binding)

Did the basic checks and ran against a single node local setup. 

— Hitesh 
 
On Jan 14, 2016, at 8:57 PM, Vinod Kumar Vavilapalli  wrote:

> Hi all,
> 
> I've created an updated release candidate RC2 for Apache Hadoop 2.7.2.
> 
> As discussed before, this is the next maintenance release to follow up 2.7.1.
> 
> The RC is available for validation at: 
> http://people.apache.org/~vinodkv/hadoop-2.7.2-RC2/
> 
> The RC tag in git is: release-2.7.2-RC2
> 
> The maven artifacts are available via repository.apache.org 
>  at 
> https://repository.apache.org/content/repositories/orgapachehadoop-1027 
> 
> 
> The release-notes are inside the tar-balls at location 
> hadoop-common-project/hadoop-common/src/main/docs/releasenotes.html. I hosted 
> this at http://people.apache.org/~vinodkv/hadoop-2.7.2-RC2/releasenotes.html 
>  for 
> your quick perusal.
> 
> As you may have noted,
> - I terminated the RC1 related voting thread after finding out that we didn’t 
> have a bunch of patches that are already in the released 2.6.3 version. After 
> a brief discussion, we decided to keep the parallel 2.6.x and 2.7.x releases 
> incremental, see [4] for this discussion.
> - The RC0 related voting thread got halted due to some critical issues. It 
> took a while again for getting all those blockers out of the way. See the 
> previous voting thread [3] for details.
> - Before RC0, an unusually long 2.6.3 release caused 2.7.2 to slip by quite a 
> bit. This release's related discussion threads are linked below: [1] and [2].
> 
> Please try the release and vote; the vote will run for the usual 5 days.
> 
> Thanks,
> Vinod
> 
> [1]: 2.7.2 release plan: http://markmail.org/message/oozq3gvd4nhzsaes 
> 
> [2]: Planning Apache Hadoop 2.7.2 
> http://markmail.org/message/iktqss2qdeykgpqk 
> 
> [3]: [VOTE] Release Apache Hadoop 2.7.2 RC0: 
> http://markmail.org/message/5txhvr2qdiqglrwc 
> 
> [4] Retracted [VOTE] Release Apache Hadoop 2.7.2 RC1: 
> http://markmail.org/thread/n7ljbsnquihn3wlw

[DISCUSS] Treating LimitedPrivate({"MapReduce"}) as Public APIs for YARN applications

2016-05-10 Thread Hitesh Shah 
There have been various discussions on various JIRAs where upstream projects 
such as YARN apps ( Tez, Slider, etc ) are called out for using the above 
so-called Private APIs. A lot of YARN applications that have been built out 
have picked up various bits and pieces of implementation from MapReduce and 
DistributedShell to get things to work.

A recent example is a backward incompatible change introduced ( where the API 
is not even directly invoked ) in the Credentials class related to the ability 
to read tokens/credentials from a file. This functionality is required by 
pretty much everyone as YARN provides the credentials to the app by writing the 
credentials/tokens to a local file which is read in when 
UserGroupInformation.getCurrentUser() is invoked.  This change breaks rolling 
upgrades for yarn applications from 2.x to 3.x (whether we end up supporting 
rolling upgrades across 2.x to 3.x is a separate discussion )

I would like to change our compatibility docs to state that any API that is 
marked as LimitedPrivate{Mapreduce} implies LimitedPrivate{YARN Applications}.

Comments/concerns? 

thanks
— Hitesh
-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: [DISCUSS] Treating LimitedPrivate({"MapReduce"}) as Public APIs for YARN applications

2016-05-10 Thread Hitesh Shah 
There seems to be some incorrect assumptions on why the application had an 
issue. For rolling upgrade deployments, the application bundles the client-side 
jars that it was compiled against and uses them in its classpath and expects to 
be able to communicate with upgraded servers. Given that hadoop-common is a 
monolithic jar, it ends up being used on both client-side and server-side. The 
problem in this case was caused by the fact that the ResourceManager was 
generating the credentials file with a format understood only by hadoop-common 
from 3.x. For an application compiled against 2.x and has *only* hadoop-common 
from 2.x on its classpath, trying to read this file fails.  

This is not about whether internal implementations can change for non-public 
APIs. The file format for the Credential file in this scenario is *not* 
internal implementation especially when you can have different versions of the 
library trying to read the file. If an older client is talking to a newer 
versioned server, the general backward compat assumption is that the client 
should receive a response that it can parse and understand. In this scenario, 
the credentials file provided to the YARN app by the RM should have been 
written out with the older version or at the very least been readable by the 
older hadoop-common.jar.

In any case, does anyone have any specific concerns with changing 
LimitedPrivate({"MapReduce"}) to Public?

And sure, if we are saying that Hadoop-3.x requires all apps built against it 
to go through a full re-compile as well as downtime as existing apps may no 
longer work out of the box, lets call it out very explicitly in the Release 
notes. 

— Hitesh

> On May 10, 2016, at 9:24 AM, Allen Wittenauer  
> wrote:
> 
> 
>> On May 10, 2016, at 8:37 AM, Hitesh Shah  wrote:
>> 
>> There have been various discussions on various JIRAs where upstream projects 
>> such as YARN apps ( Tez, Slider, etc ) are called out for using the above 
>> so-called Private APIs. A lot of YARN applications that have been built out 
>> have picked up various bits and pieces of implementation from MapReduce and 
>> DistributedShell to get things to work.
>> 
>> A recent example is a backward incompatible change introduced ( where the 
>> API is not even directly invoked ) in the Credentials class related to the 
>> ability to read tokens/credentials from a file.
> 
>   Let’s be careful here.  It should be noted that the problem happened 
> primarily because the application jar appears to have included some hadoop 
> jars in them.   So the API invocation isn’t the problem:  it’s the fact that 
> the implementation under the hood changed.  If the application jar didn’t 
> bundle hadoop jars —especially given that were already on the classpath--this 
> problem should never have happened.
> 
>> This functionality is required by pretty much everyone as YARN provides the 
>> credentials to the app by writing the credentials/tokens to a local file 
>> which is read in when UserGroupInformation.getCurrentUser() is invoked.
> 
>   What you’re effectively arguing is that implementations should never 
> change for public (and in this case LimitedPrivate) APIs.  I don’t think 
> that’s reasonable.  Hadoop is filled with changes in major branches where the 
> implementations have changed but the internals have been reworked to perform 
> the work in a slightly different manner.
> 
>> This change breaks rolling upgrades for yarn applications from 2.x to 3.x 
>> (whether we end up supporting rolling upgrades across 2.x to 3.x is a 
>> separate discussion )
> 
>   
>   At least today, according to the document attached to YARN-666 (lol), 
> rolling upgrades are only supported within the same major version.  
> 
>> 
>> I would like to change our compatibility docs to state that any API that is 
>> marked as LimitedPrivate{Mapreduce} implies LimitedPrivate{YARN 
>> Applications}.
>> 
>> Comments/concerns? 
> 
> 
>   a)  That isn’t good enough.  No one reads the compatibility guidelines 
> as it is given how many times someone says “X” isn’t covered when it clearly 
> is.
> 
>   b) LimitedPrivate{“YARN Applications”} makes zero sense.  At that point 
> it’s Public and the source should be changed to reflect that.  Especially 
> given those flags impacts things like how the javadocs are generated.


-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: development environment for hadoop core

2013-01-16 Thread Hitesh Shah 

On Jan 16, 2013, at 6:17 AM, Gopal Vijayaraghavan wrote:

> So, this is a question I have for everyone else.
> 
> How do I change the hadoop version of an entire build, so that I can
> name it something unique & use it in other builds in maven (-SNAPSHOT
> doesn't cut it, since occasionally mvn will download the hadoop snap
> poms from the remote repos).
> 

The following should work: ( from 
http://wiki.apache.org/hadoop/HowToReleasePostMavenization)

$ export version=3.0.0-TEST1
$ mvn versions:set -DnewVersion=${version}

-- Hitesh

Re: [VOTE] Release hadoop-2.0.3-alpha

2013-02-07 Thread Hitesh Shah 
+1 (non-binding) 

Downloaded ( verified checksums ) and built from source, deployed and 
successfully ran both MR and distributed shell examples. 

-- Hitesh

On Feb 6, 2013, at 7:59 PM, Arun C Murthy wrote:

> Folks,
> 
> I've created a release candidate (rc0) for hadoop-2.0.3-alpha that I would 
> like to release.
> 
> This release contains several major enhancements such as QJM for HDFS HA, 
> multi-resource scheduling for YARN, YARN ResourceManager restart etc.  
> Also YARN has achieved significant stability at scale (more details from Y! 
> folks here: http://s.apache.org/VYO).
> 
> The RC is available at: 
> http://people.apache.org/~acmurthy/hadoop-2.0.3-alpha-rc0/
> The RC tag in svn is here: 
> http://svn.apache.org/viewvc/hadoop/common/tags/release-2.0.3-alpha-rc0/
> 
> The maven artifacts are available via repository.apache.org.
> 
> Please try the release and vote; the vote will run for the usual 7 days.
> 
> thanks,
> Arun
> 
> 
> 
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
> 
>

Re: [VOTE] Release Apache Hadoop 0.23.7

2013-04-16 Thread Hitesh Shah 
+1. 

Downloaded source, built and ran a couple of sample jobs on a single node 
cluster. 

-- Hitesh

On Apr 11, 2013, at 12:55 PM, Thomas Graves wrote:

> I've created a release candidate (RC0) for hadoop-0.23.7 that I would like
> to release.
> 
> This release is a sustaining release with several important bug fixes in
> it.
> 
> The RC is available at:
> http://people.apache.org/~tgraves/hadoop-0.23.7-candidate-0/
> The RC tag in svn is here:
> http://svn.apache.org/viewvc/hadoop/common/tags/release-0.23.7-rc0/
> 
> The maven artifacts are available via repository.apache.org.
> 
> Please try the release and vote; the vote will run for the usual 7 days.
> 
> thanks,
> Tom Graves
>

Re: [VOTE] Release Apache Hadoop 2.0.4-alpha

2013-04-16 Thread Hitesh Shah 
+1 (non-binding).

Downloaded from source, tested distributed shell and simple MR jobs on a 
single-node cluster. 

-- Hitesh

On Apr 12, 2013, at 2:56 PM, Arun C Murthy wrote:

> Folks,
> 
> I've created a release candidate (RC2) for hadoop-2.0.4-alpha that I would 
> like to release.
> 
> The RC is available at: 
> http://people.apache.org/~acmurthy/hadoop-2.0.4-alpha-rc2/
> The RC tag in svn is here: 
> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.4-alpha-rc2
> 
> The maven artifacts are available via repository.apache.org.
> 
> Please try the release and vote; the vote will run for the usual 7 days.
> 
> thanks,
> Arun
> 
> 
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
> 
>

Re: [VOTE] change in bylaws to remove Release Plan vote

2013-05-21 Thread Hitesh Shah 
+1. 

-- Hitesh 

On May 21, 2013, at 7:03 PM, Matt Foley wrote:

> This was previously discussed in the thread "[PROPOSAL] change in bylaws to
> remove Release Plan vote".  13 people explicitly cast "+1"s in that thread.
> Absent objection I will count those as votes without requiring them to
> (re-)respond to this VOTE thread.
> 
> The following change is proposed in the
> Bylaws
> document:
> 
> ===
> 
> 1. In the "Decision Making" : "Actions" section of the Bylaws, the
> following text is removed:
> 
> ** Release Plan*
> 
> Defines the timetable and actions for a release. The plan also nominates a
> Release Manager.
> 
> Lazy majority of active committers
> 
> 
> 2. In the "Roles and Responsibilities" section of the Bylaws, an additional
> role is defined:
> 
> ** Release Manager*
> 
> A Release Manager (RM) is a committer who volunteers to produce a Release
> Candidate according to
> HowToRelease.
> The RM shall publish a Release Plan on the *common-dev@* list stating the
> branch from which they intend to make a Release Candidate, at least one
> week before they do so. The RM is responsible for building consensus around
> the content of the Release Candidate, in order to achieve a successful
> Product Release vote.
> 
> ===
> 
> Best regards,
> --Matt (long-time release manager)

Re: [VOTE] Release Apache Hadoop 0.23.8

2013-06-03 Thread Hitesh Shah 
+1 (non-binding)

Built from source, ran some basic tests on a single node cluster. 

-- Hitesh

On May 28, 2013, at 9:00 AM, Thomas Graves wrote:

> 
> I've created a release candidate (RC0) for hadoop-0.23.8 that I would like
> to release.
> 
> This release is a sustaining release with several important bug fixes in
> it.  The most critical one is MAPREDUCE-5211.
> 
> The RC is available at:
> http://people.apache.org/~tgraves/hadoop-0.23.8-candidate-0/
> The RC tag in svn is here:
> http://svn.apache.org/viewvc/hadoop/common/tags/release-0.23.8-rc0/
> 
> The maven artifacts are available via repository.apache.org.
> 
> Please try the release and vote; the vote will run for the usual 7 days.
> 
> I am +1 (binding).
> 
> thanks,
> Tom Graves
>

Re: [VOTE] Release Apache Hadoop 2.1.0-beta

2013-06-28 Thread Hitesh Shah 
Hi Arun, 

From a YARN perspective, YARN-791 and YARN-727 are 2 jiras that may potentially 
change the apis. They can implemented in a backward compat fashion if committed 
after 2.1.0. However, this will require adding of differently-named apis ( 
different urls in case of the webservices ) and make the current version of the 
api deprecated and/or obsolete. YARN-818 which is currently patch available 
also changes behavior.  

Assuming that as soon as 2.1.0 is released, we are to follow a very strict 
backward-compat retaining approach to all user-facing layers  ( 
api/webservices/rpc/... ) in common/hdfs/yarn/mapreduce, does it make sense to 
try and pull them in and roll out a new RC after they are ready? Perhaps Vinod 
can chime in if he is aware of any other such jiras under YARN-386 which should 
be considered compat-related blockers for a 2.1.0 RC. 

thanks
-- Hitesh

On Jun 26, 2013, at 1:17 AM, Arun C Murthy wrote:

> Folks,
> 
> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would 
> like to get released.
> 
> This release represents a *huge* amount of work done by the community (639 
> fixes) which includes several major advances including:
> # HDFS Snapshots
> # Windows support
> # YARN API stabilization
> # MapReduce Binary Compatibility with hadoop-1.x
> # Substantial amount of integration testing with rest of projects in the 
> ecosystem
> 
> The RC is available at: 
> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
> The RC tag in svn is here: 
> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
> 
> The maven artifacts are available via repository.apache.org.
> 
> Please try the release and vote; the vote will run for the usual 7 days.
> 
> thanks,
> Arun
> 
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
> 
>

Re: [VOTE] Release Apache Hadoop 2.1.0-beta

2013-08-21 Thread Hitesh Shah 
+1 (binding)

Verified checksums, built from source and ran basic MR jobs on a single-node 
cluster. 

-- Hitesh

On Aug 15, 2013, at 2:15 PM, Arun C Murthy wrote:

> Folks,
> 
> I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would 
> like to get released - this fixes the bugs we saw since the last go-around 
> (rc1).
> 
> The RC is available at: 
> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc2/
> The RC tag in svn is here: 
> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc2
> 
> The maven artifacts are available via repository.apache.org.
> 
> Please try the release and vote; the vote will run for the usual 7 days.
> 
> thanks,
> Arun
> 
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
> 
> 
> 
> -- 
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to 
> which it is addressed and may contain information that is confidential, 
> privileged and exempt from disclosure under applicable law. If the reader 
> of this message is not the intended recipient, you are hereby notified that 
> any printing, copying, dissemination, distribution, disclosure or 
> forwarding of this communication is strictly prohibited. If you have 
> received this communication in error, please contact the sender immediately 
> and delete it from your system. Thank You.

Re: [VOTE] Release Apache Hadoop 2.2.0

2013-10-13 Thread Hitesh Shah 
+1.

Built from source and ran distributed shell along with simple MR jobs on a 
single node cluster. 

-- HItesh 

On Oct 7, 2013, at 12:00 AM, Arun C Murthy wrote:

> Folks,
> 
> I've created a release candidate (rc0) for hadoop-2.2.0 that I would like to 
> get released - this release fixes a small number of bugs and some 
> protocol/api issues which should ensure they are now stable and will not 
> change in hadoop-2.x.
> 
> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.2.0-rc0
> The RC tag in svn is here: 
> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.2.0-rc0
> 
> The maven artifacts are available via repository.apache.org.
> 
> Please try the release and vote; the vote will run for the usual 7 days.
> 
> thanks,
> Arun
> 
> P.S.: Thanks to Colin, Andrew, Daryn, Chris and others for helping nail down 
> the symlinks-related issues. I'll release note the fact that we have disabled 
> it in 2.2. Also, thanks to Vinod for some heavy-lifting on the YARN side in 
> the last couple of weeks.
> 
> 
> 
> 
> 
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
> 
> 
> 
> -- 
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to 
> which it is addressed and may contain information that is confidential, 
> privileged and exempt from disclosure under applicable law. If the reader 
> of this message is not the intended recipient, you are hereby notified that 
> any printing, copying, dissemination, distribution, disclosure or 
> forwarding of this communication is strictly prohibited. If you have 
> received this communication in error, please contact the sender immediately 
> and delete it from your system. Thank You.

Re: [VOTE] Change by-laws on release votes: 5 days instead of 7

2014-06-24 Thread Hitesh Shah 
+1 (binding) 

— Hitesh 

On Jun 24, 2014, at 1:53 AM, Arun C Murthy  wrote:

> Folks,
> 
> As discussed, I'd like to call a vote on changing our by-laws to change 
> release votes from 7 days to 5.
> 
> I've attached the change to by-laws I'm proposing.
> 
> Please vote, the vote will the usual period of 7 days.
> 
> thanks,
> Arun
> 
> 
> 
> [main]$ svn diff
> Index: author/src/documentation/content/xdocs/bylaws.xml
> ===
> --- author/src/documentation/content/xdocs/bylaws.xml   (revision 1605015)
> +++ author/src/documentation/content/xdocs/bylaws.xml   (working copy)
> @@ -344,7 +344,16 @@
> Votes are open for a period of 7 days to allow all active
> voters time to consider the vote. Votes relating to code
> changes are not subject to a strict timetable but should be
> -made as timely as possible.
> +made as timely as possible.
> +
> + 
> +  Product Release - Vote Timeframe
> +   Release votes, alone, run for a period of 5 days. All other
> + votes are subject to the above timeframe of 7 days.
> + 
> +   
> +   
> +
>
>
> 
> -- 
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to 
> which it is addressed and may contain information that is confidential, 
> privileged and exempt from disclosure under applicable law. If the reader 
> of this message is not the intended recipient, you are hereby notified that 
> any printing, copying, dissemination, distribution, disclosure or 
> forwarding of this communication is strictly prohibited. If you have 
> received this communication in error, please contact the sender immediately 
> and delete it from your system. Thank You.

Re: [VOTE] Migration from subversion to git for version control

2014-08-14 Thread Hitesh Shah 
+1 

— Hitesh 

On Aug 8, 2014, at 7:57 PM, Karthik Kambatla  wrote:

> I have put together this proposal based on recent discussion on this topic.
> 
> Please vote on the proposal. The vote runs for 7 days.
> 
>   1. Migrate from subversion to git for version control.
>   2. Force-push to be disabled on trunk and branch-* branches. Applying
>   changes from any of trunk/branch-* to any of branch-* should be through
>   "git cherry-pick -x".
>   3. Force-push on feature-branches is allowed. Before pulling in a
>   feature, the feature-branch should be rebased on latest trunk and the
>   changes applied to trunk through "git rebase --onto" or "git cherry-pick
>   ".
>   4. Every time a feature branch is rebased on trunk, a tag that
>   identifies the state before the rebase needs to be created (e.g.
>   tag_feature_JIRA-2454_2014-08-07_rebase). These tags can be deleted once
>   the feature is pulled into trunk and the tags are no longer useful.
>   5. The relevance/use of tags stay the same after the migration.
> 
> Thanks
> Karthik
> 
> PS: Per Andrew Wang, this should be a "Adoption of New Codebase" kind of
> vote and will be Lazy 2/3 majority of PMC members.

Re: [VOTE] Release Apache Hadoop 2.6.0

2014-11-14 Thread Hitesh Shah 
+1 (binding)

Built Hadoop from source, compiled Tez against the hadoop jars pushed to 
staging repo and ran a few example Tez jobs on a single node cluster. 

— HItesh 


On Nov 13, 2014, at 3:08 PM, Arun C Murthy  wrote:

> Folks,
> 
> I've created another release candidate (rc1) for hadoop-2.6.0 based on the 
> feedback.
> 
> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.6.0-rc1
> The RC tag in git is: release-2.6.0-rc1
> 
> The maven artifacts are available via repository.apache.org at 
> https://repository.apache.org/content/repositories/orgapachehadoop-1013.
> 
> Please try the release and vote; the vote will run for the usual 5 days.
> 
> thanks,
> Arun
> 
> 
> -- 
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to 
> which it is addressed and may contain information that is confidential, 
> privileged and exempt from disclosure under applicable law. If the reader 
> of this message is not the intended recipient, you are hereby notified that 
> any printing, copying, dissemination, distribution, disclosure or 
> forwarding of this communication is strictly prohibited. If you have 
> received this communication in error, please contact the sender immediately 
> and delete it from your system. Thank You.

Re: [DISCUSS] Release numbering for stable 2.8 and beyond

2015-04-26 Thread Hitesh Shah 
There are a couple of different approaches we could take. 

How about publishing/releasing bits such as “2.8.0-RC0”. Downstream projects 
can depend on these and use them normally similar to the approach that they 
would have taken with release 2.8.0 or 2.8.0-alpha. After some baking ( or more 
RC suffixed releases), at some point, a proper 2.8.0 release could be made? The 
RC tag clearly indicates to downstream layers that things will be in flux 
slightly. Trying to distinguish incompatibilities between 2.8.0 and 2.8.1/2.8.2 
( just because 2.8.0 was tagged alpha in documentation ) are likely to be a 
nightmare to deal with especially for new features introduced in the 2.8 
release ( which might get changed slightly based on usage feedback ).

Furthermore, considering the release history of hadoop, the likelihood that 
2.9.0 will show up before 2.8.2 seems to be very high.

With respect to the proposed choice (1), I thought that HBase applied a 
different approach. The odd-number releases were always unstable and the even 
number releases were the stable ones. This “could" make things a bit more clear 
for downstream users without needing to resort to modified versioned numbers ( 
with alpha/beta tags ) or requiring users to go look up the website to find out 
which particular version of 2.8.x was the first stable release.

thanks
— Hitesh





On Apr 22, 2015, at 2:17 PM, Vinod Kumar Vavilapalli  
wrote:

> Forking the thread.
> 
> In the previous 2.7.1 thread [1], there were enough yays to my proposal to 
> wait for a bug-fix release or two before calling a 2.x release stable. There 
> were some concerns about the naming.
> 
> We have two options, taking 2.8 as an example
> (1) Release 2.8.0, call it as an alpha in documentation and release notes, 
> wait for a 2.8.1/2.8.2 reasonably stable enough to be called as the first 
> stable release of 2.8.
> (2) Release 2.8.0-alpha, 2.8.0-beta etc before culminating in a 2.8.0 stable 
> release.
> 
> (1) is what I preferred first up. This is what HBase used to do, and far 
> beyond, in the linux kernel releases. It helps in scenarios where we are 
> forced to downgrade a release, say due to major issues. We can simply 
> announce it as not stable retroactively, change the pointers on our website 
> and move on.
> 
> Thoughts?
> 
> Thanks,
> +Vinod
> 
> [1] http://markmail.org/thread/ogzk4phj6wsdpssu
> 
> On Apr 21, 2015, at 4:59 PM, Vinod Kumar Vavilapalli 
>  wrote:
> 
>> 
>> Sure, I agree it's better to have clear guidelines and scheme. Let me fork 
>> this thread about that.
>> 
>> Re 2.7.0, I just forgot about the naming initially though I was clear in the 
>> discussion/voting. I so had to end up calling it alpha outside of the 
>> release artifact naming.
>> 
>> Thanks
>> +Vinod
>> 
>> On Apr 21, 2015, at 4:26 PM, Andrew Wang  wrote:
>> 
>>> I would also like to support Karthik's proposal on the release thread about
>>> version numbering. 2.7.0 being "alpha" is pretty confusing since all of the
>>> other 2.x releases since GA have been stable. I think users would prefer a
>>> version like "2.8.0-alpha1" instead, which is very clear and similar to
>>> what we did for 2.0 and 2.1. Then we release 2.8.0 when we're actually
>>> stable.
>>> 
>>> I don't know if it's retroactively possible to do this for 2.7.0, but it's
>>> something to consider for the next 2.7 alpha or beta or whatever.
>>> 
>

Re: Planning Hadoop 2.6.1 release

2015-08-11 Thread Hitesh Shah 
Jason Lowe just filed YARN-4047 ( patch available too ) which probably is also 
a good candidate for 2.6.1 or 2.6.2 if the list is frozen. I have labelled the 
jira as a 2.6.1 candidate. Feel free to switch to 2.6.2 if needed. 

thanks
— Hitesh

On Aug 10, 2015, at 4:15 PM, Vinod Kumar Vavilapalli  
wrote:

> As of now, I am freezing the list. We have 139 tickets to go through, find 
> the right order, get them committed etc.
> 
> I’ll start a separate thread for this one-time long release-process of 2.6.1.
> 
> Thanks everyone for pitching in.
> +Vinod
> 
>> On Aug 10, 2015, at 4:13 PM, Vinod Kumar Vavilapalli 
>>  wrote:
>> 
>> Added them all, tx Rohit.
>> 
>> +Vinod
>> 
>>> On Aug 5, 2015, at 9:51 PM, Rohith Sharma K S  
>>> wrote:
>>> 
>>> Can we add following fixes to 2.6.1?
>>> 
>>> YARN-3733 
>>> YARN-2865
>>> YARN-3990
>>> YARN-2894
>>> 
>>> 
>>> Thanks & Regards
>>> Rohith Sharma K S
>>> 
>>> 
>>> -Original Message-
>>> From: Allan Wilson [mailto:awils...@pandora.com] 
>>> Sent: 05 August 2015 23:25
>>> To: common-dev@hadoop.apache.org; mapreduce-...@hadoop.apache.org; 
>>> yarn-...@hadoop.apache.org
>>> Cc: hdfs-...@hadoop.apache.org
>>> Subject: Re: Planning Hadoop 2.6.1 release
>>> 
>>> Another +1 to add those fixes.
>>> 
>>> YARN-3487 bug can grind a large cluster to a halt repeatedly
>>> 
>>> 
>>> 
>>> -Allan
>>> 
>>> 
>>> Allan Wilson | Sr. Software Engineer | Pandora m 919.841.2449 |  
>>> awils...@pandora.com
>>> 
>>> 
>>> 
>>> 
>>> 
>>> On 8/5/15, 1:52 PM, "Rich Haase"  wrote:
>>> 
 +1 to add those fixes.
 
 
 Rich Haase | Sr. Software Engineer | Pandora m 303.887.1146 | 
 rha...@pandora.com
 
 
 
 
 On 8/5/15, 11:42 AM, "Wangda Tan"  wrote:
 
> Can we add following two fixes to 2.6.1?
> 
> https://issues.apache.org/jira/browse/YARN-2922 and 
> https://issues.apache.org/jira/browse/YARN-3487.
> 
> They're not fatal issue, but they can cause lots of issue in a large 
> cluster.
> 
> Thanks,
> Wangda
> 
> 
> On Mon, Aug 3, 2015 at 1:21 PM, Sangjin Lee  wrote:
> 
>> See my later update in the thread. HDFS-7704 is in the list.
>> 
>> Thanks,
>> Sangjin
>> 
>> On Mon, Aug 3, 2015 at 1:19 PM, Vinod Kumar Vavilapalli < 
>> vino...@hortonworks.com> wrote:
>> 
>>> Makes sense, it was caused by HDFS-7704 which got into 2.7.0 only 
>>> and
>> is
>>> not part of the candidate list. Removed HDFS-7916 from the list.
>>> 
>>> Thanks
>>> +Vinod
>>> 
 On Jul 24, 2015, at 6:32 PM, Sangjin Lee  wrote:
 
 Out of the JIRAs we proposed, please remove HDFS-7916. I don't
>> think it
 applies to 2.6.
 
 Thanks,
 Sangjin
>>> 
>>> 
>> 
 
>>> 
>>> 
>> 
>

[jira] [Created] (HADOOP-9422) HADOOP_HOME needs to be set to be able to use org.apache.hadoop.util.Shell to launch commands

2013-03-20 Thread Hitesh Shah (JIRA) 
Hitesh Shah created HADOOP-9422:
---

 Summary: HADOOP_HOME needs to be set to be able to use 
org.apache.hadoop.util.Shell to launch commands
 Key: HADOOP-9422
 URL: https://issues.apache.org/jira/browse/HADOOP-9422
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Hitesh Shah


Not sure why this is an enforced requirement especially in cases where a 
deployment is done using multiple tar-balls ( one each for 
common/hdfs/mapreduce/yarn ). 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HADOOP-9628) Setup a daily build job for branch-2.1.0-beta

2013-06-06 Thread Hitesh Shah (JIRA) 
Hitesh Shah created HADOOP-9628:
---

 Summary: Setup a daily build job for branch-2.1.0-beta
 Key: HADOOP-9628
 URL: https://issues.apache.org/jira/browse/HADOOP-9628
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Hitesh Shah




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HADOOP-9726) org.apache.hadoop.security.SecurityUtil has its own static Configuration which cannot be overridden

2013-07-11 Thread Hitesh Shah (JIRA) 
Hitesh Shah created HADOOP-9726:
---

 Summary: org.apache.hadoop.security.SecurityUtil has its own 
static Configuration which cannot be overridden
 Key: HADOOP-9726
 URL: https://issues.apache.org/jira/browse/HADOOP-9726
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Hitesh Shah


There is a static block which loads a new Configuration object and uses it to 
initialize the SSLFactory and HostResolver.

Should this class have a similar static setConfiguration() function similar to 
UGI? 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HADOOP-9741) Make build created indexed jars

2013-07-17 Thread Hitesh Shah (JIRA) 
Hitesh Shah created HADOOP-9741:
---

 Summary: Make build created indexed jars
 Key: HADOOP-9741
 URL: https://issues.apache.org/jira/browse/HADOOP-9741
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Hitesh Shah
Assignee: Hitesh Shah
Priority: Minor




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HADOOP-8774) Incorrect maven configuration causes mrapp-generated-classpath file to be generated in multiple places

2012-09-06 Thread Hitesh Shah (JIRA) 
Hitesh Shah created HADOOP-8774:
---

 Summary: Incorrect maven configuration causes 
mrapp-generated-classpath file to be generated in multiple places
 Key: HADOOP-8774
 URL: https://issues.apache.org/jira/browse/HADOOP-8774
 Project: Hadoop Common
  Issue Type: Bug
  Components: build
Affects Versions: 2.1.0-alpha
Reporter: Hitesh Shah
Priority: Minor


Running a simple mvn clean install at the top of tree shows logs such as the 
ones below in the HDFS tree:

[INFO] --- maven-dependency-plugin:2.1:build-classpath (build-classpath) @ 
hadoop-hdfs-httpfs ---
[INFO] Wrote classpath file 
'/Users/Hitesh/dev/hadoop-common/hadoop-hdfs-project/hadoop-hdfs-httpfs/target/classes/mrapp-generated-classpath'.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HADOOP-9741) Make build created indexed jars

2014-08-04 Thread Hitesh Shah (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HADOOP-9741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hitesh Shah resolved HADOOP-9741.
-

Resolution: Won't Fix

> Make build created indexed jars
> ---
>
> Key: HADOOP-9741
> URL: https://issues.apache.org/jira/browse/HADOOP-9741
> Project: Hadoop Common
>  Issue Type: Bug
>Reporter: Hitesh Shah
>    Assignee: Hitesh Shah
>Priority: Minor
> Attachments: HADOOP-9741.1.patch, INDEX.LIST
>
>




--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HADOOP-10951) Make org.apache.hadoop.security.Groups public

2014-08-08 Thread Hitesh Shah (JIRA) 
Hitesh Shah created HADOOP-10951:


 Summary: Make org.apache.hadoop.security.Groups public
 Key: HADOOP-10951
 URL: https://issues.apache.org/jira/browse/HADOOP-10951
 Project: Hadoop Common
  Issue Type: Improvement
Reporter: Hitesh Shah


This class seems like useful functionality for most developers building yarn 
applications with respect to application acls.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (HADOOP-10951) Make org.apache.hadoop.security.Groups public

2014-08-19 Thread Hitesh Shah (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HADOOP-10951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hitesh Shah resolved HADOOP-10951.
--

Resolution: Not a Problem

> Make org.apache.hadoop.security.Groups public
> -
>
> Key: HADOOP-10951
> URL: https://issues.apache.org/jira/browse/HADOOP-10951
> Project: Hadoop Common
>  Issue Type: Improvement
>    Reporter: Hitesh Shah
>
> This class seems like useful functionality for most developers building yarn 
> applications with respect to application acls.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HADOOP-7722) Support a stop command in ShellCommandExecutor

2011-10-04 Thread Hitesh Shah (Created) (JIRA) 
Support a stop command in ShellCommandExecutor 
---

 Key: HADOOP-7722
 URL: https://issues.apache.org/jira/browse/HADOOP-7722
 Project: Hadoop Common
  Issue Type: Bug
Affects Versions: 0.23.0
Reporter: Hitesh Shah
Priority: Minor
 Fix For: 0.23.0


Add a stop function call to Shell so as to either stop the process from getting 
started if not already done so or destroy the process if it already has 
started. 

Context of usage: 
There can be race conditions when trying to kill an execution as the 
shell::execute call is blocking and using the getProcess call is not a  
feasible option as there can be a delay between the execute call and when the 
process object is actually created ( after which it can be used to kill the 
running sub-process ). 




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HADOOP-8109) Using the default rpms of hadoop, /usr/etc/hadoop symlink gets removed on upgrade

2012-02-23 Thread Hitesh Shah (Created) (JIRA) 
Using the default rpms of hadoop,  /usr/etc/hadoop symlink gets removed on 
upgrade
--

 Key: HADOOP-8109
 URL: https://issues.apache.org/jira/browse/HADOOP-8109
 Project: Hadoop Common
  Issue Type: Bug
Affects Versions: 1.0.0, 0.23.0
Reporter: Hitesh Shah
Assignee: Hitesh Shah
Priority: Minor


Given that for rpms, the pre-uninstall scripts for the older version run after 
the post-install scripts of the package being installed, the symlink created 
from /usr/etc/hadoop to /etc/hadoop gets deleted. This breaks running 
/usr/bin/hadoop without providing --config. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira