Re: [VOTE] Release Apache Hadoop 2.7.2 RC0

2015-12-10 Thread Konstantin Boudnik
Vinod, are you Ok with me putting it into 2.7.2?

Cos

On Mon, Nov 30, 2015 at 01:25PM, Vinod Kumar Vavilapalli wrote:
> I think we should get it in, I am still stuck on a couple of HDFS patches.
> 
> But I was not sure overall if the patch was right. Commented on the JIRA 
> regarding the same.
> 
> Thanks
> +Vinod
> 
> > On Nov 27, 2015, at 1:54 AM, Steve Loughran <ste...@hortonworks.com> wrote:
> > 
> > well, I'm not going to block it...it doesn't add anything more to the pom 
> > dependencies that aren't there right now
> > 
> >> On 26 Nov 2015, at 18:51, Konstantin Boudnik <c...@apache.org> wrote:
> >> 
> >> On Wed, Nov 25, 2015 at 01:51PM, Steve Loughran wrote:
> >>> 
> >>>> On 25 Nov 2015, at 02:48, Konstantin Boudnik <c...@apache.org> wrote:
> >>>> 
> >>>> Vinod, hopefully it isn't too late for a quick fix to be included into 
> >>>> 2.7.2
> >>>> (sorry for jumping too late on this train). The JIRA in question is
> >>>> HADOOP-12415 and I have committed it to trunk and branch-2 already.
> >>>> 
> >>>> Please let me know if this is still Ok to add into 2.7.2 as it seems to 
> >>>> be
> >>>> going through a respin. Thanks in advance,
> >>>> 
> >>>> Cos
> >>>> 
> >>> 
> >>> 
> >>> can you hold it off until 2.7.3? 2.7.2 was tested and we shoudn't be 
> >>> making
> >>> more changes now, even if its just to the POMs
> >> 
> >> It's up to you guys, the fix is small but pretty critical IMO. The change 
> >> in
> >> question is literally breaking the build from the get go. In Bigtop we had 
> >> to
> >> patch 2.7.1 so we at least can build it into the stack. 
> >> 
> >> For the users using official release tarball it won't simply work unless 
> >> they
> >> are going to patch it themselves.
> >> 
> >> Cos
> > 
> > 
> 


signature.asc
Description: Digital signature


Re: [VOTE] Release Apache Hadoop 2.7.2 RC0

2015-11-26 Thread Konstantin Boudnik
On Wed, Nov 25, 2015 at 01:51PM, Steve Loughran wrote:
> 
> > On 25 Nov 2015, at 02:48, Konstantin Boudnik <c...@apache.org> wrote:
> > 
> > Vinod, hopefully it isn't too late for a quick fix to be included into 2.7.2
> > (sorry for jumping too late on this train). The JIRA in question is
> > HADOOP-12415 and I have committed it to trunk and branch-2 already.
> > 
> > Please let me know if this is still Ok to add into 2.7.2 as it seems to be
> > going through a respin. Thanks in advance,
> > 
> > Cos
> > 
> 
> 
> can you hold it off until 2.7.3? 2.7.2 was tested and we shoudn't be making
> more changes now, even if its just to the POMs

It's up to you guys, the fix is small but pretty critical IMO. The change in
question is literally breaking the build from the get go. In Bigtop we had to
patch 2.7.1 so we at least can build it into the stack. 

For the users using official release tarball it won't simply work unless they
are going to patch it themselves.

Cos


signature.asc
Description: Digital signature


Re: [DISCUSS] Release numbering for stable 2.8 and beyond

2015-11-26 Thread Konstantin Boudnik
On Mon, Apr 27, 2015 at 10:45AM, Andrew Wang wrote:
> I think Karthik correctly identified the two reasons we might want to
> denote a release as "unstable":
> 
> a) Compatibility. Whether we have freedom to break compatibility before the
> final release, i.e. what "alpha" typically means.

Breaking compatibility within minor releases is a very novel idea, introduced
to the world by Scala, if I am not mistaken. In general though, it is a pretty
bad practice to have 2.8.0 be incompatible with 2.8.1 - this isn't what
normally is expected from a subminor update.

> b) Quality. As Konst says, a release can be allowed to break compatibility
> ("alpha") but itself still be a high quality release.

I don't think Konst has advocated to make subminors incopatible ;)

As for the original question, I'd really go with a normal versioning and
just documenting any stability concerns.

Cos

> We could try and separate out these two concerns when it comes to
> versioning, but I think in reality users only care about prod vs. non-prod,
> which is why many other projects (Linux, HBase, etc) just have two release
> lines: "stable" (compatible/good-quality) vs. "unstable"
> (unknown-compatible/unknown-quality).
> 
> To this end, I don't mind what we choose, as long as the difference between
> stable and unstable is denoted by the version number. I don't like the idea
> of tagging a release as good after the fact (1). The other projects we've
> used as examples make strong statements about their stable release lines,
> and we should too. Our downstreams and end users will appreciate clarity
> from the version number.
> 
> Best,
> Andrew
> 
> On Sun, Apr 26, 2015 at 9:51 AM, Hitesh Shah  wrote:
> 
> > There are a couple of different approaches we could take.
> >
> > How about publishing/releasing bits such as “2.8.0-RC0”. Downstream
> > projects can depend on these and use them normally similar to the approach
> > that they would have taken with release 2.8.0 or 2.8.0-alpha. After some
> > baking ( or more RC suffixed releases), at some point, a proper 2.8.0
> > release could be made? The RC tag clearly indicates to downstream layers
> > that things will be in flux slightly. Trying to distinguish
> > incompatibilities between 2.8.0 and 2.8.1/2.8.2 ( just because 2.8.0 was
> > tagged alpha in documentation ) are likely to be a nightmare to deal with
> > especially for new features introduced in the 2.8 release ( which might get
> > changed slightly based on usage feedback ).
> >
> > Furthermore, considering the release history of hadoop, the likelihood
> > that 2.9.0 will show up before 2.8.2 seems to be very high.
> >
> > With respect to the proposed choice (1), I thought that HBase applied a
> > different approach. The odd-number releases were always unstable and the
> > even number releases were the stable ones. This “could" make things a bit
> > more clear for downstream users without needing to resort to modified
> > versioned numbers ( with alpha/beta tags ) or requiring users to go look up
> > the website to find out which particular version of 2.8.x was the first
> > stable release.
> >
> > thanks
> > — Hitesh
> >
> >
> >
> >
> >
> > On Apr 22, 2015, at 2:17 PM, Vinod Kumar Vavilapalli <
> > vino...@hortonworks.com> wrote:
> >
> > > Forking the thread.
> > >
> > > In the previous 2.7.1 thread [1], there were enough yays to my proposal
> > to wait for a bug-fix release or two before calling a 2.x release stable.
> > There were some concerns about the naming.
> > >
> > > We have two options, taking 2.8 as an example
> > > (1) Release 2.8.0, call it as an alpha in documentation and release
> > notes, wait for a 2.8.1/2.8.2 reasonably stable enough to be called as the
> > first stable release of 2.8.
> > > (2) Release 2.8.0-alpha, 2.8.0-beta etc before culminating in a 2.8.0
> > stable release.
> > >
> > > (1) is what I preferred first up. This is what HBase used to do, and far
> > beyond, in the linux kernel releases. It helps in scenarios where we are
> > forced to downgrade a release, say due to major issues. We can simply
> > announce it as not stable retroactively, change the pointers on our website
> > and move on.
> > >
> > > Thoughts?
> > >
> > > Thanks,
> > > +Vinod
> > >
> > > [1] http://markmail.org/thread/ogzk4phj6wsdpssu
> > >
> > > On Apr 21, 2015, at 4:59 PM, Vinod Kumar Vavilapalli <
> > vino...@hortonworks.com> wrote:
> > >
> > >>
> > >> Sure, I agree it's better to have clear guidelines and scheme. Let me
> > fork this thread about that.
> > >>
> > >> Re 2.7.0, I just forgot about the naming initially though I was clear
> > in the discussion/voting. I so had to end up calling it alpha outside of
> > the release artifact naming.
> > >>
> > >> Thanks
> > >> +Vinod
> > >>
> > >> On Apr 21, 2015, at 4:26 PM, Andrew Wang 
> > wrote:
> > >>
> > >>> I would also like to support Karthik's proposal on the release thread
> > about
> > >>> version numbering. 2.7.0 being 

Re: [VOTE] Release Apache Hadoop 2.7.2 RC0

2015-11-24 Thread Konstantin Boudnik
Vinod, hopefully it isn't too late for a quick fix to be included into 2.7.2
(sorry for jumping too late on this train). The JIRA in question is
HADOOP-12415 and I have committed it to trunk and branch-2 already.

Please let me know if this is still Ok to add into 2.7.2 as it seems to be
going through a respin. Thanks in advance,

Cos

On Wed, Nov 11, 2015 at 08:31PM, Vinod Kumar Vavilapalli wrote:
> Hi all,
> 
> 
> I've created a release candidate RC0 for Apache Hadoop 2.7.2.
> 
> 
> As discussed before, this is the next maintenance release to follow up
> 2.7.1.
> 
> 
> The RC is available for validation at:
> 
> *http://people.apache.org/~vinodkv/hadoop-2.7.2-RC0/
> 
> *
> 
> 
> The RC tag in git is: release-2.7.2-RC0
> 
> 
> The maven artifacts are available via repository.apache.org at
> 
> *https://repository.apache.org/content/repositories/orgapachehadoop-1023/
> 
> *
> 
> 
> As you may have noted, an unusually long 2.6.3 release caused 2.7.2 to slip
> by quite a bit. This release's related discussion threads are linked below:
> [1] and [2].
> 
> 
> Please try the release and vote; the vote will run for the usual 5 days.
> 
> 
> Thanks,
> 
> Vinod
> 
> 
> [1]: 2.7.2 release plan: http://markmail.org/message/oozq3gvd4nhzsaes
> 
> [2]: Planning Apache Hadoop 2.7.2
> http://markmail.org/message/iktqss2qdeykgpqk


signature.asc
Description: Digital signature


[jira] [Resolved] (HADOOP-7730) Allow TestCLI to be run against a cluster

2015-10-07 Thread Konstantin Boudnik (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-7730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik resolved HADOOP-7730.

  Resolution: Won't Fix
Release Note: The resolution has been done in the Bigtop a few years 
ago. Closing this one as of irrelevant to the project.
Target Version/s: 1.3.0, 0.22.1  (was: 0.22.1, 1.3.0)

> Allow TestCLI to be run against a cluster
> -
>
> Key: HADOOP-7730
> URL: https://issues.apache.org/jira/browse/HADOOP-7730
> Project: Hadoop Common
>  Issue Type: Test
>  Components: test
>Affects Versions: 0.20.205.0, 0.22.0
>        Reporter: Konstantin Boudnik
>    Assignee: Konstantin Boudnik
> Attachments: HADOOP-7730.patch, HADOOP-7730.trunk.patch, 
> HADOOP-7730.trunk.patch
>
>
> Use the same CLI test to test cluster bits (see HDFS-1762 for more info)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (HADOOP-12415) Hadoop build is broken because of missed compile-time dependency

2015-09-15 Thread Konstantin Boudnik (JIRA)
Konstantin Boudnik created HADOOP-12415:
---

 Summary: Hadoop build is broken because of missed compile-time 
dependency
 Key: HADOOP-12415
 URL: https://issues.apache.org/jira/browse/HADOOP-12415
 Project: Hadoop Common
  Issue Type: Bug
  Components: nfs
Affects Versions: 2.7.1
Reporter: Konstantin Boudnik


As discovered in BIGTOP-2049 {{hadoop-nfs}} module compilation is broken. Looks 
like that HADOOP-11489 is the root-cause of it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (HADOOP-11945) Add Apache Ignite (incubating) to the Hadoop-related projects

2015-05-08 Thread Konstantin Boudnik (JIRA)
Konstantin Boudnik created HADOOP-11945:
---

 Summary: Add Apache Ignite (incubating) to the Hadoop-related 
projects
 Key: HADOOP-11945
 URL: https://issues.apache.org/jira/browse/HADOOP-11945
 Project: Hadoop Common
  Issue Type: Improvement
  Components: site
Affects Versions: 2.5.2
Reporter: Konstantin Boudnik
Assignee: Konstantin Boudnik
 Fix For: site


Apache Ignite (incubating) has the implementation of HDFS compatible in-memory 
file system (IGFS) and in-memory MapReduce accelerator. These two components 
allow Ignite users to run OLTP/SQL/computational workloads on top of HDFS 
storage.

Let's add this project to the list of Hadoop related ones.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Re: Dropping support for JDK6 in Apache Hadoop

2014-08-19 Thread Konstantin Boudnik
While it sounds like a topic for a different discussion (or list@), do you
think it would be too crazy to skip JDK7 completely and just go to JDK8
directly? I think downstream components like HBase, Spark and so on would be a
huge driving factor to make the same change in Hadoop and other parts of the
ecosystem.

Thoughts?
  Cos
   
On Tue, Aug 19, 2014 at 11:23AM, Devaraj Das wrote:
 I think we are good on HBase side, at least on the 1.x+ branches. We
 are dropping support for 1.6 from HBase-1.0 onwards.
 
 On Tue, Aug 19, 2014 at 10:52 AM, Arun C Murthy a...@hortonworks.com wrote:
  [Apologies for the wide distribution.]
 
  Dear HBase/Hive/Pig/Oozie communities,
 
   We, over at Hadoop are considering dropping support for JDK6 this year.
 
   As you maybe aware we just released hadoop-2.5.0 and are now considering 
  making the next release i.e. hadoop-2.6.0 the *last* release of Apache 
  Hadoop which supports JDK6. This means, from hadoop-2.7.0 onwards we will 
  not support JDK6 anymore and we *may* start relying on JDK7-specific apis.
 
   Now, the above releases a proposal and we do not want to pull the trigger 
  without talking to projects downstream - hence the request for you feedback.
 
   Please feel free to forward this to other communities you might deem to be 
  at risk from this too.
 
  thanks,
  Arun
 
 
  --
  CONFIDENTIALITY NOTICE
  NOTICE: This message is intended for the use of the individual or entity to
  which it is addressed and may contain information that is confidential,
  privileged and exempt from disclosure under applicable law. If the reader
  of this message is not the intended recipient, you are hereby notified that
  any printing, copying, dissemination, distribution, disclosure or
  forwarding of this communication is strictly prohibited. If you have
  received this communication in error, please contact the sender immediately
  and delete it from your system. Thank You.
 
 -- 
 CONFIDENTIALITY NOTICE
 NOTICE: This message is intended for the use of the individual or entity to 
 which it is addressed and may contain information that is confidential, 
 privileged and exempt from disclosure under applicable law. If the reader 
 of this message is not the intended recipient, you are hereby notified that 
 any printing, copying, dissemination, distribution, disclosure or 
 forwarding of this communication is strictly prohibited. If you have 
 received this communication in error, please contact the sender immediately 
 and delete it from your system. Thank You.


Re: Why Hadoop-trunk-commit always fails?

2014-07-21 Thread Konstantin Boudnik
If we use maven jar plugin or maven archivers to create any of these then 
adding 
  useUniqueVersionsfalse/useUniqueVersions
should solve the issue.

Cos

On Mon, Jul 21, 2014 at 01:55PM, Andrew Wang wrote:
 I dug around a bit with Tucu, and I think it's essentially the dependency
 analyzer screwing up with snapshot artifacts. I found a different error for
 HttpFS that looks similar:
 
 
 [WARNING]
 Dependency convergence error for
 org.apache.hadoop:hadoop-hdfs:3.0.0-SNAPSHOT paths to dependency are:
 +-org.apache.hadoop:hadoop-hdfs-httpfs:3.0.0-SNAPSHOT
   +-org.apache.hadoop:hadoop-hdfs:3.0.0-SNAPSHOT
 and
 +-org.apache.hadoop:hadoop-hdfs-httpfs:3.0.0-SNAPSHOT
   +-org.apache.hadoop:hadoop-hdfs:3.0.0-20140718.221409-4777
 
 [WARNING] Rule 0:
 org.apache.maven.plugins.enforcer.DependencyConvergence failed with
 message:
 Failed while enforcing releasability the error(s) are [
 Dependency convergence error for
 org.apache.hadoop:hadoop-hdfs:3.0.0-SNAPSHOT paths to dependency are:
 +-org.apache.hadoop:hadoop-hdfs-httpfs:3.0.0-SNAPSHOT
   +-org.apache.hadoop:hadoop-hdfs:3.0.0-SNAPSHOT
 and
 +-org.apache.hadoop:hadoop-hdfs-httpfs:3.0.0-SNAPSHOT
   +-org.apache.hadoop:hadoop-hdfs:3.0.0-20140718.221409-4777
 
 
 You can see that it sees 3.0.0-SNAPSHOT being used for one, and
 3.0.0-20140718.221409-4777 for the other (which causes the error). The same
 thing happened in the stuff Ted posted, but for the KMS. Somehow the local
 maven repo is getting screwed up non-deterministically.
 
 Tucu recommends we remove this check from the post-commit build, and
 instead make it part of the maven job used to build releases. At release
 time, there shouldn't be any ambiguity about version numbers.
 
 Any brave volunteers out there? I am not a maven maven, but am happy to
 review pom.xml changes that do this, and I'll make sure the maven job used
 to build releases still does the dep check.
 
 Best,
 Andrew
 
 
 
 
 On Thu, Jul 17, 2014 at 9:50 PM, Ted Yu yuzhih...@gmail.com wrote:
 
  Here is the warning from enforcer:
 
  [WARNING] Rule 0: org.apache.maven.plugins.enforcer.DependencyConvergence
  failed with message:
  Failed while enforcing releasability the error(s) are [
  Dependency convergence error for
  org.apache.hadoop:hadoop-auth:3.0.0-20140718.043141-4847 paths to
  dependency are:
  +-org.apache.hadoop:hadoop-kms:3.0.0-SNAPSHOT
+-org.apache.hadoop:hadoop-auth:3.0.0-20140718.043141-4847
  and
  +-org.apache.hadoop:hadoop-kms:3.0.0-SNAPSHOT
+-org.apache.hadoop:hadoop-common:3.0.0-20140718.043201-4831
  +-org.apache.hadoop:hadoop-auth:3.0.0-SNAPSHOT
  and
  +-org.apache.hadoop:hadoop-kms:3.0.0-SNAPSHOT
+-org.apache.hadoop:hadoop-common:3.0.0-20140718.043201-4831
  +-org.apache.hadoop:hadoop-auth:3.0.0-SNAPSHOT
  ]
 
  FYI
 
 
  On Thu, Jul 17, 2014 at 9:38 PM, Vinayakumar B vinayakum...@apache.org
  wrote:
 
   Hi,
   Hadoop-trunk-commit build always fails with message similar to below.
   Anybody knows about this?
  
   [ERROR] Failed to execute goal
   org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce
   (depcheck) on project hadoop-yarn-server-tests: Some Enforcer rules
   have failed. Look above for specific messages explaining why the rule
   failed. - [Help 1]
  
  
  
   Regards,
   Vinay
  
 


Phone bridge info [Re: Meetup invitation: Consensus based replication in Hadoop]

2014-07-14 Thread Konstantin Boudnik
To enable remote people to join the conversation, we'll keep open the
following GotoMeeting session. You can join either from your computer or by
using dial-in number. 

 =
Please join my meeting, Jul 15, 2014 at 12:00 PM PDT.
https://www4.gotomeeting.com/join/357791919

2. Use your microphone and speakers (VoIP) - a headset is recommended. Or,
call in using your telephone.

Dial +1 (805) 309-0014
Access Code: 357-791-919
Audio PIN: Shown after joining the meeting

Meeting ID: 357-791-919
 =

BTW, we have one or two spots left, so if you didn't make your reservation -
now is the time.

See you all tomorrow.
  Cos

On Fri, Jul 11, 2014 at 04:33PM, Konstantin Boudnik wrote:
 One more update: it seema that for ppl in SF, who oftentimes might not even
 have a car, getting to San Ramon can represent a certain difficulty.
 
 So we'll do a shuttle pickup from West Dublin BART station if there's at least
 a few people who want to use the option.
 
 Please respond directly to me if you're interested before end of Sunday, 13th.
 Cheers,
   Cos
 
 On Tue, Jul 08, 2014 at 12:23PM, Konstantin Boudnik wrote:
  All,
  
  Re-sending this announcement in case it fell through over the long weekend
  when people were away. We still have seats left, so register soon.
  
  Regards,
Cos
  
  On Wed, Jul 02, 2014 at 06:37PM, Konstantin Boudnik wrote:
   We'd like to invite you to the 
   Consensus based replication in Hadoop: A deep dive
   event that we are happy to hold in our San Ramon office on July 15th at 
   noon.
   We'd like to accommodate as many people as possible, but I think are 
   physically
   limited to 30 (+/- a few), so please RSVP to this Eventbrite invitation:
   
   https://www.eventbrite.co.uk/e/consensus-based-replication-in-hadoop-a-deep-dive-tickets-12158236613
   
   We'll provide pizza and beverages (feel free to express your special 
   dietary
   requirements if any).
   
   See you soon!
   With regards,
 Cos
   
   On Wed, Jun 18, 2014 at 08:45PM, Konstantin Boudnik wrote:
Guys,

In the last a couple of weeks, we had a very good and productive 
initial round
of discussions on the JIRAs. I think it is worthy to keep the momentum 
going
and have a more detailed conversation. For that, we'd like to host s 
Hadoop
developers meetup to get into the bowls of the consensus-based 
coordination
implementation for HDFS. The proposed venue is our office in San Ramon, 
CA.

Considering that it is already a mid week and the following one looks 
short
because of the holidays, how would the week of July 7th looks for yall?
Tuesday or Thursday look pretty good on our end.

Please chime in on your preference either here or reach of directly to 
me.
Once I have a few RSVPs I will setup an event on Eventbrite or similar.

Looking forward to your input. Regards,
  Cos

On Thu, May 29, 2014 at 02:09PM, Konstantin Shvachko wrote:
 Hello hadoop developers,
 
 I just opened two jiras proposing to introduce ConsensusNode into 
 HDFS and
 a Coordination Engine into Hadoop Common. The latter should benefit 
 HDFS
 and  HBase as well as potentially other projects. See HDFS-6469 and
 HADOOP-10641 for details.
 The effort is based on the system we built at Wandisco with my 
 colleagues,
 who are glad to contribute it to Apache, as quite a few people in the
 community expressed interest in this ideas and their potential 
 applications.
 
 We should probably keep technical discussions in the jiras. Here on 
 the dev
 list I wanted to touch-base on any logistic issues / questions.
 - First of all, any ideas and help are very much welcome.
 - We would like to set up a meetup to discuss this if people are
 interested. Hadoop Summit next week may be a potential time-place to 
 meet.
 Not sure in what form. If not, we can organize one in our San Ramon 
 office
 later on.
 - The effort may take a few months depending on the contributors 
 schedules.
 Would it make sense to open a branch for the ConsensusNode work?
 - APIs and the implementation of the Coordination Engine should be a 
 fairly
 independent, so it may be reasonable to add it directly to Hadoop 
 Common
 trunk.
 
 Thanks,
 --Konstantin




Re: Meetup invitation: Consensus based replication in Hadoop

2014-07-11 Thread Konstantin Boudnik
One more update: it seema that for ppl in SF, who oftentimes might not even
have a car, getting to San Ramon can represent a certain difficulty.

So we'll do a shuttle pickup from West Dublin BART station if there's at least
a few people who want to use the option.

Please respond directly to me if you're interested before end of Sunday, 13th.
Cheers,
  Cos

On Tue, Jul 08, 2014 at 12:23PM, Konstantin Boudnik wrote:
 All,
 
 Re-sending this announcement in case it fell through over the long weekend
 when people were away. We still have seats left, so register soon.
 
 Regards,
   Cos
 
 On Wed, Jul 02, 2014 at 06:37PM, Konstantin Boudnik wrote:
  We'd like to invite you to the 
  Consensus based replication in Hadoop: A deep dive
  event that we are happy to hold in our San Ramon office on July 15th at 
  noon.
  We'd like to accommodate as many people as possible, but I think are 
  physically
  limited to 30 (+/- a few), so please RSVP to this Eventbrite invitation:
  
  https://www.eventbrite.co.uk/e/consensus-based-replication-in-hadoop-a-deep-dive-tickets-12158236613
  
  We'll provide pizza and beverages (feel free to express your special dietary
  requirements if any).
  
  See you soon!
  With regards,
Cos
  
  On Wed, Jun 18, 2014 at 08:45PM, Konstantin Boudnik wrote:
   Guys,
   
   In the last a couple of weeks, we had a very good and productive initial 
   round
   of discussions on the JIRAs. I think it is worthy to keep the momentum 
   going
   and have a more detailed conversation. For that, we'd like to host s 
   Hadoop
   developers meetup to get into the bowls of the consensus-based 
   coordination
   implementation for HDFS. The proposed venue is our office in San Ramon, 
   CA.
   
   Considering that it is already a mid week and the following one looks 
   short
   because of the holidays, how would the week of July 7th looks for yall?
   Tuesday or Thursday look pretty good on our end.
   
   Please chime in on your preference either here or reach of directly to me.
   Once I have a few RSVPs I will setup an event on Eventbrite or similar.
   
   Looking forward to your input. Regards,
 Cos
   
   On Thu, May 29, 2014 at 02:09PM, Konstantin Shvachko wrote:
Hello hadoop developers,

I just opened two jiras proposing to introduce ConsensusNode into HDFS 
and
a Coordination Engine into Hadoop Common. The latter should benefit HDFS
and  HBase as well as potentially other projects. See HDFS-6469 and
HADOOP-10641 for details.
The effort is based on the system we built at Wandisco with my 
colleagues,
who are glad to contribute it to Apache, as quite a few people in the
community expressed interest in this ideas and their potential 
applications.

We should probably keep technical discussions in the jiras. Here on the 
dev
list I wanted to touch-base on any logistic issues / questions.
- First of all, any ideas and help are very much welcome.
- We would like to set up a meetup to discuss this if people are
interested. Hadoop Summit next week may be a potential time-place to 
meet.
Not sure in what form. If not, we can organize one in our San Ramon 
office
later on.
- The effort may take a few months depending on the contributors 
schedules.
Would it make sense to open a branch for the ConsensusNode work?
- APIs and the implementation of the Coordination Engine should be a 
fairly
independent, so it may be reasonable to add it directly to Hadoop Common
trunk.

Thanks,
--Konstantin


signature.asc
Description: Digital signature


Re: Meetup invitation: Consensus based replication in Hadoop

2014-07-08 Thread Konstantin Boudnik
All,

Re-sending this announcement in case it fell through over the long weekend
when people were away. We still have seats left, so register soon.

Regards,
  Cos

On Wed, Jul 02, 2014 at 06:37PM, Konstantin Boudnik wrote:
 We'd like to invite you to the 
 Consensus based replication in Hadoop: A deep dive
 event that we are happy to hold in our San Ramon office on July 15th at noon.
 We'd like to accommodate as many people as possible, but I think are 
 physically
 limited to 30 (+/- a few), so please RSVP to this Eventbrite invitation:
 
 https://www.eventbrite.co.uk/e/consensus-based-replication-in-hadoop-a-deep-dive-tickets-12158236613
 
 We'll provide pizza and beverages (feel free to express your special dietary
 requirements if any).
 
 See you soon!
 With regards,
   Cos
 
 On Wed, Jun 18, 2014 at 08:45PM, Konstantin Boudnik wrote:
  Guys,
  
  In the last a couple of weeks, we had a very good and productive initial 
  round
  of discussions on the JIRAs. I think it is worthy to keep the momentum going
  and have a more detailed conversation. For that, we'd like to host s Hadoop
  developers meetup to get into the bowls of the consensus-based coordination
  implementation for HDFS. The proposed venue is our office in San Ramon, CA.
  
  Considering that it is already a mid week and the following one looks short
  because of the holidays, how would the week of July 7th looks for yall?
  Tuesday or Thursday look pretty good on our end.
  
  Please chime in on your preference either here or reach of directly to me.
  Once I have a few RSVPs I will setup an event on Eventbrite or similar.
  
  Looking forward to your input. Regards,
Cos
  
  On Thu, May 29, 2014 at 02:09PM, Konstantin Shvachko wrote:
   Hello hadoop developers,
   
   I just opened two jiras proposing to introduce ConsensusNode into HDFS and
   a Coordination Engine into Hadoop Common. The latter should benefit HDFS
   and  HBase as well as potentially other projects. See HDFS-6469 and
   HADOOP-10641 for details.
   The effort is based on the system we built at Wandisco with my colleagues,
   who are glad to contribute it to Apache, as quite a few people in the
   community expressed interest in this ideas and their potential 
   applications.
   
   We should probably keep technical discussions in the jiras. Here on the 
   dev
   list I wanted to touch-base on any logistic issues / questions.
   - First of all, any ideas and help are very much welcome.
   - We would like to set up a meetup to discuss this if people are
   interested. Hadoop Summit next week may be a potential time-place to meet.
   Not sure in what form. If not, we can organize one in our San Ramon office
   later on.
   - The effort may take a few months depending on the contributors 
   schedules.
   Would it make sense to open a branch for the ConsensusNode work?
   - APIs and the implementation of the Coordination Engine should be a 
   fairly
   independent, so it may be reasonable to add it directly to Hadoop Common
   trunk.
   
   Thanks,
   --Konstantin


Re: Introducing ConsensusNode and a Coordination Engine

2014-06-22 Thread Konstantin Boudnik
They are actually listed in the first email on this thread. Here they are:
HADOOP-10641
HDFS-6469

Cos

On Wed, Jun 18, 2014 at 10:30PM, Sujeet Varakhedi wrote:
 Can you point me to the JIRA's
 
 Sujeet
 
 
 On Wed, Jun 18, 2014 at 8:45 PM, Konstantin Boudnik c...@apache.org wrote:
 
  Guys,
 
  In the last a couple of weeks, we had a very good and productive initial
  round
  of discussions on the JIRAs. I think it is worthy to keep the momentum
  going
  and have a more detailed conversation. For that, we'd like to host s Hadoop
  developers meetup to get into the bowls of the consensus-based coordination
  implementation for HDFS. The proposed venue is our office in San Ramon, CA.
 
  Considering that it is already a mid week and the following one looks short
  because of the holidays, how would the week of July 7th looks for yall?
  Tuesday or Thursday look pretty good on our end.
 
  Please chime in on your preference either here or reach of directly to me.
  Once I have a few RSVPs I will setup an event on Eventbrite or similar.
 
  Looking forward to your input. Regards,
Cos
 
  On Thu, May 29, 2014 at 02:09PM, Konstantin Shvachko wrote:
   Hello hadoop developers,
  
   I just opened two jiras proposing to introduce ConsensusNode into HDFS
  and
   a Coordination Engine into Hadoop Common. The latter should benefit HDFS
   and  HBase as well as potentially other projects. See HDFS-6469 and
   HADOOP-10641 for details.
   The effort is based on the system we built at Wandisco with my
  colleagues,
   who are glad to contribute it to Apache, as quite a few people in the
   community expressed interest in this ideas and their potential
  applications.
  
   We should probably keep technical discussions in the jiras. Here on the
  dev
   list I wanted to touch-base on any logistic issues / questions.
   - First of all, any ideas and help are very much welcome.
   - We would like to set up a meetup to discuss this if people are
   interested. Hadoop Summit next week may be a potential time-place to
  meet.
   Not sure in what form. If not, we can organize one in our San Ramon
  office
   later on.
   - The effort may take a few months depending on the contributors
  schedules.
   Would it make sense to open a branch for the ConsensusNode work?
   - APIs and the implementation of the Coordination Engine should be a
  fairly
   independent, so it may be reasonable to add it directly to Hadoop Common
   trunk.
  
   Thanks,
   --Konstantin
 


Re: Introducing ConsensusNode and a Coordination Engine

2014-06-18 Thread Konstantin Boudnik
Guys,

In the last a couple of weeks, we had a very good and productive initial round
of discussions on the JIRAs. I think it is worthy to keep the momentum going
and have a more detailed conversation. For that, we'd like to host s Hadoop
developers meetup to get into the bowls of the consensus-based coordination
implementation for HDFS. The proposed venue is our office in San Ramon, CA.

Considering that it is already a mid week and the following one looks short
because of the holidays, how would the week of July 7th looks for yall?
Tuesday or Thursday look pretty good on our end.

Please chime in on your preference either here or reach of directly to me.
Once I have a few RSVPs I will setup an event on Eventbrite or similar.

Looking forward to your input. Regards,
  Cos

On Thu, May 29, 2014 at 02:09PM, Konstantin Shvachko wrote:
 Hello hadoop developers,
 
 I just opened two jiras proposing to introduce ConsensusNode into HDFS and
 a Coordination Engine into Hadoop Common. The latter should benefit HDFS
 and  HBase as well as potentially other projects. See HDFS-6469 and
 HADOOP-10641 for details.
 The effort is based on the system we built at Wandisco with my colleagues,
 who are glad to contribute it to Apache, as quite a few people in the
 community expressed interest in this ideas and their potential applications.
 
 We should probably keep technical discussions in the jiras. Here on the dev
 list I wanted to touch-base on any logistic issues / questions.
 - First of all, any ideas and help are very much welcome.
 - We would like to set up a meetup to discuss this if people are
 interested. Hadoop Summit next week may be a potential time-place to meet.
 Not sure in what form. If not, we can organize one in our San Ramon office
 later on.
 - The effort may take a few months depending on the contributors schedules.
 Would it make sense to open a branch for the ConsensusNode work?
 - APIs and the implementation of the Coordination Engine should be a fairly
 independent, so it may be reasonable to add it directly to Hadoop Common
 trunk.
 
 Thanks,
 --Konstantin


Re: Re-swizzle 2.3

2014-02-06 Thread Konstantin Boudnik
Do you guys think that committing 
https://issues.apache.org/jira/browse/HDFS-4858
to branch-2.3 is still Ok? It is a small change that bring fixes broken
timeout behavior of DN to NN RPC.

We have been testing this fix on top of 2.0.6 for a long time now and it seems
to be a real help.

Appreciate the feedback on 2.3 scope. If it is too late then I will commit it
to trunk, branch-2.4 and branch-2 only.

Regards,
  Cos

On Thu, Feb 06, 2014 at 05:44PM, Alejandro Abdelnur wrote:
 yep, the idea is to pull all of them out from branch2.3. things go back to 
 normal then. 
 
 thanks
 
 Alejandro
 (phone typing)
 
  On Feb 6, 2014, at 17:39, Zhijie Shen zs...@hortonworks.com wrote:
  
  Recently I brought 4 JIRAs to branch-2.3, which are MAPREDUCE-5743, 
  YARN-1628,
  YARN-1661 and YARN-1689. Recall that we mark test failure fixes as blockers
  for pior releases as closing to release, thus I brought to branch-2.3
  MAPREDUCE-5743
  and YARN-1628 that are the fixes for the test failure on 2.3.0, but didn't
  marked them as blockers. Please let me know if I should do that.
  
  YARN-1661 is a fix for exit log of DS AppMaster, otherwise the exit log of
  it will always be failure, which sounds a critical issue to me. Feel free
  to pull it out if any objects.
  
  YARN-1689 is brought to branch-2.3 as YARN-1493 is still in this branch. It
  fixes one bug caused by YARN-1493. Those should be included or excluded
  together upon the decision.
  
  Thanks,
  Zhijie
  
  
  On Thu, Feb 6, 2014 at 5:15 PM, Sandy Ryza sandy.r...@cloudera.com wrote:
  
  +1 to reverting those JIRAs from branch-2.3.  As YARN-1689 is fixing a
  problem caused by YARN-1493 I think we can revert it in branch-2.3 as well.
  
  I think we should leave them in branch-2 for now.  We can revert if 2.4 is
  imminent and they're holding it up, but hopefully the issues they caused
  will be fixed by then.
  
  -Sandy
  
  
  On Thu, Feb 6, 2014 at 5:07 PM, Alejandro Abdelnur t...@cloudera.com
  wrote:
  
  Thanks Robert,
  
  All,
  
  So it seems that YARN-1493 and YARN-1490 are introducing serious
  regressions.
  
  I would propose to revert them and the follow up JIRAs from the 2.3
  branch
  and keep working on them on trunk/branch-2 until the are stable (I would
  even prefer reverting them from branch-2 not to block a 2.4 if they are
  not
  ready in time).
  
  As I've mentioned before, the list of JIRAs to revert were:
  
  YARN-1493
  YARN-1490
  YARN-1166
  YARN-1041
  YARN-1566
  
  Plus 2 additional JIRAs committed since my email on this issue 2 days
  ago:
  
  *YARN-1661
  *YARN-1689 (not sure if this JIRA is related in functionality to the
  previous ones but it is creating conflicts).
  
  I think we should hold on continuing work on top of something that is
  broken until the broken stuff is fixed.
  
  Quoting Arun, Committers - Henceforth, please use extreme caution while
  committing to branch-2.3. Please commit *only* blockers to 2.3.
  
  YARN-1661  YARN-1689 are not blockers.
  
  Unless there are objections, I'll revert all these JIRAs from branch-2.3
  tomorrow around noon and I'll update fixedVersion in the JIRAs.
  
  I'm inclined to revert them from branch-2 as well.
  
  Thoughts?
  
  Thanks.
  
  
  On Thu, Feb 6, 2014 at 3:54 PM, Robert Kanter rkan...@cloudera.com
  wrote:
  
  I think we should revert YARN-1490 from Hadoop 2.3 branch.  I think it
  was
  causing some strange behavior in the Oozie unit tests:
  
  Basically, we use a single MiniMRCluster and MiniDFSCluster across all
  unit
  tests in a module.  With YARN-1490 we saw that, regardless of test
  order,
  the last few tests would timeout waiting for an MR job to finish; on
  slower
  machines, the entire test suite would timeout.  Through some digging, I
  found that we were getting a ton of Connection refused Exceptions on
  LeaseRenewer talking to the NN and a few on the AM talking to the RM.
  
  After a bunch of investigation, I found that the problem went away once
  YARN-1490 was removed.  Though I couldn't figure out the exact problem.
  Even though this occurred in unit tests, it does make me concerned
  that
  it
  could indicate some bigger issue in a long-running real cluster (where
  everything isn't running on the same machine) that we haven't seen yet.
  
  
  
  On Thu, Feb 6, 2014 at 3:06 PM, Karthik Kambatla ka...@cloudera.com
  wrote:
  
  I have marked MAPREDUCE-5744 a blocker for 2.3. Committing it
  shortly.
  Will
  pull it out of branch-2.3 if anyone objects.
  
  
  On Thu, Feb 6, 2014 at 2:04 PM, Arpit Agarwal 
  aagar...@hortonworks.com
  wrote:
  
  Merged HADOOP-10273 to branch-2.3 as r1565456.
  
  
  On Wed, Feb 5, 2014 at 4:49 PM, Arpit Agarwal 
  aagar...@hortonworks.com
  wrote:
  
  IMO HADOOP-10273 (Fix 'mvn site') should be included in 2.3.
  
  I will merge it to branch-2.3 tomorrow PST if no one disagrees.
  
  
  On Tue, Feb 4, 2014 at 5:03 PM, Alejandro Abdelnur 
  t...@cloudera.com
  wrote:
  
  IMO 

Troubles committing

2014-01-20 Thread Konstantin Boudnik
Guys,

anyone experience any issues with committing to Hadoop SVN today? Looks like
my password is getting rejected by the SVN server (same password works for
login into people.apache.org). 

Want to make before troubling INFRA. Thanks in advance!
-- 
Take care,
Cos



Re: Troubles committing

2014-01-20 Thread Konstantin Boudnik
Wow, seems that my account was blocked after 18 consequent attempts to login
to my account since Jan 15th. I am pretty sure all this activity hasn't been
caused by any of devices. Hence, if you guys see something like this happening
to you - let INFRA know ASAP.

My issue is addressed now - thanks INFRA guys for quick response!
  Cos


On Mon, Jan 20, 2014 at 08:57PM, Konstantin Boudnik wrote:
 Guys,
 
 anyone experience any issues with committing to Hadoop SVN today? Looks like
 my password is getting rejected by the SVN server (same password works for
 login into people.apache.org). 
 
 Want to make before troubling INFRA. Thanks in advance!
 -- 
 Take care,
   Cos
 


[jira] [Reopened] (HADOOP-10110) hadoop-auth has a build break due to missing dependency

2013-12-05 Thread Konstantin Boudnik (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-10110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik reopened HADOOP-10110:
-


Needed to be ported back into 2.0.x line as it's breaking the build there as 
well

 hadoop-auth has a build break due to missing dependency
 ---

 Key: HADOOP-10110
 URL: https://issues.apache.org/jira/browse/HADOOP-10110
 Project: Hadoop Common
  Issue Type: Bug
  Components: build
Affects Versions: 3.0.0, 2.0.6-alpha
Reporter: Chuan Liu
Assignee: Chuan Liu
Priority: Blocker
 Fix For: 3.0.0, 2.3.0

 Attachments: HADOOP-10110.patch


 We have a build break in hadoop-auth if build with maven cache cleaned. The 
 error looks like the follows. The problem exists on both Windows and Linux. 
 If you have old jetty jars in your maven cache, you won't see the error.
 {noformat}
 [INFO] 
 
 [INFO] BUILD FAILURE
 [INFO] 
 
 [INFO] Total time: 1:29.469s
 [INFO] Finished at: Mon Nov 18 12:30:36 PST 2013
 [INFO] Final Memory: 37M/120M
 [INFO] 
 
 [ERROR] Failed to execute goal 
 org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile 
 (default-testCompile) on project hadoop-auth: Compilation failure: 
 Compilation failure:
 [ERROR] 
 /home/chuan/trunk/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[84,13]
  cannot access org.mortbay.component.AbstractLifeCycle
 [ERROR] class file for org.mortbay.component.AbstractLifeCycle not found
 [ERROR] server = new Server(0);
 [ERROR] 
 /home/chuan/trunk/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[94,29]
  cannot access org.mortbay.component.LifeCycle
 [ERROR] class file for org.mortbay.component.LifeCycle not found
 [ERROR] server.getConnectors()[0].setHost(host);
 [ERROR] 
 /home/chuan/trunk/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[96,10]
  cannot find symbol
 [ERROR] symbol  : method start()
 [ERROR] location: class org.mortbay.jetty.Server
 [ERROR] 
 /home/chuan/trunk/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[102,12]
  cannot find symbol
 [ERROR] symbol  : method stop()
 [ERROR] location: class org.mortbay.jetty.Server
 [ERROR] - [Help 1]
 [ERROR]
 [ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
 switch.
 [ERROR] Re-run Maven using the -X switch to enable full debug logging.
 [ERROR]
 [ERROR] For more information about the errors and possible solutions, please 
 read the following articles:
 [ERROR] [Help 1] 
 http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
 [ERROR]
 [ERROR] After correcting the problems, you can resume the build with the 
 command
 [ERROR]   mvn goals -rf :hadoop-auth
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.1#6144)


Re: [VOTE] Release Apache Hadoop 2.0.6-alpha (RC1)

2013-08-23 Thread Konstantin Boudnik
I am clearly +1 (non-binding) on the release.

With 8 +1s (5 binding), no -1s or 0s the vote passes.

Thanks to all who verified the bits, I'll push them out shortly.

Thanks,
  Cos

On Thu, Aug 15, 2013 at 10:29PM, Konstantin Boudnik wrote:
 All,
 
 I have created a release candidate (rc1) for hadoop-2.0.6-alpha that I would
 like to release.
 
 This is a stabilization release that includes fixed for a couple a of issues
 as outlined on the security list.
 
 The RC is available at: http://people.apache.org/~cos/hadoop-2.0.6-alpha-rc1/
 The RC tag in svn is here: 
 http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.6-alpha-rc1
 
 The maven artifacts are available via repository.apache.org.
 
 The only difference between rc0 and rc1 is ASL added to releasenotes.html and
 updated release dates in CHANGES.txt files.
 
 Please try the release bits and vote; the vote will run for the usual 7 days.
 
 Thanks for your voting
   Cos
 




signature.asc
Description: Digital signature


Release Apache Hadoop 2.0.6-alpha

2013-08-23 Thread Konstantin Boudnik
All,

I have pushed the bits of 2.0.6-alpha into the open and released staging bits
in Nexus. Site is updated with the release news. 

2.0.6-alpha is officially live now. Thanks everybody for your help!
  Cos

On Thu, Aug 22, 2013 at 11:09PM, Konstantin Boudnik wrote:
 I am clearly +1 (non-binding) on the release.
 
 With 8 +1s (5 binding), no -1s or 0s the vote passes.
 
 Thanks to all who verified the bits, I'll push them out shortly.
 
 Thanks,
   Cos
 
 On Thu, Aug 15, 2013 at 10:29PM, Konstantin Boudnik wrote:
  All,
  
  I have created a release candidate (rc1) for hadoop-2.0.6-alpha that I would
  like to release.
  
  This is a stabilization release that includes fixed for a couple a of issues
  as outlined on the security list.
  
  The RC is available at: 
  http://people.apache.org/~cos/hadoop-2.0.6-alpha-rc1/
  The RC tag in svn is here: 
  http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.6-alpha-rc1
  
  The maven artifacts are available via repository.apache.org.
  
  The only difference between rc0 and rc1 is ASL added to releasenotes.html 
  and
  updated release dates in CHANGES.txt files.
  
  Please try the release bits and vote; the vote will run for the usual 7 
  days.
  
  Thanks for your voting
Cos
  
 
 




Re: Release Apache Hadoop 2.0.6-alpha

2013-08-23 Thread Konstantin Boudnik
Can anyone with admin rights in YARN JIRA project release 2.0.6-alpha version?

Thanks,
  Cos

On Fri, Aug 23, 2013 at 11:44AM, Konstantin Boudnik wrote:
 All,
 
 I have pushed the bits of 2.0.6-alpha into the open and released staging bits
 in Nexus. Site is updated with the release news. 
 
 2.0.6-alpha is officially live now. Thanks everybody for your help!
   Cos
 
 On Thu, Aug 22, 2013 at 11:09PM, Konstantin Boudnik wrote:
  I am clearly +1 (non-binding) on the release.
  
  With 8 +1s (5 binding), no -1s or 0s the vote passes.
  
  Thanks to all who verified the bits, I'll push them out shortly.
  
  Thanks,
Cos
  
  On Thu, Aug 15, 2013 at 10:29PM, Konstantin Boudnik wrote:
   All,
   
   I have created a release candidate (rc1) for hadoop-2.0.6-alpha that I 
   would
   like to release.
   
   This is a stabilization release that includes fixed for a couple a of 
   issues
   as outlined on the security list.
   
   The RC is available at: 
   http://people.apache.org/~cos/hadoop-2.0.6-alpha-rc1/
   The RC tag in svn is here: 
   http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.6-alpha-rc1
   
   The maven artifacts are available via repository.apache.org.
   
   The only difference between rc0 and rc1 is ASL added to releasenotes.html 
   and
   updated release dates in CHANGES.txt files.
   
   Please try the release bits and vote; the vote will run for the usual 7 
   days.
   
   Thanks for your voting
 Cos
   
  
  
 
 


signature.asc
Description: Digital signature


Re: [VOTE] Release Apache Hadoop 2.0.6-alpha

2013-08-15 Thread Konstantin Boudnik
Darn CHANGES - apparently they just hate me. I agree that needs to be
addressed. I will spin-up rc1 tonight and restart the vote.

Thanks,
  Cos

On Wed, Aug 14, 2013 at 10:40AM, Alejandro Abdelnur wrote:
 OK:
 * verified MD5
 * verified signature
 * expanded source tar and did a build
 * configured pseudo cluster and run a couple of example MR jobs
 * did a few HTTP calls to HTTFS
 
 NOT OK:
 * CHANGES.txt files have 2.0.6 as UNRELEASED, they should have the date the
 RC vote ends
 * 'mvn apache-rat:check' fails, releasenotes HTML files don't have license
 headers,
 
 I think we need to address the NO OK points (specially the last one), they
 are trivial.
 
 Thanks.
 
 
 
 On Sat, Aug 10, 2013 at 5:46 PM, Konstantin Boudnik c...@apache.org wrote:
 
  All,
 
  I have created a release candidate (rc0) for hadoop-2.0.6-alpha that I
  would
  like to release.
 
  This is a stabilization release that includes fixed for a couple a of
  issues
  as outlined on the security list.
 
  The RC is available at:
  http://people.apache.org/~cos/hadoop-2.0.6-alpha-rc0/
  The RC tag in svn is here:
  http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.6-alpha-rc0
 
  The maven artifacts are available via repository.apache.org.
 
  Please try the release bits and vote; the vote will run for the usual 7
  days.
 
  Thanks for your voting
Cos
 
 
 
 
 -- 
 Alejandro


Re: [VOTE] Release Apache Hadoop 2.0.6-alpha

2013-08-15 Thread Konstantin Boudnik
Alejandro,

looking into the source code: it seems that release notes never had license
boilerplate in it, hence 2.0.6-alpha doesn't have as well.

I have fixed CHANGES with new optimistic date of the release and upload rc1
right now.

Please let me know if you feel like we need start doing the license for the
releasenotes in this release.

Thanks,
  Cos

On Wed, Aug 14, 2013 at 10:40AM, Alejandro Abdelnur wrote:
 OK:
 * verified MD5
 * verified signature
 * expanded source tar and did a build
 * configured pseudo cluster and run a couple of example MR jobs
 * did a few HTTP calls to HTTFS
 
 NOT OK:
 * CHANGES.txt files have 2.0.6 as UNRELEASED, they should have the date the
 RC vote ends
 * 'mvn apache-rat:check' fails, releasenotes HTML files don't have license
 headers,
 
 I think we need to address the NO OK points (specially the last one), they
 are trivial.
 
 Thanks.
 
 
 
 On Sat, Aug 10, 2013 at 5:46 PM, Konstantin Boudnik c...@apache.org wrote:
 
  All,
 
  I have created a release candidate (rc0) for hadoop-2.0.6-alpha that I
  would
  like to release.
 
  This is a stabilization release that includes fixed for a couple a of
  issues
  as outlined on the security list.
 
  The RC is available at:
  http://people.apache.org/~cos/hadoop-2.0.6-alpha-rc0/
  The RC tag in svn is here:
  http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.6-alpha-rc0
 
  The maven artifacts are available via repository.apache.org.
 
  Please try the release bits and vote; the vote will run for the usual 7
  days.
 
  Thanks for your voting
Cos
 
 
 
 
 -- 
 Alejandro


signature.asc
Description: Digital signature


Re: [VOTE] Release Apache Hadoop 2.0.6-alpha

2013-08-15 Thread Konstantin Boudnik
Sure Alejandro, not a big deal - I agree. Uploaded RC1 and restartig the vote
right away. Thanks for catching this!

Cos

On Thu, Aug 15, 2013 at 09:08PM, Alejandro Abdelnur wrote:
 it should be straight forward adding the license headers to the release
 notes. please make sure apache-rat:check passes on the RC before publishing
 it.
 
 Arun, as you are about to cut the new RC for 2.1.0-beta, can you please
 make sure the license headers are used in the releasenotes HTML files?
 
 Thx
 
 
 On Thu, Aug 15, 2013 at 8:02 PM, Konstantin Boudnik c...@apache.org wrote:
 
  Alejandro,
 
  looking into the source code: it seems that release notes never had license
  boilerplate in it, hence 2.0.6-alpha doesn't have as well.
 
  I have fixed CHANGES with new optimistic date of the release and upload rc1
  right now.
 
  Please let me know if you feel like we need start doing the license for the
  releasenotes in this release.
 
  Thanks,
Cos
 
  On Wed, Aug 14, 2013 at 10:40AM, Alejandro Abdelnur wrote:
   OK:
   * verified MD5
   * verified signature
   * expanded source tar and did a build
   * configured pseudo cluster and run a couple of example MR jobs
   * did a few HTTP calls to HTTFS
  
   NOT OK:
   * CHANGES.txt files have 2.0.6 as UNRELEASED, they should have the date
  the
   RC vote ends
   * 'mvn apache-rat:check' fails, releasenotes HTML files don't have
  license
   headers,
  
   I think we need to address the NO OK points (specially the last one),
  they
   are trivial.
  
   Thanks.
  
  
  
   On Sat, Aug 10, 2013 at 5:46 PM, Konstantin Boudnik c...@apache.org
  wrote:
  
All,
   
I have created a release candidate (rc0) for hadoop-2.0.6-alpha that I
would
like to release.
   
This is a stabilization release that includes fixed for a couple a of
issues
as outlined on the security list.
   
The RC is available at:
http://people.apache.org/~cos/hadoop-2.0.6-alpha-rc0/
The RC tag in svn is here:
   
  http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.6-alpha-rc0
   
The maven artifacts are available via repository.apache.org.
   
Please try the release bits and vote; the vote will run for the usual 7
days.
   
Thanks for your voting
  Cos
   
   
  
  
   --
   Alejandro
 
 
 
 
 -- 
 Alejandro


signature.asc
Description: Digital signature


[VOTE] Release Apache Hadoop 2.0.6-alpha (RC1)

2013-08-15 Thread Konstantin Boudnik
All,

I have created a release candidate (rc1) for hadoop-2.0.6-alpha that I would
like to release.

This is a stabilization release that includes fixed for a couple a of issues
as outlined on the security list.

The RC is available at: http://people.apache.org/~cos/hadoop-2.0.6-alpha-rc1/
The RC tag in svn is here: 
http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.6-alpha-rc1

The maven artifacts are available via repository.apache.org.

The only difference between rc0 and rc1 is ASL added to releasenotes.html and
updated release dates in CHANGES.txt files.

Please try the release bits and vote; the vote will run for the usual 7 days.

Thanks for your voting
  Cos



signature.asc
Description: Digital signature


[VOTE] Release Apache Hadoop 2.0.6-alpha

2013-08-10 Thread Konstantin Boudnik
All,

I have created a release candidate (rc0) for hadoop-2.0.6-alpha that I would
like to release.

This is a stabilization release that includes fixed for a couple a of issues
as outlined on the security list.

The RC is available at: http://people.apache.org/~cos/hadoop-2.0.6-alpha-rc0/
The RC tag in svn is here: 
http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.6-alpha-rc0

The maven artifacts are available via repository.apache.org.

Please try the release bits and vote; the vote will run for the usual 7 days.

Thanks for your voting
  Cos



signature.asc
Description: Digital signature


Re: creating 2.2.0 version in JIRA

2013-07-05 Thread Konstantin Boudnik
Same here.

On Tue, Jul 02, 2013 at 03:54PM, Alejandro Abdelnur wrote:
 We need clarification on this then.
 
 I was under the impression that branch-2 would be 2.2.0.
 
 thx
 
 On Tue, Jul 2, 2013 at 2:38 PM, Jason Lowe jl...@yahoo-inc.com wrote:
 
  I thought Arun intends for 2.2.0 to be created off of branch-2.1.0-beta
  and not off of branch-2.  As I understand it, only critical blockers will
  be the delta between 2.1.0-beta and 2.2.0 and items checked into branch-2
  should be marked as  fixed in 2.3.0.
 
  Part of the confusion is that currently branch-2 builds as 2.2.0-SNAPSHOT,
  but I believe Arun intended it to be 2.3.0-SNAPSHOT.
 
  Jason
 
 
  On 06/21/2013 12:05 PM, Alejandro Abdelnur wrote:
 
  Thanks Suresh, didn't know that, will do.
 
 
  On Fri, Jun 21, 2013 at 9:48 AM, Suresh Srinivas sur...@hortonworks.com
  wrote:
 
   I have added in to HDFS, HADOOP, MAPREDUCE projects. Can someone add it
  for
  YARN?
 
 
  On Fri, Jun 21, 2013 at 9:35 AM, Alejandro Abdelnur t...@cloudera.com
 
  wrote:
  When Arun created branch-2.1-beta he stated:
 
   The expectation is that 2.2.0 will be limited to content in
 
  branch-2.1-beta
 
  and we stick to stabilizing it henceforth (I've deliberately not
 
  created
 
  2.2.0
 
  fix-version on jira yet).
 
  I working/committing some JIRAs that I'm putting in branch-2 (testcases
 
  and
 
  improvements) but I don't want to put them in branch-2.1-beta as they
  are
  not critical and I don't won't add unnecessary noise to the
 
  branch-2.1-beta
 
  release work.
 
  Currently branch-2 POMs have a version 2.2.0 and the CHANGES.txt files
  as
  well.
 
  But because we did not create a JIRA version I cannot close those JIRAs.
 
  Can we please create the JIRA versions? later we can rename them.
 
  Thx
 
 
  --
  Alejandro
 
 
 
  --
  http://hortonworks.com/**download/ http://hortonworks.com/download/
 
 
 
 
 
 
 
 -- 
 Alejandro


[VOTE RESULT] Release Apache Hadoop 2.0.5-alpha (rc2)

2013-06-06 Thread Konstantin Boudnik
I am clearly +1 (non-binding) on the release.

With 8 +1s (3 binding), no -1s or 0s the vote passes.

Thanks to all who verified the bits, I'll push them out shortly.

Thanks,
  Cos

On Mon, Jun 03, 2013 at 12:51PM, Konstantin Boudnik wrote:
 I have rolled out release candidate (rc2) for hadoop-2.0.5-alpha.
 
 The difference between rc1 and rc2 is the optimistic release date is set for
 06/06/2013 in the CHANGES.txt files.
 
 The binary artifact is the same - there's no need to rebuild it. The maven
 artifacts are the same.
 
 The difference between the two RCs:
 
 svn diff \
   
 https://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc1/ \
   https://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc2/
 
 New RC builds are uploaded to the web.
 The RC is available at: http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc2/
 The RC tag in svn is here: 
 http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc2
 
 I would like to extend the vote for another three days for it is such a minor
 change that doesn't affect anything but the recorded release date. Please
 cast your vote before 06/06/2013 5pm PDT.
 
 Thanks for your patience!
   Cos
 
 On Fri, May 31, 2013 at 09:27PM, Konstantin Boudnik wrote:
  All,
  
  I have created a release candidate (rc1) for hadoop-2.0.5-alpha that I would
  like to release.
  
  This is a stabilization release that includes fixed for a couple a of issues
  discovered in the testing with BigTop 0.6.0 release candidate.
  
  The RC is available at: 
  http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc1/
  The RC tag in svn is here: 
  http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc1
  
  The maven artifacts will be available via repository.apache.org on Sat, June
  1st, 2013 at 2 pm PDT as outlined here
  http://s.apache.org/WKD
  
  Please try the release bits and vote; the vote will run for the 3 days,
  because this is just a version name change. The bits are identical to the 
  ones
  voted on before in 
  http://s.apache.org/2041move
  
  Thanks for your voting
Cos
  




Re: [VOTE] Release Apache Hadoop 2.0.5-alpha (rc2)

2013-06-04 Thread Konstantin Boudnik
Good point Thomas - I have just fixed the name of the file. Nice catch!

Sigh... these three text files... If anyone feels strongly about having them
in place - I will update the tarballs and recalculate the checksums. Or just
leave it as is, perhaps? So, either way is fine with me.

Thanks,
  Cos

On Tue, Jun 04, 2013 at 08:47PM, Thomas Graves wrote:
 Cos,
 
 The name of the mds file for the binary tar is missing a . in your
 apache directory: hadoop-2.0.5-alphatar.gz.mds
 http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc2/hadoop-2.0.5-alphatar
 .gz.mds
 
 Contents of it are fine though so you might just add the ..
 
 Also the NOTICE.txt, README.txt, and LICENSE.txt files are missing from
 the top level directory in both the src and binary tar ball.
 
 Everything else looks good. Verified checksums, signatures, built, ran
 single node cluster.
 
 Thanks,
 Tom
  
 
 
 On 6/3/13 2:51 PM, Konstantin Boudnik c...@apache.org wrote:
 
 I have rolled out release candidate (rc2) for hadoop-2.0.5-alpha.
 
 The difference between rc1 and rc2 is the optimistic release date is
 set for
 06/06/2013 in the CHANGES.txt files.
 
 The binary artifact is the same - there's no need to rebuild it. The maven
 artifacts are the same.
 
 The difference between the two RCs:
 
 svn diff \
   
 https://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc
 1/ \
   
 https://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc
 2/
 
 New RC builds are uploaded to the web.
 The RC is available at:
 http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc2/
 The RC tag in svn is here:
 http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc2
 
 I would like to extend the vote for another three days for it is such a
 minor
 change that doesn't affect anything but the recorded release date. Please
 cast your vote before 06/06/2013 5pm PDT.
 
 Thanks for your patience!
   Cos
 
 On Fri, May 31, 2013 at 09:27PM, Konstantin Boudnik wrote:
  All,
  
  I have created a release candidate (rc1) for hadoop-2.0.5-alpha that I
 would
  like to release.
  
  This is a stabilization release that includes fixed for a couple a of
 issues
  discovered in the testing with BigTop 0.6.0 release candidate.
  
  The RC is available at:
 http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc1/
  The RC tag in svn is here:
 http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc
 1
  
  The maven artifacts will be available via repository.apache.org on Sat,
 June
  1st, 2013 at 2 pm PDT as outlined here
  http://s.apache.org/WKD
  
  Please try the release bits and vote; the vote will run for the 3 days,
  because this is just a version name change. The bits are identical to
 the ones
  voted on before in
  http://s.apache.org/2041move
  
  Thanks for your voting
Cos
  
 


Re: [VOTE] Release Apache Hadoop 2.0.5-alpha

2013-06-03 Thread Konstantin Boudnik
Ok, if this the consensus on the issue, then tonight I will cut rc2 with only
CHANGES.txt release dates update. After that I will extend the vote once
again.

Sorry for the hassle to all who did voted before.
  Cos

On Mon, Jun 03, 2013 at 12:18AM, Doug Cutting wrote:
 On Sun, Jun 2, 2013 at 9:01 AM, Alejandro Abdelnur t...@cloudera.com wrote:
  [...] an easy way to solve it it would be using as release date the date 
  the vote ends.
 
 +1 This is what I've done for release candidate builds.
 
 Doug


Re: [VOTE] Release Apache Hadoop 2.0.5-alpha (rc2)

2013-06-03 Thread Konstantin Boudnik
I have rolled out release candidate (rc2) for hadoop-2.0.5-alpha.

The difference between rc1 and rc2 is the optimistic release date is set for
06/06/2013 in the CHANGES.txt files.

The binary artifact is the same - there's no need to rebuild it. The maven
artifacts are the same.

The difference between the two RCs:

svn diff \
  https://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc1/ \
  https://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc2/

New RC builds are uploaded to the web.
The RC is available at: http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc2/
The RC tag in svn is here: 
http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc2

I would like to extend the vote for another three days for it is such a minor
change that doesn't affect anything but the recorded release date. Please
cast your vote before 06/06/2013 5pm PDT.

Thanks for your patience!
  Cos

On Fri, May 31, 2013 at 09:27PM, Konstantin Boudnik wrote:
 All,
 
 I have created a release candidate (rc1) for hadoop-2.0.5-alpha that I would
 like to release.
 
 This is a stabilization release that includes fixed for a couple a of issues
 discovered in the testing with BigTop 0.6.0 release candidate.
 
 The RC is available at: http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc1/
 The RC tag in svn is here: 
 http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc1
 
 The maven artifacts will be available via repository.apache.org on Sat, June
 1st, 2013 at 2 pm PDT as outlined here
 http://s.apache.org/WKD
 
 Please try the release bits and vote; the vote will run for the 3 days,
 because this is just a version name change. The bits are identical to the ones
 voted on before in 
 http://s.apache.org/2041move
 
 Thanks for your voting
   Cos
 


signature.asc
Description: Digital signature


Re: [VOTE] Release Apache Hadoop 2.0.5-alpha

2013-06-02 Thread Konstantin Boudnik
Alejandro,

I believe this is chicken and egg problem: I can't put release date into
unreleased tarball. Looking into 2.0.4-alpha source tarball I see the same
situation:

Release 2.0.4-alpha - UNRELEASED

But in the trunk and branch-2 the release data is in place. I don't think this
is an issue. But I would be happy to fix it if this seems to be a problem.

Thanks,
Cos

On Sat, Jun 01, 2013 at 08:04PM, Alejandro Abdelnur wrote:
 On RC1, verified MD5  signature, built, configured pseudo cluster, run a
 couple of sample jobs, tested HTTPFS.
 
 CHANGES.txt files contents are correct now. Still, a minor NIT, they have
 2.0.5 as UNRELEASED, shouldn't they have a date (I would assume the date
 the vote ends).
 
 Thanks
 
 
 On Fri, May 31, 2013 at 9:39 PM, J. Rottinghuis jrottingh...@gmail.comwrote:
 
  Thanks for fixing Cos.
  http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc1/
  looks good to me.
  +1 (non-binding)
 
  Thanks,
 
  Joep
 
 
  On Fri, May 31, 2013 at 8:25 PM, Konstantin Boudnik c...@apache.org
  wrote:
 
   Ok, WRT HDFS-4646 - it is all legit and the code is in branch-2.0.4-alpha
   and
   later. It has been committed as
 r1465124
   The reason it isn't normally visible because of the weird commit message:
  
   svn merge -c1465121 from trunk
  
   So, we good. I am done with the CHANGES.txt fixed that you guys have
  noted
   earlier and will be re-spinning RC1 in a few.
  
   Cos
  
   On Fri, May 31, 2013 at 08:07PM, Konstantin Boudnik wrote:
Alejandro,
   
thanks for looking into this. Indeed - I missed the 2.0.5-alpha section
   in
YARN CHANGES.txt. Added now. As for HDFS-4646: apparently I didn't get
   into
to branch-2.0.4-alpha back then, although I distinctively remember
  doing
   this.
Let me pull it into 2.0.5-alpha and update CHANGES.txt to reflect it.
   Also, I
will do JIRA in a moment.
   
Joep, appreciate the thorough examination. I have fixed the dates for
  the
releases 2.0.4-alpha. As for the top-level readme file - sorry I wasn't
   aware
about them. As for the binary: I am pretty sure we are only releasing
   source
code, but I will put binaries into the rc1 respin.
   
I will respin rc1 shortly. Appreciate the feedback!
  Cos
   
On Fri, May 31, 2013 at 05:27PM, Alejandro Abdelnur wrote:
 Verified MD5  signature, built, configured pseudo cluster, run a
   couple of
 sample jobs, tested HTTPFS.

 Still, something seems odd.

 The HDFS CHANGES.txt has the following entry under 2.0.5-alpha:

  HDFS-4646. createNNProxyWithClientProtocol ignores configured
  timeout
 value  (Jagane Sundar via cos)

 but I don't see that in the branch.

 And, the YARN CHANGES.txt does not have the 2.0.5-alpha section (it
   should
 be there empty).

 Cos, can you please look at these 2 things and explain/fix?

 Thanks.



 On Fri, May 31, 2013 at 4:04 PM, Konstantin Boudnik c...@apache.org
   wrote:

  All,
 
  I have created a release candidate (rc0) for hadoop-2.0.5-alpha
  that
   I
  would
  like to release.
 
  This is a stabilization release that includes fixed for a couple a
  of
  issues
  discovered in the testing with BigTop 0.6.0 release candidate.
 
  The RC is available at:
  http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc0/
  The RC tag in svn is here:
 
  
  http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc0
 
  The maven artifacts will be available via repository.apache.org on
   Sat,
  June
  1st, 2013 at 2 pm PDT as outlined here
  http://s.apache.org/WKD
 
  Please try the release bits and vote; the vote will run for the 3
   days,
  because this is just a version name change. The bits are identical
   to the
  ones
  voted on before in
  http://s.apache.org/2041move
 
  Thanks for your voting
Cos
 
 


 --
 Alejandro
  
  
  
 
 
 
 
 -- 
 Alejandro


Re: [VOTE] Release Apache Hadoop 2.0.5-alpha

2013-06-02 Thread Konstantin Boudnik
This is my first Hadoop release, so I am all ears ;) I looked at how Arun was
cutting the previous 2.0.x and followed the example. 

Thanks for your input and help Alejandro!

Regards,
  Cos

On Sun, Jun 02, 2013 at 09:01AM, Alejandro Abdelnur wrote:
 I know we had this issue before. But and easy way to solve it it would be
 using as release date the date the vote ends. Anyway, not a big deal if we
 are not cutting a new rc for other reason
 
 +1 on rc1
 
 Thx
 
 On Jun 2, 2013, at 12:04 AM, Konstantin Boudnik c...@apache.org wrote:
 
  Alejandro,
  
  I believe this is chicken and egg problem: I can't put release date into
  unreleased tarball. Looking into 2.0.4-alpha source tarball I see the same
  situation:
  
  Release 2.0.4-alpha - UNRELEASED
  
  But in the trunk and branch-2 the release data is in place. I don't think 
  this
  is an issue. But I would be happy to fix it if this seems to be a problem.
  
  Thanks,
  Cos
  
  On Sat, Jun 01, 2013 at 08:04PM, Alejandro Abdelnur wrote:
  On RC1, verified MD5  signature, built, configured pseudo cluster, run a
  couple of sample jobs, tested HTTPFS.
  
  CHANGES.txt files contents are correct now. Still, a minor NIT, they have
  2.0.5 as UNRELEASED, shouldn't they have a date (I would assume the date
  the vote ends).
  
  Thanks
  
  
  On Fri, May 31, 2013 at 9:39 PM, J. Rottinghuis 
  jrottingh...@gmail.comwrote:
  
  Thanks for fixing Cos.
  http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc1/
  looks good to me.
  +1 (non-binding)
  
  Thanks,
  
  Joep
  
  
  On Fri, May 31, 2013 at 8:25 PM, Konstantin Boudnik c...@apache.org
  wrote:
  
  Ok, WRT HDFS-4646 - it is all legit and the code is in branch-2.0.4-alpha
  and
  later. It has been committed as
   r1465124
  The reason it isn't normally visible because of the weird commit message:
  
 svn merge -c1465121 from trunk
  
  So, we good. I am done with the CHANGES.txt fixed that you guys have
  noted
  earlier and will be re-spinning RC1 in a few.
  
  Cos
  
  On Fri, May 31, 2013 at 08:07PM, Konstantin Boudnik wrote:
  Alejandro,
  
  thanks for looking into this. Indeed - I missed the 2.0.5-alpha section
  in
  YARN CHANGES.txt. Added now. As for HDFS-4646: apparently I didn't get
  into
  to branch-2.0.4-alpha back then, although I distinctively remember
  doing
  this.
  Let me pull it into 2.0.5-alpha and update CHANGES.txt to reflect it.
  Also, I
  will do JIRA in a moment.
  
  Joep, appreciate the thorough examination. I have fixed the dates for
  the
  releases 2.0.4-alpha. As for the top-level readme file - sorry I wasn't
  aware
  about them. As for the binary: I am pretty sure we are only releasing
  source
  code, but I will put binaries into the rc1 respin.
  
  I will respin rc1 shortly. Appreciate the feedback!
   Cos
  
  On Fri, May 31, 2013 at 05:27PM, Alejandro Abdelnur wrote:
  Verified MD5  signature, built, configured pseudo cluster, run a
  couple of
  sample jobs, tested HTTPFS.
  
  Still, something seems odd.
  
  The HDFS CHANGES.txt has the following entry under 2.0.5-alpha:
  
  HDFS-4646. createNNProxyWithClientProtocol ignores configured
  timeout
  value  (Jagane Sundar via cos)
  
  but I don't see that in the branch.
  
  And, the YARN CHANGES.txt does not have the 2.0.5-alpha section (it
  should
  be there empty).
  
  Cos, can you please look at these 2 things and explain/fix?
  
  Thanks.
  
  
  
  On Fri, May 31, 2013 at 4:04 PM, Konstantin Boudnik c...@apache.org
  wrote:
  
  All,
  
  I have created a release candidate (rc0) for hadoop-2.0.5-alpha
  that
  I
  would
  like to release.
  
  This is a stabilization release that includes fixed for a couple a
  of
  issues
  discovered in the testing with BigTop 0.6.0 release candidate.
  
  The RC is available at:
  http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc0/
  The RC tag in svn is here:
  
  
  http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc0
  
  The maven artifacts will be available via repository.apache.org on
  Sat,
  June
  1st, 2013 at 2 pm PDT as outlined here
 http://s.apache.org/WKD
  
  Please try the release bits and vote; the vote will run for the 3
  days,
  because this is just a version name change. The bits are identical
  to the
  ones
  voted on before in
 http://s.apache.org/2041move
  
  Thanks for your voting
   Cos
  
  
  
  
  --
  Alejandro
  
  
  
  
  
  
  
  -- 
  Alejandro


signature.asc
Description: Digital signature


Re: [VOTE] Release Apache Hadoop 2.0.5-alpha

2013-06-02 Thread Konstantin Boudnik
On Sun, Jun 02, 2013 at 03:04PM, Arun C Murthy wrote:
 I record the 'release date' as the one on which the build is created. Not
 sure what Matt actually does on branch-1. Matt?

Arun, but you do this not in the release source tarball, right? Just in the
rest of integration branches, ie branch-2 and trunk. Am I correct?

Thanks,
  Cos

 hth,
 Arun
 
 On Jun 2, 2013, at 12:33 PM, Konstantin Boudnik wrote:
 
  This is my first Hadoop release, so I am all ears ;) I looked at how Arun 
  was
  cutting the previous 2.0.x and followed the example. 
  
  Thanks for your input and help Alejandro!
  
  Regards,
   Cos
  
  On Sun, Jun 02, 2013 at 09:01AM, Alejandro Abdelnur wrote:
  I know we had this issue before. But and easy way to solve it it would be
  using as release date the date the vote ends. Anyway, not a big deal if we
  are not cutting a new rc for other reason
  
  +1 on rc1
  
  Thx
  
  On Jun 2, 2013, at 12:04 AM, Konstantin Boudnik c...@apache.org wrote:
  
  Alejandro,
  
  I believe this is chicken and egg problem: I can't put release date into
  unreleased tarball. Looking into 2.0.4-alpha source tarball I see the same
  situation:
  
  Release 2.0.4-alpha - UNRELEASED
  
  But in the trunk and branch-2 the release data is in place. I don't think 
  this
  is an issue. But I would be happy to fix it if this seems to be a problem.
  
  Thanks,
  Cos
  
  On Sat, Jun 01, 2013 at 08:04PM, Alejandro Abdelnur wrote:
  On RC1, verified MD5  signature, built, configured pseudo cluster, run a
  couple of sample jobs, tested HTTPFS.
  
  CHANGES.txt files contents are correct now. Still, a minor NIT, they have
  2.0.5 as UNRELEASED, shouldn't they have a date (I would assume the date
  the vote ends).
  
  Thanks
  
  
  On Fri, May 31, 2013 at 9:39 PM, J. Rottinghuis 
  jrottingh...@gmail.comwrote:
  
  Thanks for fixing Cos.
  http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc1/
  looks good to me.
  +1 (non-binding)
  
  Thanks,
  
  Joep
  
  
  On Fri, May 31, 2013 at 8:25 PM, Konstantin Boudnik c...@apache.org
  wrote:
  
  Ok, WRT HDFS-4646 - it is all legit and the code is in 
  branch-2.0.4-alpha
  and
  later. It has been committed as
  r1465124
  The reason it isn't normally visible because of the weird commit 
  message:
  
svn merge -c1465121 from trunk
  
  So, we good. I am done with the CHANGES.txt fixed that you guys have
  noted
  earlier and will be re-spinning RC1 in a few.
  
  Cos
  
  On Fri, May 31, 2013 at 08:07PM, Konstantin Boudnik wrote:
  Alejandro,
  
  thanks for looking into this. Indeed - I missed the 2.0.5-alpha 
  section
  in
  YARN CHANGES.txt. Added now. As for HDFS-4646: apparently I didn't get
  into
  to branch-2.0.4-alpha back then, although I distinctively remember
  doing
  this.
  Let me pull it into 2.0.5-alpha and update CHANGES.txt to reflect it.
  Also, I
  will do JIRA in a moment.
  
  Joep, appreciate the thorough examination. I have fixed the dates for
  the
  releases 2.0.4-alpha. As for the top-level readme file - sorry I 
  wasn't
  aware
  about them. As for the binary: I am pretty sure we are only releasing
  source
  code, but I will put binaries into the rc1 respin.
  
  I will respin rc1 shortly. Appreciate the feedback!
  Cos
  
  On Fri, May 31, 2013 at 05:27PM, Alejandro Abdelnur wrote:
  Verified MD5  signature, built, configured pseudo cluster, run a
  couple of
  sample jobs, tested HTTPFS.
  
  Still, something seems odd.
  
  The HDFS CHANGES.txt has the following entry under 2.0.5-alpha:
  
  HDFS-4646. createNNProxyWithClientProtocol ignores configured
  timeout
  value  (Jagane Sundar via cos)
  
  but I don't see that in the branch.
  
  And, the YARN CHANGES.txt does not have the 2.0.5-alpha section (it
  should
  be there empty).
  
  Cos, can you please look at these 2 things and explain/fix?
  
  Thanks.
  
  
  
  On Fri, May 31, 2013 at 4:04 PM, Konstantin Boudnik c...@apache.org
  wrote:
  
  All,
  
  I have created a release candidate (rc0) for hadoop-2.0.5-alpha
  that
  I
  would
  like to release.
  
  This is a stabilization release that includes fixed for a couple a
  of
  issues
  discovered in the testing with BigTop 0.6.0 release candidate.
  
  The RC is available at:
  http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc0/
  The RC tag in svn is here:
  
  
  http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc0
  
  The maven artifacts will be available via repository.apache.org on
  Sat,
  June
  1st, 2013 at 2 pm PDT as outlined here
http://s.apache.org/WKD
  
  Please try the release bits and vote; the vote will run for the 3
  days,
  because this is just a version name change. The bits are identical
  to the
  ones
  voted on before in
http://s.apache.org/2041move
  
  Thanks for your voting
  Cos
  
  
  
  
  --
  Alejandro
  
  
  
  
  
  
  
  -- 
  Alejandro
 
 --
 Arun C. Murthy
 Hortonworks Inc.
 http://hortonworks.com/
 
 


Re: [VOTE] Release Apache Hadoop 2.0.5-alpha

2013-06-02 Thread Konstantin Boudnik
On Sun, Jun 02, 2013 at 09:17PM, Arun C Murthy wrote:
 I do it in the release branch, so the release source tarball has the date.

I guess 2.0.4-alpha was cut differently for whatever reason, 'cause i see the
UNRELEASED date in the source tarball.

As I said - if anyone feels strongly about it I will alter the CHANGES.txt
files

thanks,
  Cos

 Arun
 
 On Jun 2, 2013, at 6:41 PM, Konstantin Boudnik wrote:
 
  On Sun, Jun 02, 2013 at 03:04PM, Arun C Murthy wrote:
  I record the 'release date' as the one on which the build is created. Not
  sure what Matt actually does on branch-1. Matt?
  
  Arun, but you do this not in the release source tarball, right? Just in the
  rest of integration branches, ie branch-2 and trunk. Am I correct?
  
  Thanks,
   Cos
  
  hth,
  Arun
  
  On Jun 2, 2013, at 12:33 PM, Konstantin Boudnik wrote:
  
  This is my first Hadoop release, so I am all ears ;) I looked at how Arun 
  was
  cutting the previous 2.0.x and followed the example. 
  
  Thanks for your input and help Alejandro!
  
  Regards,
  Cos
  
  On Sun, Jun 02, 2013 at 09:01AM, Alejandro Abdelnur wrote:
  I know we had this issue before. But and easy way to solve it it would be
  using as release date the date the vote ends. Anyway, not a big deal if 
  we
  are not cutting a new rc for other reason
  
  +1 on rc1
  
  Thx
  
  On Jun 2, 2013, at 12:04 AM, Konstantin Boudnik c...@apache.org wrote:
  
  Alejandro,
  
  I believe this is chicken and egg problem: I can't put release date into
  unreleased tarball. Looking into 2.0.4-alpha source tarball I see the 
  same
  situation:
  
  Release 2.0.4-alpha - UNRELEASED
  
  But in the trunk and branch-2 the release data is in place. I don't 
  think this
  is an issue. But I would be happy to fix it if this seems to be a 
  problem.
  
  Thanks,
  Cos
  
  On Sat, Jun 01, 2013 at 08:04PM, Alejandro Abdelnur wrote:
  On RC1, verified MD5  signature, built, configured pseudo cluster, 
  run a
  couple of sample jobs, tested HTTPFS.
  
  CHANGES.txt files contents are correct now. Still, a minor NIT, they 
  have
  2.0.5 as UNRELEASED, shouldn't they have a date (I would assume the 
  date
  the vote ends).
  
  Thanks
  
  
  On Fri, May 31, 2013 at 9:39 PM, J. Rottinghuis 
  jrottingh...@gmail.comwrote:
  
  Thanks for fixing Cos.
  http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc1/
  looks good to me.
  +1 (non-binding)
  
  Thanks,
  
  Joep
  
  
  On Fri, May 31, 2013 at 8:25 PM, Konstantin Boudnik c...@apache.org
  wrote:
  
  Ok, WRT HDFS-4646 - it is all legit and the code is in 
  branch-2.0.4-alpha
  and
  later. It has been committed as
  r1465124
  The reason it isn't normally visible because of the weird commit 
  message:
  
   svn merge -c1465121 from trunk
  
  So, we good. I am done with the CHANGES.txt fixed that you guys have
  noted
  earlier and will be re-spinning RC1 in a few.
  
  Cos
  
  On Fri, May 31, 2013 at 08:07PM, Konstantin Boudnik wrote:
  Alejandro,
  
  thanks for looking into this. Indeed - I missed the 2.0.5-alpha 
  section
  in
  YARN CHANGES.txt. Added now. As for HDFS-4646: apparently I didn't 
  get
  into
  to branch-2.0.4-alpha back then, although I distinctively remember
  doing
  this.
  Let me pull it into 2.0.5-alpha and update CHANGES.txt to reflect 
  it.
  Also, I
  will do JIRA in a moment.
  
  Joep, appreciate the thorough examination. I have fixed the dates 
  for
  the
  releases 2.0.4-alpha. As for the top-level readme file - sorry I 
  wasn't
  aware
  about them. As for the binary: I am pretty sure we are only 
  releasing
  source
  code, but I will put binaries into the rc1 respin.
  
  I will respin rc1 shortly. Appreciate the feedback!
  Cos
  
  On Fri, May 31, 2013 at 05:27PM, Alejandro Abdelnur wrote:
  Verified MD5  signature, built, configured pseudo cluster, run a
  couple of
  sample jobs, tested HTTPFS.
  
  Still, something seems odd.
  
  The HDFS CHANGES.txt has the following entry under 2.0.5-alpha:
  
  HDFS-4646. createNNProxyWithClientProtocol ignores configured
  timeout
  value  (Jagane Sundar via cos)
  
  but I don't see that in the branch.
  
  And, the YARN CHANGES.txt does not have the 2.0.5-alpha section (it
  should
  be there empty).
  
  Cos, can you please look at these 2 things and explain/fix?
  
  Thanks.
  
  
  
  On Fri, May 31, 2013 at 4:04 PM, Konstantin Boudnik 
  c...@apache.org
  wrote:
  
  All,
  
  I have created a release candidate (rc0) for hadoop-2.0.5-alpha
  that
  I
  would
  like to release.
  
  This is a stabilization release that includes fixed for a couple a
  of
  issues
  discovered in the testing with BigTop 0.6.0 release candidate.
  
  The RC is available at:
  http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc0/
  The RC tag in svn is here:
  
  
  http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc0
  
  The maven artifacts will be available via repository.apache.org on
  Sat,
  June
  1st, 2013 at 2 pm PDT

Re: [VOTE] Release Apache Hadoop 2.0.4.1-alpha

2013-06-01 Thread Konstantin Boudnik
Chris,

with 2.0.5-alpha (rc1) out, would you please take a look at the release bits?
I assume the four-digit numbering scheme issue has been resolved now.

Regards,
  Cos

On Thu, May 30, 2013 at 06:18PM, Chris Douglas wrote:
 On Thu, May 30, 2013 at 5:51 PM, Konstantin Boudnik c...@apache.org wrote:
  I have no issues of changing the version to 2.0.5-alpha and restarting to 
  vote
  for the release content, e.g. 2 bug fixes. Shall I call 3 days re-vote 
  because
  of the number change?
 
 +1 Sounds great.
 
  Does the result of bylaw vote nullifies the unfinished vote started by Arun?
  Sorry, I am dense, apparently.
 
 Yes, nobody should feel bound by either vote. The bylaw change
 clarifies that release plans are for RMs to solicit feedback and gauge
 PMC support for an artifact, not pre-approvals for doing work.
 
  Can we limit the vote thread to the merits of the release then?
 
 Happily.
 
  That sound like adding an insult to injury, if my forth-language skills do 
  not
  mislead me.
 
 They do mislead you, or I've expressed the point imprecisely. We can
 take this offline. -C
 
On Thu, May 30, 2013 at 01:48PM, Chris Douglas wrote:
On Thu, May 30, 2013 at 10:57 AM, Arun C Murthy 
a...@hortonworks.com wrote:
 Why not include MAPREDUCE-4211 as well rather than create one 
 release per patch?
   
From Cos's description, it sounded like these were backports of fixes
to help Sqoop2 and fix some build issues. If it's not just to fixup
leftover bugs in 2.0.4 *once* so downstream projects can integrate
against 2.0.4.1, and this a release series, then I've completely
misunderstood the purpose.
   
Cos, are you planning 2.0.4.2?
   
 Also, this is the first time we are seeing a four-numbered scheme 
 in Hadoop. Why not call this 2.0.5-alpha?
   
Good point. Since it contains only backports from branch-2, it would
make sense for it to be an intermediate release.
   
I shouldn't have to say this, but I'm changing my vote to -1 while we
work this out. -C
   
 On May 24, 2013, at 8:48 PM, Konstantin Boudnik wrote:

 All,

 I have created a release candidate (rc0) for hadoop-2.0.4.1-alpha 
 that I would
 like to release.

 This is a stabilization release that includes fixed for a couple 
 a of issues
 discovered in the testing with BigTop 0.6.0 release candidate.

 The RC is available at: 
 http://people.apache.org/~cos/hadoop-2.0.4.1-alpha-rc0/
 The RC tag in svn is here: 
 http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.4.1-alpha-rc0

 The maven artifacts are available via repository.apache.org.

 Please try the release bits and vote; the vote will run for the 
 usual 7 days.

 Thanks for your voting
  Cos





signature.asc
Description: Digital signature


Re: Heads up: moving from 2.0.4.1-alpha to 2.0.5-alpha

2013-06-01 Thread Konstantin Boudnik
I changes are completed, I have updated branch-2 and trunk CHANGES.txt files
as planned. Version of the branch-2 is set to 2.1.0-beta; 2.0.5-alpha release
artifacts are deployed to the staging area.

Thanks for your patience, guys!
  Cos

On Fri, May 31, 2013 at 12:45PM, Konstantin Boudnik wrote:
 Guys,
 
 I will be performing some changes wrt to moving 2.0.4.1 release candidate to
 2.0.5 space. As outline below by Alejandro:
 
 1. I will create new 2.0.5-alpha branch from the current head of 2.0.4-alpha
 that contains 2.0.4.1 changes
 2. consequently, set the artifacts version on the new branch to be 2.0.5-alpha
 3. the CHANGES.txt will be updated accordingly on the new 2.0.5 branch
 4. At this point I can cut an RC and put it out for re-vote. The staging can
 be done after the next two steps.
 
 I will be doing all these modifications in the next hour or so.
 
 Tomorrow at 1 pm PDT I would like to:
 1. update the version of the artifacts on branch-2 to become 2.1.0-SNAPSHOT
 2. update the CHANGES.txt in the trunk and branch-2 to reflect new version 
 names
 3. at this point it should safe to do the staging for 2.0.5-alpha RC
 
 To avoid any collisions during the last two steps - especially 2. - I would
 ask everyone to hold off the modifications of the CHANGES.txt files on trunk
 and branch-2 between 1 pm and 2 pm PDT.
 
 Please let me know if you see any flaw above, questions.
   Cos
 
  As we change from 2.0.4.1 to 2.0.5 you'll need to do the following
  housekeeping as you work the new RC.
  
  * rename the svn branch
  * update the versions in the POMs
  * update the CHANGES.txt in trunk, branch-2 and the release branch
  * change the current 2.0.5 version in JIRA to 2.1.0, create a new 2.0.5
  version, change the fix version of the 2 JIRAs that make the RC
 
  I renamed 2.0.5-beta to 2.1.0-beta and 2.0.4.1-alpha to 2.0.5-alpha versions
  in jira for HADOOP, HDFS, YARN  MAPREDUCE.
 
  Please take care of the rest.
 
  Also, in branch-2, the version should be 2.1.0-SNAPSHOT.


signature.asc
Description: Digital signature


Re: [VOTE] Release Apache Hadoop 2.0.4.1-alpha

2013-05-31 Thread Konstantin Boudnik
Thanks for the headstart Arun!

I will be sending a separate email to the *-dev@ lists outlining the plan and
the schedule of the changes, so we won't step on each other feet, like Vinod
expressed earlier.

Cos

On Fri, May 31, 2013 at 11:20AM, Arun C Murthy wrote:
 
 On May 30, 2013, at 8:41 PM, Alejandro Abdelnur wrote:
 
  Konstantin, Cos,
  
  As we change from 2.0.4.1 to 2.0.5 you'll need to do the following
  housekeeping as you work the new RC.
  
  * rename the svn branch
  * update the versions in the POMs
  * update the CHANGES.txt in trunk, branch-2 and the release branch
  * change the current 2.0.5 version in JIRA to 2.1.0, create a new 2.0.5
  version, change the fix version of the 2 JIRAs that make the RC
 
 I renamed 2.0.5-beta to 2.1.0-beta and 2.0.4.1-alpha to 2.0.5-alpha versions
 in jira for HADOOP, HDFS, YARN  MAPREDUCE.
 
 Please take care of the rest.
 
 Also, in branch-2, the version should be 2.1.0-SNAPSHOT.
 
 thanks,
 Arun
 
  
  Thanks.
  
  
  On Thu, May 30, 2013 at 6:18 PM, Chris Douglas cdoug...@apache.org wrote:
  
  On Thu, May 30, 2013 at 5:51 PM, Konstantin Boudnik c...@apache.org
  wrote:
  I have no issues of changing the version to 2.0.5-alpha and restarting
  to vote
  for the release content, e.g. 2 bug fixes. Shall I call 3 days re-vote
  because
  of the number change?
  
  +1 Sounds great.
  
  Does the result of bylaw vote nullifies the unfinished vote started by
  Arun?
  Sorry, I am dense, apparently.
  
  Yes, nobody should feel bound by either vote. The bylaw change
  clarifies that release plans are for RMs to solicit feedback and gauge
  PMC support for an artifact, not pre-approvals for doing work.
  
  Can we limit the vote thread to the merits of the release then?
  
  Happily.
  
  That sound like adding an insult to injury, if my forth-language skills
  do not
  mislead me.
  
  They do mislead you, or I've expressed the point imprecisely. We can
  take this offline. -C
  
  On Thu, May 30, 2013 at 01:48PM, Chris Douglas wrote:
  On Thu, May 30, 2013 at 10:57 AM, Arun C Murthy 
  a...@hortonworks.com wrote:
  Why not include MAPREDUCE-4211 as well rather than create one
  release per patch?
  
  From Cos's description, it sounded like these were backports of
  fixes
  to help Sqoop2 and fix some build issues. If it's not just to
  fixup
  leftover bugs in 2.0.4 *once* so downstream projects can integrate
  against 2.0.4.1, and this a release series, then I've completely
  misunderstood the purpose.
  
  Cos, are you planning 2.0.4.2?
  
  Also, this is the first time we are seeing a four-numbered
  scheme in Hadoop. Why not call this 2.0.5-alpha?
  
  Good point. Since it contains only backports from branch-2, it
  would
  make sense for it to be an intermediate release.
  
  I shouldn't have to say this, but I'm changing my vote to -1
  while we
  work this out. -C
  
  On May 24, 2013, at 8:48 PM, Konstantin Boudnik wrote:
  
  All,
  
  I have created a release candidate (rc0) for
  hadoop-2.0.4.1-alpha that I would
  like to release.
  
  This is a stabilization release that includes fixed for a
  couple a of issues
  discovered in the testing with BigTop 0.6.0 release candidate.
  
  The RC is available at:
  http://people.apache.org/~cos/hadoop-2.0.4.1-alpha-rc0/
  The RC tag in svn is here:
  http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.4.1-alpha-rc0
  
  The maven artifacts are available via repository.apache.org.
  
  Please try the release bits and vote; the vote will run for
  the usual 7 days.
  
  Thanks for your voting
  Cos
  
  
  
  
  
  
  
  -- 
  Alejandro
 
 --
 Arun C. Murthy
 Hortonworks Inc.
 http://hortonworks.com/
 
 


Heads up: moving from 2.0.4.1-alpha to 2.0.5-alpha

2013-05-31 Thread Konstantin Boudnik
Guys,

I will be performing some changes wrt to moving 2.0.4.1 release candidate to
2.0.5 space. As outline below by Alejandro:

1. I will create new 2.0.5-alpha branch from the current head of 2.0.4-alpha
that contains 2.0.4.1 changes
2. consequently, set the artifacts version on the new branch to be 2.0.5-alpha
3. the CHANGES.txt will be updated accordingly on the new 2.0.5 branch
4. At this point I can cut an RC and put it out for re-vote. The staging can
be done after the next two steps.

I will be doing all these modifications in the next hour or so.

Tomorrow at 1 pm PDT I would like to:
1. update the version of the artifacts on branch-2 to become 2.1.0-SNAPSHOT
2. update the CHANGES.txt in the trunk and branch-2 to reflect new version names
3. at this point it should safe to do the staging for 2.0.5-alpha RC

To avoid any collisions during the last two steps - especially 2. - I would
ask everyone to hold off the modifications of the CHANGES.txt files on trunk
and branch-2 between 1 pm and 2 pm PDT.

Please let me know if you see any flaw above, questions.
  Cos

 As we change from 2.0.4.1 to 2.0.5 you'll need to do the following
 housekeeping as you work the new RC.
 
 * rename the svn branch
 * update the versions in the POMs
 * update the CHANGES.txt in trunk, branch-2 and the release branch
 * change the current 2.0.5 version in JIRA to 2.1.0, create a new 2.0.5
 version, change the fix version of the 2 JIRAs that make the RC

 I renamed 2.0.5-beta to 2.1.0-beta and 2.0.4.1-alpha to 2.0.5-alpha versions
 in jira for HADOOP, HDFS, YARN  MAPREDUCE.

 Please take care of the rest.

 Also, in branch-2, the version should be 2.1.0-SNAPSHOT.


Re: Heads up: moving from 2.0.4.1-alpha to 2.0.5-alpha

2013-05-31 Thread Konstantin Boudnik
Indeed! The second part is happening on Saturday, JUN01 2013 1PM-2PM PST. 

On Fri, May 31, 2013 at 01:01PM, Alejandro Abdelnur wrote:
 Cos, just to be clear, this is happening SAT JUN01 1PM-2PM PST, not now
 (FRI MAY31 1PM PST). Correct?
 
 Thx
 
 On Fri, May 31, 2013 at 12:45 PM, Konstantin Boudnik c...@apache.org wrote:
 
  Guys,
 
  I will be performing some changes wrt to moving 2.0.4.1 release candidate
  to
  2.0.5 space. As outline below by Alejandro:
 
  1. I will create new 2.0.5-alpha branch from the current head of
  2.0.4-alpha
  that contains 2.0.4.1 changes
  2. consequently, set the artifacts version on the new branch to be
  2.0.5-alpha
  3. the CHANGES.txt will be updated accordingly on the new 2.0.5 branch
  4. At this point I can cut an RC and put it out for re-vote. The staging
  can
  be done after the next two steps.
 
  I will be doing all these modifications in the next hour or so.
 
  Tomorrow at 1 pm PDT I would like to:
  1. update the version of the artifacts on branch-2 to become 2.1.0-SNAPSHOT
  2. update the CHANGES.txt in the trunk and branch-2 to reflect new version
  names
  3. at this point it should safe to do the staging for 2.0.5-alpha RC
 
  To avoid any collisions during the last two steps - especially 2. - I would
  ask everyone to hold off the modifications of the CHANGES.txt files on
  trunk
  and branch-2 between 1 pm and 2 pm PDT.
 
  Please let me know if you see any flaw above, questions.
Cos
 
   As we change from 2.0.4.1 to 2.0.5 you'll need to do the following
   housekeeping as you work the new RC.
  
   * rename the svn branch
   * update the versions in the POMs
   * update the CHANGES.txt in trunk, branch-2 and the release branch
   * change the current 2.0.5 version in JIRA to 2.1.0, create a new 2.0.5
   version, change the fix version of the 2 JIRAs that make the RC
 
   I renamed 2.0.5-beta to 2.1.0-beta and 2.0.4.1-alpha to 2.0.5-alpha
  versions
   in jira for HADOOP, HDFS, YARN  MAPREDUCE.
 
   Please take care of the rest.
 
   Also, in branch-2, the version should be 2.1.0-SNAPSHOT.
 
 
 
 
 -- 
 Alejandro


[VOTE] Release Apache Hadoop 2.0.5-alpha

2013-05-31 Thread Konstantin Boudnik
All,

I have created a release candidate (rc0) for hadoop-2.0.5-alpha that I would
like to release.

This is a stabilization release that includes fixed for a couple a of issues
discovered in the testing with BigTop 0.6.0 release candidate.

The RC is available at: http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc0/
The RC tag in svn is here: 
http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc0

The maven artifacts will be available via repository.apache.org on Sat, June
1st, 2013 at 2 pm PDT as outlined here
http://s.apache.org/WKD

Please try the release bits and vote; the vote will run for the 3 days,
because this is just a version name change. The bits are identical to the ones
voted on before in 
http://s.apache.org/2041move

Thanks for your voting
  Cos



Re: [VOTE] Release Apache Hadoop 2.0.5-alpha

2013-05-31 Thread Konstantin Boudnik
Alejandro,

thanks for looking into this. Indeed - I missed the 2.0.5-alpha section in
YARN CHANGES.txt. Added now. As for HDFS-4646: apparently I didn't get into
to branch-2.0.4-alpha back then, although I distinctively remember doing this.
Let me pull it into 2.0.5-alpha and update CHANGES.txt to reflect it. Also, I
will do JIRA in a moment.

Joep, appreciate the thorough examination. I have fixed the dates for the
releases 2.0.4-alpha. As for the top-level readme file - sorry I wasn't aware
about them. As for the binary: I am pretty sure we are only releasing source
code, but I will put binaries into the rc1 respin.

I will respin rc1 shortly. Appreciate the feedback!
  Cos

On Fri, May 31, 2013 at 05:27PM, Alejandro Abdelnur wrote:
 Verified MD5  signature, built, configured pseudo cluster, run a couple of
 sample jobs, tested HTTPFS.
 
 Still, something seems odd.
 
 The HDFS CHANGES.txt has the following entry under 2.0.5-alpha:
 
  HDFS-4646. createNNProxyWithClientProtocol ignores configured timeout
 value  (Jagane Sundar via cos)
 
 but I don't see that in the branch.
 
 And, the YARN CHANGES.txt does not have the 2.0.5-alpha section (it should
 be there empty).
 
 Cos, can you please look at these 2 things and explain/fix?
 
 Thanks.
 
 
 
 On Fri, May 31, 2013 at 4:04 PM, Konstantin Boudnik c...@apache.org wrote:
 
  All,
 
  I have created a release candidate (rc0) for hadoop-2.0.5-alpha that I
  would
  like to release.
 
  This is a stabilization release that includes fixed for a couple a of
  issues
  discovered in the testing with BigTop 0.6.0 release candidate.
 
  The RC is available at:
  http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc0/
  The RC tag in svn is here:
  http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc0
 
  The maven artifacts will be available via repository.apache.org on Sat,
  June
  1st, 2013 at 2 pm PDT as outlined here
  http://s.apache.org/WKD
 
  Please try the release bits and vote; the vote will run for the 3 days,
  because this is just a version name change. The bits are identical to the
  ones
  voted on before in
  http://s.apache.org/2041move
 
  Thanks for your voting
Cos
 
 
 
 
 -- 
 Alejandro


signature.asc
Description: Digital signature


Re: [VOTE] Release Apache Hadoop 2.0.5-alpha

2013-05-31 Thread Konstantin Boudnik
Ok, WRT HDFS-4646 - it is all legit and the code is in branch-2.0.4-alpha and
later. It has been committed as
  r1465124
The reason it isn't normally visible because of the weird commit message:

svn merge -c1465121 from trunk

So, we good. I am done with the CHANGES.txt fixed that you guys have noted
earlier and will be re-spinning RC1 in a few.

Cos

On Fri, May 31, 2013 at 08:07PM, Konstantin Boudnik wrote:
 Alejandro,
 
 thanks for looking into this. Indeed - I missed the 2.0.5-alpha section in
 YARN CHANGES.txt. Added now. As for HDFS-4646: apparently I didn't get into
 to branch-2.0.4-alpha back then, although I distinctively remember doing this.
 Let me pull it into 2.0.5-alpha and update CHANGES.txt to reflect it. Also, I
 will do JIRA in a moment.
 
 Joep, appreciate the thorough examination. I have fixed the dates for the
 releases 2.0.4-alpha. As for the top-level readme file - sorry I wasn't aware
 about them. As for the binary: I am pretty sure we are only releasing source
 code, but I will put binaries into the rc1 respin.
 
 I will respin rc1 shortly. Appreciate the feedback!
   Cos
 
 On Fri, May 31, 2013 at 05:27PM, Alejandro Abdelnur wrote:
  Verified MD5  signature, built, configured pseudo cluster, run a couple of
  sample jobs, tested HTTPFS.
  
  Still, something seems odd.
  
  The HDFS CHANGES.txt has the following entry under 2.0.5-alpha:
  
   HDFS-4646. createNNProxyWithClientProtocol ignores configured timeout
  value  (Jagane Sundar via cos)
  
  but I don't see that in the branch.
  
  And, the YARN CHANGES.txt does not have the 2.0.5-alpha section (it should
  be there empty).
  
  Cos, can you please look at these 2 things and explain/fix?
  
  Thanks.
  
  
  
  On Fri, May 31, 2013 at 4:04 PM, Konstantin Boudnik c...@apache.org wrote:
  
   All,
  
   I have created a release candidate (rc0) for hadoop-2.0.5-alpha that I
   would
   like to release.
  
   This is a stabilization release that includes fixed for a couple a of
   issues
   discovered in the testing with BigTop 0.6.0 release candidate.
  
   The RC is available at:
   http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc0/
   The RC tag in svn is here:
   http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc0
  
   The maven artifacts will be available via repository.apache.org on Sat,
   June
   1st, 2013 at 2 pm PDT as outlined here
   http://s.apache.org/WKD
  
   Please try the release bits and vote; the vote will run for the 3 days,
   because this is just a version name change. The bits are identical to the
   ones
   voted on before in
   http://s.apache.org/2041move
  
   Thanks for your voting
 Cos
  
  
  
  
  -- 
  Alejandro




signature.asc
Description: Digital signature


[VOTE] Release Apache Hadoop 2.0.5-alpha (rc1)

2013-05-31 Thread Konstantin Boudnik
All,

I have created a release candidate (rc1) for hadoop-2.0.5-alpha that I would
like to release.

This is a stabilization release that includes fixed for a couple a of issues
discovered in the testing with BigTop 0.6.0 release candidate.

The RC is available at: http://people.apache.org/~cos/hadoop-2.0.5-alpha-rc1/
The RC tag in svn is here: 
http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.5-alpha-rc1

The maven artifacts will be available via repository.apache.org on Sat, June
1st, 2013 at 2 pm PDT as outlined here
http://s.apache.org/WKD

Please try the release bits and vote; the vote will run for the 3 days,
because this is just a version name change. The bits are identical to the ones
voted on before in 
http://s.apache.org/2041move

Thanks for your voting
  Cos



Re: [VOTE] Release Apache Hadoop 2.0.4.1-alpha

2013-05-30 Thread Konstantin Boudnik
On Thu, May 30, 2013 at 10:57AM, Arun C Murthy wrote:
 I see you just re-opened MAPREUDCE-5211.
 
 Why not include MAPREDUCE-5211 as well rather than create one release per 
 patch?

Arun, it is unclear if MAPREDUCE-5211 has implications in 2.0.4 as per 
https://issues.apache.org/jira/browse/MAPREDUCE-5211?focusedCommentId=13670574page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13670574

Hence, there's a good chance that it never will be backported. And I don't
have any plans to created 'a release per patch'.

 Also, this is the first time we are seeing a four-numbered scheme in Hadoop.
 Why not call this 2.0.5-alpha?

There were precedents in four-numbered schemes before: 0.20.20[3-5].0 comes to
mind.

As for 2.0.5-alpha: The release numbering games and votes that had happened in
the last few weeks are very confusing. Some of them never been concluded, the
branches are moved and artifact versions seem to be colliding. 2.0.4.x seems
to work well for the stabilization purposes and it will allow to unblock
downstream and integration projects quickly.

Cos

 Arun
 
 On May 24, 2013, at 8:48 PM, Konstantin Boudnik wrote:
 
  All,
  
  I have created a release candidate (rc0) for hadoop-2.0.4.1-alpha that I 
  would
  like to release.
  
  This is a stabilization release that includes fixed for a couple a of issues
  discovered in the testing with BigTop 0.6.0 release candidate.
  
  The RC is available at: 
  http://people.apache.org/~cos/hadoop-2.0.4.1-alpha-rc0/
  The RC tag in svn is here: 
  http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.4.1-alpha-rc0
  
  The maven artifacts are available via repository.apache.org.
  
  Please try the release bits and vote; the vote will run for the usual 7 
  days.
  
  Thanks for your voting
   Cos
  
 
 


Re: [VOTE] Release Apache Hadoop 2.0.4.1-alpha

2013-05-30 Thread Konstantin Boudnik
There's no misunderstanding Chris - this release is to unblock downstream.

As for your question: I don't have a crystal ball; I wish though. I think the
answer depends on will be there more blocking bugs found in the later releases
of Bigtop or other downstream components.
This is bugfix release and, I guess, if there are more bugs found in the
future - more releases would have to be cut. Isn't this is why the software is
being released?

Now, the -1: I am not clear about the justification. What exactly we expect to
work out?

Cos

On Thu, May 30, 2013 at 01:48PM, Chris Douglas wrote:
 On Thu, May 30, 2013 at 10:57 AM, Arun C Murthy a...@hortonworks.com wrote:
  Why not include MAPREDUCE-4211 as well rather than create one release per 
  patch?
 
 From Cos's description, it sounded like these were backports of fixes
 to help Sqoop2 and fix some build issues. If it's not just to fixup
 leftover bugs in 2.0.4 *once* so downstream projects can integrate
 against 2.0.4.1, and this a release series, then I've completely
 misunderstood the purpose.
 
 Cos, are you planning 2.0.4.2?
 
  Also, this is the first time we are seeing a four-numbered scheme in 
  Hadoop. Why not call this 2.0.5-alpha?
 
 Good point. Since it contains only backports from branch-2, it would
 make sense for it to be an intermediate release.
 
 I shouldn't have to say this, but I'm changing my vote to -1 while we
 work this out. -C
 
  On May 24, 2013, at 8:48 PM, Konstantin Boudnik wrote:
 
  All,
 
  I have created a release candidate (rc0) for hadoop-2.0.4.1-alpha that I 
  would
  like to release.
 
  This is a stabilization release that includes fixed for a couple a of 
  issues
  discovered in the testing with BigTop 0.6.0 release candidate.
 
  The RC is available at: 
  http://people.apache.org/~cos/hadoop-2.0.4.1-alpha-rc0/
  The RC tag in svn is here: 
  http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.4.1-alpha-rc0
 
  The maven artifacts are available via repository.apache.org.
 
  Please try the release bits and vote; the vote will run for the usual 7 
  days.
 
  Thanks for your voting
   Cos
 
 
 


Re: Cannot update PoweredBy wiki page

2013-05-30 Thread Konstantin Boudnik
Someone with proper karma needs to add you into ACL for this page.

On Thu, May 30, 2013 at 11:14PM, Adam Kawa wrote:
 Hi,
 
 When uploading new content (and information about my company), I got the
 exception
 Sorry, can not save page because rubbelloselotto.de is not allowed in
 this wiki.
 
 How could I solve it?
 
 Kind regards,
 Adam


Re: [VOTE] Release Apache Hadoop 2.0.4.1-alpha

2013-05-30 Thread Konstantin Boudnik
On Thu, May 30, 2013 at 03:18PM, Chris Douglas wrote:
 On Thu, May 30, 2013 at 2:39 PM, Konstantin Boudnik c...@apache.org wrote:
  There's no misunderstanding Chris - this release is to unblock downstream.
 
  As for your question: I don't have a crystal ball; I wish though. I think 
  the
  answer depends on will be there more blocking bugs found in the later 
  releases
  of Bigtop or other downstream components.
  This is bugfix release and, I guess, if there are more bugs found in the
  future - more releases would have to be cut. Isn't this is why the software 
  is
  being released?
 
 Sure, but they're all backports from the release currently marked for
 2.0.5. Either (a) these are really blocker bugs and we should roll a
 patch release or (b) some bleeding-edge work needs to work around this
 while branch-2 is released in the next few weeks. If it's not severe
 enough to justify disrupting the versioning of snapshot maven
 artifacts in branch-2, then we're clearly not in case (a).
 
 I thought this was the result of extensive testing, and 2.0.4.1 was a
 release to enable additional integration before 2.0.5. If we plan to
 roll more releases as a subset of the bug fixes committed to branch-2
 then just call it 2.0.5. Please make sure it- and any future,
 intermediate release- is worth the disruption.

There's no plans to release anything else at this point - this is a bug-fix
release, as I pointed out on a numerous occasions. There's no new features -
just 2 fixes.

2.0.5 matter became and still is too controversial at some point. The vote
started by Arun to override the results of the Konstantin's vote never been
closed. The downstream projects are handing in the middle of the air because
of that confusion. 

  Now, the -1: I am not clear about the justification. What exactly we expect 
  to
  work out?
 
 It's become fashionable to close threads and count votes in the middle
 of the discussion. I changed my vote instead of trusting you. -C

Have I missed something or you just called me a cheater and a lair right to my 
face?

Cos

  On Thu, May 30, 2013 at 01:48PM, Chris Douglas wrote:
  On Thu, May 30, 2013 at 10:57 AM, Arun C Murthy a...@hortonworks.com 
  wrote:
   Why not include MAPREDUCE-4211 as well rather than create one release 
   per patch?
 
  From Cos's description, it sounded like these were backports of fixes
  to help Sqoop2 and fix some build issues. If it's not just to fixup
  leftover bugs in 2.0.4 *once* so downstream projects can integrate
  against 2.0.4.1, and this a release series, then I've completely
  misunderstood the purpose.
 
  Cos, are you planning 2.0.4.2?
 
   Also, this is the first time we are seeing a four-numbered scheme in 
   Hadoop. Why not call this 2.0.5-alpha?
 
  Good point. Since it contains only backports from branch-2, it would
  make sense for it to be an intermediate release.
 
  I shouldn't have to say this, but I'm changing my vote to -1 while we
  work this out. -C
 
   On May 24, 2013, at 8:48 PM, Konstantin Boudnik wrote:
  
   All,
  
   I have created a release candidate (rc0) for hadoop-2.0.4.1-alpha that 
   I would
   like to release.
  
   This is a stabilization release that includes fixed for a couple a of 
   issues
   discovered in the testing with BigTop 0.6.0 release candidate.
  
   The RC is available at: 
   http://people.apache.org/~cos/hadoop-2.0.4.1-alpha-rc0/
   The RC tag in svn is here: 
   http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.4.1-alpha-rc0
  
   The maven artifacts are available via repository.apache.org.
  
   Please try the release bits and vote; the vote will run for the usual 7 
   days.
  
   Thanks for your voting
Cos
  
  
  


Re: [VOTE] Release Apache Hadoop 2.0.4.1-alpha

2013-05-30 Thread Konstantin Boudnik
On Thu, May 30, 2013 at 03:11PM, Matt Foley wrote:
 Hi Cos,
 I would also request that you renumber the release candidate to just
 three-numbers, hence 2.0.5-alpha.
 
 Arun, are you willing to start the 2.1.x name-space for your next release,
 so that 2.0.x-alpha can become an intermediate stabilization branch as Cos
 and Konst want?

Let's get the facts straight, Matt, please: this want has been expressed in
the official vote here http://s.apache.org/ZMf Apparently, 2.0.5-alpha is
blocked now because of the another vote that hasn't been closed yet for
whatever reason. In order to unblock a number of releases in downstream
component I have moved forward with this release. Do you have any material
objections to the release that pursue this goal?

 I just think that using four-number schemes was symptomatic of the
 near-forking we had back in the 0.20.xxx.y days, and I really don't want to
 go back there.  Especially since you could say that 0.20.xxx.y is just
 three significant numbers, the leading zero being inconsequential.

I dare to remind that forth part of the version is reserved - not in a
parallel universe, of course - for patch level aka bug fixes. It hardly can
be taken a sign of 'forking' by any definition.

Cos

 So, would you please consider using 2.0.5-alpha?
 
 As for the 2.0.5-SNAPSHOT in the branch-2 versioning, that's standard
 usage.  Whoever makes the 2.0.5 release (or any next release) is expected
 to update the parent branch's SNAPSHOT default versioning, per
 HowToReleasePostMavenization#Branchinghttps://wiki.apache.org/hadoop/HowToReleasePostMavenization#Branching,
 step 6.
 
 Thanks,
 --Matt
 
 
 On Thu, May 30, 2013 at 11:52 AM, Konstantin Boudnik c...@apache.org wrote:
 
  On Thu, May 30, 2013 at 10:57AM, Arun C Murthy wrote:
   I see you just re-opened MAPREUDCE-5211.
  
   Why not include MAPREDUCE-5211 as well rather than create one release
  per patch?
 
  Arun, it is unclear if MAPREDUCE-5211 has implications in 2.0.4 as per
 
  https://issues.apache.org/jira/browse/MAPREDUCE-5211?focusedCommentId=13670574page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13670574
 
  Hence, there's a good chance that it never will be backported. And I don't
  have any plans to created 'a release per patch'.
 
   Also, this is the first time we are seeing a four-numbered scheme in
  Hadoop.
   Why not call this 2.0.5-alpha?
 
  There were precedents in four-numbered schemes before: 0.20.20[3-5].0
  comes to
  mind.
 
  As for 2.0.5-alpha: The release numbering games and votes that had
  happened in
  the last few weeks are very confusing. Some of them never been concluded,
  the
  branches are moved and artifact versions seem to be colliding. 2.0.4.x
  seems
  to work well for the stabilization purposes and it will allow to unblock
  downstream and integration projects quickly.
 
  Cos
 
   Arun
  
   On May 24, 2013, at 8:48 PM, Konstantin Boudnik wrote:
  
All,
   
I have created a release candidate (rc0) for hadoop-2.0.4.1-alpha that
  I would
like to release.
   
This is a stabilization release that includes fixed for a couple a of
  issues
discovered in the testing with BigTop 0.6.0 release candidate.
   
The RC is available at:
  http://people.apache.org/~cos/hadoop-2.0.4.1-alpha-rc0/
The RC tag in svn is here:
  http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.4.1-alpha-rc0
   
The maven artifacts are available via repository.apache.org.
   
Please try the release bits and vote; the vote will run for the usual
  7 days.
   
Thanks for your voting
 Cos
   
  
  
 


Re: [VOTE] Release Apache Hadoop 2.0.4.1-alpha

2013-05-30 Thread Konstantin Boudnik
On Thu, May 30, 2013 at 04:08PM, Jean-Daniel Cryans wrote:
 FWIW, not that I have a dog in this fight, but the only release with a
 4th number (not including .0 like the 0.20.20x releases did) we had
 was:
 
 http://hadoop.6.n7.nabble.com/VOTE-Release-0-17-2-1-rc-0-td13398.html
 
 0.17.2 was missing some native libs so 0.17.2.1 was released to fix
 that critical issue instead of calling it .3

Exactly the point - the _bigfix_ release. Thanks for pointing out the
similarities.

Cos

 
 J-D
 
 On Thu, May 30, 2013 at 3:52 PM, Konstantin Boudnik c...@apache.org wrote:
  On Thu, May 30, 2013 at 03:11PM, Matt Foley wrote:
  Hi Cos,
  I would also request that you renumber the release candidate to just
  three-numbers, hence 2.0.5-alpha.
 
  Arun, are you willing to start the 2.1.x name-space for your next release,
  so that 2.0.x-alpha can become an intermediate stabilization branch as Cos
  and Konst want?
 
  Let's get the facts straight, Matt, please: this want has been expressed 
  in
  the official vote here http://s.apache.org/ZMf Apparently, 2.0.5-alpha is
  blocked now because of the another vote that hasn't been closed yet for
  whatever reason. In order to unblock a number of releases in downstream
  component I have moved forward with this release. Do you have any material
  objections to the release that pursue this goal?
 
  I just think that using four-number schemes was symptomatic of the
  near-forking we had back in the 0.20.xxx.y days, and I really don't want to
  go back there.  Especially since you could say that 0.20.xxx.y is just
  three significant numbers, the leading zero being inconsequential.
 
  I dare to remind that forth part of the version is reserved - not in a
  parallel universe, of course - for patch level aka bug fixes. It hardly 
  can
  be taken a sign of 'forking' by any definition.
 
  Cos
 
  So, would you please consider using 2.0.5-alpha?
 
  As for the 2.0.5-SNAPSHOT in the branch-2 versioning, that's standard
  usage.  Whoever makes the 2.0.5 release (or any next release) is expected
  to update the parent branch's SNAPSHOT default versioning, per
  HowToReleasePostMavenization#Branchinghttps://wiki.apache.org/hadoop/HowToReleasePostMavenization#Branching,
  step 6.
 
  Thanks,
  --Matt
 
 
  On Thu, May 30, 2013 at 11:52 AM, Konstantin Boudnik c...@apache.org 
  wrote:
 
   On Thu, May 30, 2013 at 10:57AM, Arun C Murthy wrote:
I see you just re-opened MAPREUDCE-5211.
   
Why not include MAPREDUCE-5211 as well rather than create one release
   per patch?
  
   Arun, it is unclear if MAPREDUCE-5211 has implications in 2.0.4 as per
  
   https://issues.apache.org/jira/browse/MAPREDUCE-5211?focusedCommentId=13670574page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13670574
  
   Hence, there's a good chance that it never will be backported. And I 
   don't
   have any plans to created 'a release per patch'.
  
Also, this is the first time we are seeing a four-numbered scheme in
   Hadoop.
Why not call this 2.0.5-alpha?
  
   There were precedents in four-numbered schemes before: 0.20.20[3-5].0
   comes to
   mind.
  
   As for 2.0.5-alpha: The release numbering games and votes that had
   happened in
   the last few weeks are very confusing. Some of them never been concluded,
   the
   branches are moved and artifact versions seem to be colliding. 2.0.4.x
   seems
   to work well for the stabilization purposes and it will allow to unblock
   downstream and integration projects quickly.
  
   Cos
  
Arun
   
On May 24, 2013, at 8:48 PM, Konstantin Boudnik wrote:
   
 All,

 I have created a release candidate (rc0) for hadoop-2.0.4.1-alpha 
 that
   I would
 like to release.

 This is a stabilization release that includes fixed for a couple a of
   issues
 discovered in the testing with BigTop 0.6.0 release candidate.

 The RC is available at:
   http://people.apache.org/~cos/hadoop-2.0.4.1-alpha-rc0/
 The RC tag in svn is here:
   http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.4.1-alpha-rc0

 The maven artifacts are available via repository.apache.org.

 Please try the release bits and vote; the vote will run for the usual
   7 days.

 Thanks for your voting
  Cos

   
   
  


Re: [VOTE] Release Apache Hadoop 2.0.4.1-alpha

2013-05-30 Thread Konstantin Boudnik
On Thu, May 30, 2013 at 05:30PM, Chris Douglas wrote:
 On Thu, May 30, 2013 at 3:25 PM, Konstantin Boudnik c...@apache.org wrote:
  There's no plans to release anything else at this point - this is a bug-fix
  release, as I pointed out on a numerous occasions. There's no new features -
  just 2 fixes.
 
 If you're worried about extending the voting by a week, I don't think
 anyone can reasonably object to a truncated schedule if the only
 change is the version number. Downstream fixes discovered in Bigtop
 are a sufficient reason for a patch release and I think we'd all like
 them to become routine. Not just in 2.0.x, but in later release
 branches.

I have no issues of changing the version to 2.0.5-alpha and restarting to vote
for the release content, e.g. 2 bug fixes. Shall I call 3 days re-vote because
of the number change?

  2.0.5 matter became and still is too controversial at some point. The vote
  started by Arun to override the results of the Konstantin's vote never been
  closed.
 
 For the nth time, neither release plan vote was binding. We recently
 amended the bylaws to eliminate this confusion. There is no ambiguity
 between votes when neither is in scope.

Does the result of bylaw vote nullifies the unfinished vote started by Arun?
Sorry, I am dense, apparently.

  The downstream projects are handing in the middle of the air because
  of that confusion.
 
 Can we please ground our discussion when discussing compatibility and
 bugs? Breathless alarm is not proportional to the severity, here.

Good point. Can we limit the vote thread to the merits of the release then?

  Have I missed something or you just called me a cheater and a lair right to 
  my face?
 
 I called you neither. The prenominate votes were closed, counted, and
 declared to be binding over objections. I'm annoyed that I have to
 toggle my vote to prevent that tactic, but based on recent experience
 I don't expect you to forgo it. I'd be happy to learn my caution is
 unnecessary. -C

That sound like adding an insult to injury, if my forth-language skills do not
mislead me.

Cos

   On Thu, May 30, 2013 at 01:48PM, Chris Douglas wrote:
   On Thu, May 30, 2013 at 10:57 AM, Arun C Murthy a...@hortonworks.com 
   wrote:
Why not include MAPREDUCE-4211 as well rather than create one release 
per patch?
  
   From Cos's description, it sounded like these were backports of fixes
   to help Sqoop2 and fix some build issues. If it's not just to fixup
   leftover bugs in 2.0.4 *once* so downstream projects can integrate
   against 2.0.4.1, and this a release series, then I've completely
   misunderstood the purpose.
  
   Cos, are you planning 2.0.4.2?
  
Also, this is the first time we are seeing a four-numbered scheme in 
Hadoop. Why not call this 2.0.5-alpha?
  
   Good point. Since it contains only backports from branch-2, it would
   make sense for it to be an intermediate release.
  
   I shouldn't have to say this, but I'm changing my vote to -1 while we
   work this out. -C
  
On May 24, 2013, at 8:48 PM, Konstantin Boudnik wrote:
   
All,
   
I have created a release candidate (rc0) for hadoop-2.0.4.1-alpha 
that I would
like to release.
   
This is a stabilization release that includes fixed for a couple a 
of issues
discovered in the testing with BigTop 0.6.0 release candidate.
   
The RC is available at: 
http://people.apache.org/~cos/hadoop-2.0.4.1-alpha-rc0/
The RC tag in svn is here: 
http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.4.1-alpha-rc0
   
The maven artifacts are available via repository.apache.org.
   
Please try the release bits and vote; the vote will run for the 
usual 7 days.
   
Thanks for your voting
 Cos
   
   
   


Re: [VOTE] Release Apache Hadoop 2.0.4.1-alpha

2013-05-30 Thread Konstantin Boudnik
Thanks Alejandro,

that's what my plan for the morning. Thanks for putting together the
check-list - would be easier for me not to miss anything. I am aiming to have
the bits out by noon or so. Appreciate the help!

Cos

On Thu, May 30, 2013 at 08:41PM, Alejandro Abdelnur wrote:
 Konstantin, Cos,
 
 As we change from 2.0.4.1 to 2.0.5 you'll need to do the following
 housekeeping as you work the new RC.
 
 * rename the svn branch
 * update the versions in the POMs
 * update the CHANGES.txt in trunk, branch-2 and the release branch
 * change the current 2.0.5 version in JIRA to 2.1.0, create a new 2.0.5
 version, change the fix version of the 2 JIRAs that make the RC
 
 Thanks.
 
 
 On Thu, May 30, 2013 at 6:18 PM, Chris Douglas cdoug...@apache.org wrote:
 
  On Thu, May 30, 2013 at 5:51 PM, Konstantin Boudnik c...@apache.org
  wrote:
   I have no issues of changing the version to 2.0.5-alpha and restarting
  to vote
   for the release content, e.g. 2 bug fixes. Shall I call 3 days re-vote
  because
   of the number change?
 
  +1 Sounds great.
 
   Does the result of bylaw vote nullifies the unfinished vote started by
  Arun?
   Sorry, I am dense, apparently.
 
  Yes, nobody should feel bound by either vote. The bylaw change
  clarifies that release plans are for RMs to solicit feedback and gauge
  PMC support for an artifact, not pre-approvals for doing work.
 
   Can we limit the vote thread to the merits of the release then?
 
  Happily.
 
   That sound like adding an insult to injury, if my forth-language skills
  do not
   mislead me.
 
  They do mislead you, or I've expressed the point imprecisely. We can
  take this offline. -C
 
 On Thu, May 30, 2013 at 01:48PM, Chris Douglas wrote:
 On Thu, May 30, 2013 at 10:57 AM, Arun C Murthy 
  a...@hortonworks.com wrote:
  Why not include MAPREDUCE-4211 as well rather than create one
  release per patch?

 From Cos's description, it sounded like these were backports of
  fixes
 to help Sqoop2 and fix some build issues. If it's not just to
  fixup
 leftover bugs in 2.0.4 *once* so downstream projects can integrate
 against 2.0.4.1, and this a release series, then I've completely
 misunderstood the purpose.

 Cos, are you planning 2.0.4.2?

  Also, this is the first time we are seeing a four-numbered
  scheme in Hadoop. Why not call this 2.0.5-alpha?

 Good point. Since it contains only backports from branch-2, it
  would
 make sense for it to be an intermediate release.

 I shouldn't have to say this, but I'm changing my vote to -1
  while we
 work this out. -C

  On May 24, 2013, at 8:48 PM, Konstantin Boudnik wrote:
 
  All,
 
  I have created a release candidate (rc0) for
  hadoop-2.0.4.1-alpha that I would
  like to release.
 
  This is a stabilization release that includes fixed for a
  couple a of issues
  discovered in the testing with BigTop 0.6.0 release candidate.
 
  The RC is available at:
  http://people.apache.org/~cos/hadoop-2.0.4.1-alpha-rc0/
  The RC tag in svn is here:
  http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.4.1-alpha-rc0
 
  The maven artifacts are available via repository.apache.org.
 
  Please try the release bits and vote; the vote will run for
  the usual 7 days.
 
  Thanks for your voting
   Cos
 
 
 
 
 
 
 
 -- 
 Alejandro


signature.asc
Description: Digital signature


Re: Bugfix release 2.0.4.1

2013-05-24 Thread Konstantin Boudnik
Thanks guys! I will start the release vote shortly.

Cos

On Thu, May 23, 2013 at 06:23PM, Anatoli Fomenko wrote:
 From Bigtop perspective, Hadoop══2.0.4.1 is unblocked with regards to a
 Sqoop 2 blocker, and the voting can be started.
 
 Anatoli
 
 -- Forwarded message --
 From: Konstantin Boudnik c...@apache.org
 Date: Mon, May 20, 2013 at 9:03 PM
 Subject: Re: Bugfix release 2.0.4.1
 To:═common-dev@hadoop.apache.org
 
 
 The build is over, all unit tests have passed.
 Roman, the 2.0.4.1 tarballs are available from
 https://builds.apache.org/job/ Hadoop-branch-2.0.4/47/
 
 For Bigtop to use as the base.
 ═ Cos
 
 On Mon, May 20, 2013 at 12:15PM, Konstantin Boudnik wrote:
  I have ran the full set of unit tests on the patched version and I see 3
  failures in HDFS HA stress tests, which are clearly not related to fix in
  question. I will commit it shortly.
 
  Cos
 
  On Thu, May 16, 2013 at 12:03AM, Roman Shaposhnik wrote:
   On Wed, May 15, 2013 at 11:09 PM, Vinod Kumar Vavilapalli
   vino...@hortonworks.com wrote:
   
It's a little dirty, but Mark and Jarek (presumably from BigTop) ran a 
patched version with my change at MAPREDUCE-5240.
  
   I've just updated the patch to apply cleanly on branch-2.0.4 (had to 
   refactor
   quite a bit of unit tests). Hopefully Cos can apply it═tomorrow═and we
   can start testing Bigtop.
  
Otherwise we may end up creating lots of bug-fix releases.
  
   Agreed. The point of 2.0.4.1 is to unblock Bigtop 0.6.0 release. We're
   trying to come with stable base line for Hadoop 2.0.x where
   all the downstream components will be running smoothly.
  
   At this point a MAPREDUCE-5240 is the only thing holding us
   up, but once we start our own round of testing other issues
   could pop up. We shall see how it goes.
  
   Thanks,
   Roman.══


[VOTE] Release Apache Hadoop 2.0.4.1-alpha

2013-05-24 Thread Konstantin Boudnik
All,

I have created a release candidate (rc0) for hadoop-2.0.4.1-alpha that I would
like to release.

This is a stabilization release that includes fixed for a couple a of issues
discovered in the testing with BigTop 0.6.0 release candidate.

The RC is available at: http://people.apache.org/~cos/hadoop-2.0.4.1-alpha-rc0/
The RC tag in svn is here: 
http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.4.1-alpha-rc0

The maven artifacts are available via repository.apache.org.

Please try the release bits and vote; the vote will run for the usual 7 days.

Thanks for your voting
  Cos



signature.asc
Description: Digital signature


Re: Bugfix release 2.0.4.1

2013-05-20 Thread Konstantin Boudnik
The build is over, all unit tests have passed.
Roman, the 2.0.4.1 tarballs are available from
  https://builds.apache.org/job/Hadoop-branch-2.0.4/47/

For Bigtop to use as the base.
  Cos

On Mon, May 20, 2013 at 12:15PM, Konstantin Boudnik wrote:
 I have ran the full set of unit tests on the patched version and I see 3
 failures in HDFS HA stress tests, which are clearly not related to fix in
 question. I will commit it shortly.
 
 Cos
 
 On Thu, May 16, 2013 at 12:03AM, Roman Shaposhnik wrote:
  On Wed, May 15, 2013 at 11:09 PM, Vinod Kumar Vavilapalli
  vino...@hortonworks.com wrote:
  
   It's a little dirty, but Mark and Jarek (presumably from BigTop) ran a 
   patched version with my change at MAPREDUCE-5240.
  
  I've just updated the patch to apply cleanly on branch-2.0.4 (had to 
  refactor
  quite a bit of unit tests). Hopefully Cos can apply it tomorrow and we
  can start testing Bigtop.
  
   Otherwise we may end up creating lots of bug-fix releases.
  
  Agreed. The point of 2.0.4.1 is to unblock Bigtop 0.6.0 release. We're
  trying to come with stable base line for Hadoop 2.0.x where
  all the downstream components will be running smoothly.
  
  At this point a MAPREDUCE-5240 is the only thing holding us
  up, but once we start our own round of testing other issues
  could pop up. We shall see how it goes.
  
  Thanks,
  Roman.


signature.asc
Description: Digital signature


Re: [VOTE] - Release 2.0.5-beta

2013-05-16 Thread Konstantin Boudnik
Guys,

I guess what you're missing is that Bigtop isn't a testing framework for
Hadoop. It is stack framework that verifies that components are dealing with
each other nicely. Every single stack is different: Bigtop 0.5.0 differs from
0.6.0, and so on. Bigtop - as any other ASF project - has its releases that
might or might not be aligned with particular version of Hadoop. Hence, an
ethalon stack needs to be defined first and foremost.

Before we even start talking about running it nightly (another question is on
what hardware, let's not get there for now) let's understand who will can help
with triage'ing test failures? Downstreams, Hadoop or Bigtop?

Judging by a number of other emails there's a number of people on this list
who care plenty about integration issues. Any volunteers to help with
integration testing in the open?

 Is this a previously solved problem?

Yes. The problem is solved by separating actively developed (aka unstable)
release from more mature and less volatile ones. This is what has been
concluded upon two days ago in this voting thread http://s.apache.org/MnU

Cos

On Wed, May 15, 2013 at 04:54PM, Matt Foley wrote:
 Roman, what is your model for how test results from Bigtop should feed back
 into Hadoop-2 development?
 With the understanding that (a) software does have bugs, and (b) you're not
 going to get an SLA on community-sponsored software,
 what are your ideas for how to close the loop better?
 
 Would CI runs of Bigtop against branch-2 be feasible, as Arun suggests?
 How should we accomodate changes in individual components (Hadoop Core, but
 others as well) that may require changes in one or more other components?
 How does Bigtop keep doing a viable nightly build in that chaotic
 environment?
 Is this a previously solved problem?
 
 Thanks,
 --Matt
 
 
 On Wed, May 15, 2013 at 4:47 PM, Arun C Murthy a...@hortonworks.com wrote:
 
 
  On May 15, 2013, at 3:50 PM, Roman Shaposhnik wrote:
 
   Arun,
  
   am I reading yours answer to my binary question correctly? It is a 'no'.
 
  No.
 
  
   My reading of your response is that while you appreciate the feedback
   Bigtop is providing you're not of an opinion that investigating the level
   of stability of Hadoop wrt. downstream any further than what is currently
   happening would be a worthy investment of Hadoop's community
   (or your personal for that matter) time?
 
  Everyone is welcome to contribute in any and all manner. I can't speak for
  everyone.
  It would be useful if Bigtop could run regressions on releases here
  consistently. We've also talked in the past about running Bigtop on
  branch-2, nightly. Is that something you could help with? You'd earn my
  personal gratitude.
 
  thanks,
  Arun
 
  
   Thanks,
   Roman.
  
   On Wed, May 15, 2013 at 3:02 PM, Arun C Murthy a...@hortonworks.com
  wrote:
   Roman,
  
   Furthermore, before we rush into finding flaws and scaring kids at
  night it would be useful to remember one thing:
   Software has *bugs*. We can't block any release till the entire
  universe validates it, in fact they won't validate it if we don't release
  since are at the bottom of the stack.
  
   Any help prior to the release is welcome; I know people who work for
  the same employer as I do have plans to do further testing after we freeze
  apis via the beta release(s). I hope and pray others can join this effort -
  thanks to everyone who already has.
  
   Again, freezing APIs and protocols is the primary aim of 2.0.5-beta.
  There are no guarantees it's 100% bug-free, we can never make such
  guarantees anyway.
  
   If, and when, we find bugs with 2.0.5-beta I'm more than happy to
  quickly turn around and make more releases (2.0.6-beta, 2.0.7-beta).
  Obviously I'll make a call on which bugs are critical - feedback to help me
  decide is, as always, welcome.
   I've been clear, many times, that we might need more than one beta
  release to iron out bugs etc.
  
   None of this should be a surprise - this has happened many, many times
  in the lifetime of this and other projects. 2.0.3-alpha vis-a-vis
  2.0.4-alpha is the most recent example - it won't be the last.
  
   So, I hope, concludes this meme.
  
   thanks,
   Arun
  
   On May 15, 2013, at 2:20 PM, Arun C Murthy wrote:
  
   Great summary, thanks Vinod.
  
   On May 15, 2013, at 2:14 PM, Vinod Kumar Vavilapalli wrote:
  
  
   Roman, I keep this same argument again and again. Should've refuted
  earlier.
  
   Please list down all the issues that BigTop ran into *because of* new
  features. You continue to argue that new features are destabilizing 2.0.*,
  which I don't agree with at all. 2.0.3-alpha was the last time major
  features got merged in, and we found blockers irrespective of those.
  
   MAPREDUCE-5240 specifically isn't due to any feature merge. This was
  a bug. I'd say this is a long standing bug in 2.0.x. You sure this passed
  in 2.0.3? Even so, this is mostly broken by another bug-fix and *not*
  because of 

Re: [VOTE] - Release 2.0.5-beta

2013-05-16 Thread Konstantin Boudnik
On Wed, May 15, 2013 at 10:52PM, Suresh Srinivas wrote:
  Assuming that you are talking about HDFS features when you say
 
   features going into a beta on a very short short timetable and
   laundry list etc,
  
 
  No, that would not be a correct assumption.
 
  So these features are not something that are impulsively developed
   and irresponsibly pushed to a release. They have gone through
   considerable testing and have been developed over a long time.
  
 
  There is no need to reframe my comment in combative terms and read in
  insults that are not there.
 
 
 No insult taken. But I want to make a case that feature are not proposed
 lightly and due diligence both during development and testing are done.
 
 
  As I read Arun's mail the plan is to integrate several feature branches
  into branch-2. That would of course result in brand new never before tested
  code. I do not believe that should have the label alpha. This is just my
  personal opinion. Shit happens when commits happen. - You know this as
  well as I. That does not mean I am here to attack or insult you by pointing
  that out and suggesting more measured alternatives.
 
  There is little to gain in engaging in debate club. If you are not
  interested in hearing these opinions, that is fine, I have received that
  message already, nothing further need be said.
 
 
 Andy, I value your feedback. I am only trying to allay the concerns by
 sharing my perspective.

What I am seeing times and again in these endless discussion threads is this:

  a) downstream or bigtop: we are seeing a bunch of integration issues with
every new feature introduced/something even a commit made
  b) feature developers: no-no, these features are developed for a long time,
tests are ran, no need to be concerned

The same pattern is repeated times and again. The only conclusion that I can
make out of it, is that either the meaning of integration testing is
different for a) and b) or that a) and b) are using very different validation
mechanisms.

Which one is that? I am puzzled.

Bugs are quite expected - Andrew put it very eloquently, actually. But you
only can deal with them effectively if the flow of changes is controlled, e.g.
via smaller and focused releases. The development process has to be
converging, and not fanning-out. Case in point? Sure. 2.0.3-alpha had to be
followed by 2.0.4-alpha release (officially called bugfix release); it - in
turn - requires 2.0.4.1-alpha to make it suitable for other downstream
components. So, it took 2 releases to simply fix issues caused by a bunch of
bugfixes and no major new features being committed into 2.0.3-alpha. These are
just cold facts - not attacking any ones' ego here.

  Cos



signature.asc
Description: Digital signature


Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Konstantin Boudnik
Indeed. I think the root of the issue is deeper. ASF software practices are
great to deal with isolated, relatively contained projects like httpd,
libreoffice, trac, etc. However, Hadoop based stack - essentially, software
aimed at enterprises with bigger scale operations - is a different animal,
that requires balancing of a huge number of moving parts and an unbroken
flow of feedback up the stream. Anyone who have delivered any enterprise
grade software system knows perfectly well how hard is that.

However, in the environment where a release pushed out in the rush
(essentially causing DOA issues downstream), these are got fixed in consequent
releases.  That ironically is likely to contain some other DOAs because an
integration testing - and I mean real world integration system testing - is
done by this small project, that is treated like a toy for adolescent kids.
And there's no other real integration testing happening OPENLY on the full
stack. Despite numerous claims, that is.

Software comes with bugs - this is a somewhat expected phenomena. However, bug
fixes shouldn't be mixed with new features, increasing entropy in the system.
In other words, the development process should fan-in. A process with multiple
consequent stable releases helps to achieve it; and compatibility issues would
be addressed by working on the next major release.

The model above leaves downstream with a choice of sticking to the 3.x or 
switching
to 4.x and so on. Where's having permanent alpha tag is a convenient way to 
control
software project that effectively became a vendor-controlled effort.

And yes - this leads to fragmentation, makes no mistakes about it. Because no
one can sit on the hands for a year and wait until a usable release with all
great features will come about: lot of organizations just silently forking away
to make their own environment suitable for production or sale; some of them
might sporadically contribute something back. And of course - this is not the
aim of Apache project to produce commercial grade platform.

Cos

On Wed, May 15, 2013 at 02:54PM, Roman Shaposhnik wrote:
 On Wed, May 15, 2013 at 2:14 PM, Vinod Kumar Vavilapalli
 vino...@hortonworks.com wrote:
  Please list down all the issues that BigTop ran into *because of* new 
  features.
 
 Whether the bug is *because of* new feature or not is a red herring
 for my argument. Please lets drop this distinction. I never used it.
 
  You continue to argue that new features are destabilizing 2.0.*,
  which I don't agree with at all. 2.0.3-alpha was the last time major
  features got merged in, and we found blockers irrespective of those.
 
 This is not my argument at all. I apologize if somehow I failed to
 communicate it, but here's what my argument boils down to:
 given *my* experience with Hadoop 2.0.x series and Bigtop
 release every time I try a different release of Hadoop 2.0.x
 I run into issues that scare me. They scare me because
 they are so basic yet they make component like Sqoop
 and Oozie (and I believe Giraph on one occasion)
 pretty much DOA for YARN-base mapreduce implementation.
 
 In my mind, what that translates into is the fact that nobody
 did *any* real testing of a particular downstream component
 running on a given Hadoop 2.0.x release. Like I said --
 the issues so far make the components in question DOA.

 Effectively the onion of issues remain unpeeled, so to speak.
 
 What I'm asking on this thread (and somehow nobody is willing
 to give me a straight answer) is whether the Hadoop community
 is willing to invest in peeling this onion of issues somewhat more
 before declaring Hadoop 2.0.5 a beta release.
 
 Once again it is a binary question -- please give me an answer
 of yes or no.
 
  I am not arguing that new features *may* destabilize the branch, but you've 
  repeatedly stated this as if that were a fact.
 
 Your list of issues is pretty complete (give or take a few that I didn't file
 but Cos and others did). And I'd be the first one to agree that
 it is not a large list of issues. What scares me is not its size,
 but the fact how basic they are and how the block the *rest*
 of the testing completely.
 
 To be extra clear -- what scares me about something like
 MAPREDUCE-5240 is not whether it came as a result of
 a merge or was sitting there since day one. What scares
 me is that we've identified it last week and yet Sqoop 2 is
 DOA in its presense.
 
 How many more issues like that one (regardless of how
 they originated) are in branch-2? Wouldn't we want to
 know before declaring Hadoop 2.0.5 beta?
 
 Now, knowing would require work -- that's what my
 argument is all about.
 
 
 Thanks,
 Roman.


Bugfix release 2.0.4.1

2013-05-14 Thread Konstantin Boudnik
I'd like to spin-off a bugfix release for 2.0.4-alpha containing fix for
  MAPREDUCE-5240
Vinod has the fix on branch-2 already, so it seems like a no-brainer backport.

However, the issue is blocking Bigtop 0.6.0 from being released on top of
2.0.4-alpha. I am planning to backport the fix into 2.0.4-alpha branch and run
the tests, etc. Please let me know if there are any other critical fixes that
needs to be released along.

If someone with proper JIRA credentials can open 2.0.4.1-alpha version it
would be greatly appreciated!
  
-- 
Thank you,
  Cos



[jira] [Created] (HADOOP-9528) org.apache.hadoop.ha.TestZKFailoverController.testCedeActive is failing intermittently

2013-04-30 Thread Konstantin Boudnik (JIRA)
Konstantin Boudnik created HADOOP-9528:
--

 Summary: 
org.apache.hadoop.ha.TestZKFailoverController.testCedeActive is failing 
intermittently 
 Key: HADOOP-9528
 URL: https://issues.apache.org/jira/browse/HADOOP-9528
 Project: Hadoop Common
  Issue Type: Bug
  Components: ha
Affects Versions: 2.0.4-alpha
Reporter: Konstantin Boudnik


Unit tests is failing with a message like 
{{Should take ~3 seconds to rejoin. Only took 2276ms before rejoining.}}

A looking into the test shows the only 2.8 seconds and more is considered to be 
{{about 3 seconds}}. Say 2.7 seconds won't be considered {{about 3 seconds}} 
contrary to elementary school arithmetic.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


Re: [DISCUSS] Getting to Hadoop 2.X beta

2013-04-24 Thread Konstantin Boudnik
To add to it, here's what have been experiences as a disconnect during the
release cycle of 2.0.4-alpha using BigTop 0.6.6 as the integration platform and
management stack of choice:

1. Need for improved feedback from the downstream projects
2. Need for improved feedback from the DevOps

Which boils down to getting sufficient buy-in from downstreams and devops
communities that they are now willing to consider branch-2 as being landing
spot for their Hadoop 2.X needs.

I belive the recent effort around 2.0.4-alpha and BigTop 0.6.0 might be
sufficient to convince both communities to:
 1. migrate their Hadoop 2.X profiles to at least 2.0.4-alpha
 2. start to run integration tests against Hadoop 2.X profile
 3. start to publish Maven artifacts
 4. in general agree to treat Hadoop 2.X issues with at least
the same priority as the issues in default profiles are treated (which
are mostly Hadoop 1.X profiles).

that would be a good start. BigTop can certainly help with a lot of
techicalities and moving parts of the process. That would require tighter
coordination between releases, of course.

The question I don't have an answer for is how to engage DevOps community.

Thoughts?
  Cos

On Wed, Apr 24, 2013 at 03:38PM, Roman Shaposhnik wrote:
 On Wed, Apr 24, 2013 at 9:42 AM, Steve Loughran ste...@hortonworks.com 
 wrote:
  There's quite a few preconditions to be met for a piece of
  software to reach beta status. Quite a few of them are now
  being discussed in a neighboring thread 'Compatibility in
  Apache Hadoop' ( http://s.apache.org/VE1 ) but I'd like to
  kick off a broader discussion: what do you think a *complete*
  list should look like, before we can honestly signal this
  change to the rest of the world.
 
  It would be very nice if the things that are offered as part
  of the criteria are easily quantifiable, but lets brainstorm
  first anyway. We can always stack-rank and quantify later.
 
  I need to get YARN-117 in before the service lifecycle is frozen forever
 
 We certainly need to get a list of blocker JIRAs for what would be
 our first beta release. This actually, raises a practical, question:
 what do we tag those?
 
 Or to put it another way: can we all agree that 2.0.5 should be our
 first beta release?
 
 Thanks,
 Roman.


signature.asc
Description: Digital signature


Re: [VOTE] Release Apache Hadoop 2.0.4-alpha

2013-04-18 Thread Konstantin Boudnik
-0

the release is missing HADOOP-9704 that has critical effect on downstream
projects e.g. build are affected. The issue has been raised for the first time
back in 4/10/13 http://is.gd/OGb3GG and never been even sneezed upon.

Cos

On Sat, Apr 13, 2013 at 03:26AM, Arun C Murthy wrote:
 Folks,
 
 I've created a release candidate (RC2) for hadoop-2.0.4-alpha that I would 
 like to release.
 
 The RC is available at: 
 http://people.apache.org/~acmurthy/hadoop-2.0.4-alpha-rc2/
 The RC tag in svn is here: 
 http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.0.4-alpha-rc2
 
 The maven artifacts are available via repository.apache.org.
 
 Please try the release and vote; the vote will run for the usual 7 days.
 
 thanks,
 Arun
 
 
 --
 Arun C. Murthy
 Hortonworks Inc.
 http://hortonworks.com/
 
 


signature.asc
Description: Digital signature


[jira] [Reopened] (HADOOP-9407) commons-daemon 1.0.3 dependency has bad group id causing build issues

2013-04-17 Thread Konstantin Boudnik (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-9407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik reopened HADOOP-9407:



I think it needs to be added to branch-2.0.4-alpha as well before the release 
goes out.

 commons-daemon 1.0.3 dependency has bad group id causing build issues
 -

 Key: HADOOP-9407
 URL: https://issues.apache.org/jira/browse/HADOOP-9407
 Project: Hadoop Common
  Issue Type: Bug
  Components: build
Affects Versions: 2.0.3-alpha
Reporter: Sangjin Lee
Assignee: Sangjin Lee
 Fix For: 2.0.5-beta

 Attachments: HDFS-4497.patch


 The commons-daemon dependency of the hadoop-hdfs module has been at version 
 1.0.3 for a while. However, 1.0.3 has a pretty well-known groupId error in 
 its pom (org.apache.commons as opposed to commons-daemon). This problem 
 has since been corrected on commons-daemon starting 1.0.4.
 This causes build problems for many who depend on hadoop-hdfs directly and 
 indirectly, however. Maven can skip over this metadata inconsistency. But 
 other less forgiving build systems such as ivy and gradle have much harder 
 time working around this problem. For example, in gradle, pretty much the 
 only obvious way to work around this is to override this dependency version.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Reopened] (HADOOP-9405) TestGridmixSummary#testExecutionSummarizer is broken

2013-04-01 Thread Konstantin Boudnik (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik reopened HADOOP-9405:



Need to backport to branch-2.0.4

 TestGridmixSummary#testExecutionSummarizer is broken
 

 Key: HADOOP-9405
 URL: https://issues.apache.org/jira/browse/HADOOP-9405
 Project: Hadoop Common
  Issue Type: Bug
  Components: test, tools
Affects Versions: 3.0.0
Reporter: Andrew Wang
Assignee: Andrew Wang
Priority: Minor
 Fix For: 3.0.0, 2.0.5-beta

 Attachments: hdfs-4599-1.patch


 HADOOP-9252 changed how human readable numbers are printed, and required 
 updating a number of test cases. This one was missed because the Jenkins 
 precommit job apparently isn't running the tests in hadoop-tools.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Resolved] (HADOOP-9405) TestGridmixSummary#testExecutionSummarizer is broken

2013-04-01 Thread Konstantin Boudnik (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-9405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik resolved HADOOP-9405.


Resolution: Fixed

I have committed the patch to 2.0.4-alpha branch

 TestGridmixSummary#testExecutionSummarizer is broken
 

 Key: HADOOP-9405
 URL: https://issues.apache.org/jira/browse/HADOOP-9405
 Project: Hadoop Common
  Issue Type: Bug
  Components: test, tools
Affects Versions: 3.0.0, 2.0.4-alpha
Reporter: Andrew Wang
Assignee: Andrew Wang
Priority: Minor
 Fix For: 3.0.0, 2.0.5-beta, 2.0.4-alpha

 Attachments: hdfs-4599-1.patch


 HADOOP-9252 changed how human readable numbers are printed, and required 
 updating a number of test cases. This one was missed because the Jenkins 
 precommit job apparently isn't running the tests in hadoop-tools.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


Re: [Vote] Merge branch-trunk-win to trunk

2013-03-25 Thread Konstantin Boudnik
It doesn't look like any progress has been done on the ticket below in the
last 3 weeks. And now branch-2 can't be compiled because of 

hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java:[895,15]
WINDOWS is not public in org.apache.hadoop.fs.Path; cannot be accessed from
outside package

That's exactly why I was -1'ing this...
  Cos

On Mon, Mar 04, 2013 at 05:41PM, Matt Foley wrote:
 Thanks, gentlemen.  I've opened and taken responsibility for
 https://issues.apache.org/jira/browse/HADOOP-9359.  Giri Kesavan has agreed
 to help with the parts that require Jenkins admin access.
 
 Thanks,
 --Matt
 
 
 
 On Mon, Mar 4, 2013 at 5:00 PM, Konstantin Shvachko 
 shv.had...@gmail.comwrote:
 
  +1 on the merge.
 
  I am glad we agreed.
  Having Jira to track the CI effort is a good idea.
 
  Thanks,
  --Konstantin
 
  On Mon, Mar 4, 2013 at 3:29 PM, Matt Foley mfo...@hortonworks.com wrote:
   Thanks.  I agree Windows -1's in test-patch should not block commits.
  
   --Matt
  
  
  
   On Mon, Mar 4, 2013 at 2:30 PM, Konstantin Shvachko 
  shv.had...@gmail.com
   wrote:
  
   On Mon, Mar 4, 2013 at 12:22 PM, Matt Foley mfo...@hortonworks.com
   wrote:
Konstantine, you have voted -1, and stated some requirements before
you'll
withdraw that -1.  As I plan to do work to fulfill those
  requirements, I
want to make sure that what I'm proposing will, in fact, satisfy you.
That's why I'm asking, if we implement full test-patch integration
  for
Windows, does it seem to you that that would provide adequate support?
  
   Yes.
  
I have learned not to presume that my interpretation is correct.  My
interpretation of item #1 is that test-patch provides pre-commit
  build,
so
it would satisfy item #1.  But rather than assuming that I am
interpreting
it correctly, I simply want your agreement that it would, or if not,
clarification why it won't.
  
   I agree it will satisfy my item #1.
   I did not agree in my previous email, but I changed my mind based on
   the latest discussion. I have to explain why now.
   I was proposing nightly build because I did not want pre-commit build
   for Windows block commits to Linux. But if people are fine just ignoring
   -1s for the Windows part of the build it should be good.
  
Regarding item #2, it is also my interpretation that test-patch
  provides
an
on-demand (perhaps 20-minutes deferred) Jenkins build and unit test,
with
logs available to the developer, so it would satisfy item #2.  But
rather
than assuming that I am interpreting it correctly, I simply want your
agreement that it would, or if not, clarification why it won't.
  
   It will satisfy my item #2 in the following way:
   I can duplicate your pre-commit build for Windows and add an input
   parameter, which would let people run the build on their patches
   chosen from local machine rather than attaching them to Jiras.
  
   Thanks,
   --Konstantin
  
In agile terms, you are the Owner of these requirements.  Please give
  me
owner feedback as to whether my proposed work sounds like it will
satisfy
the requirements.
   
Thank you,
--Matt
   
   
On Sun, Mar 3, 2013 at 12:16 PM, Konstantin Shvachko
shv.had...@gmail.com
wrote:
   
Didn't I explain in details what I am asking for?
   
Thanks,
--Konst
   
On Sun, Mar 3, 2013 at 11:08 AM, Matt Foley mfo...@hortonworks.com
wrote:
 Hi Konstantin,
 I'd like to point out two things:
 First, I already committed in this thread (email of Thu, Feb 28,
  2013
 at
 6:01 PM) to providing CI for Windows builds.  So please stop acting
 like
 I'm
 resisting this idea or something.
 Second, you didn't answer my question, you just kvetched about the
 phrasing.
 So I ask again:

 Will providing full test-patch integration (pre-commit build and
 unit
 test
 triggered by Jira Patch Available state) satisfy your request for
 functionality #1 and #2?  Yes or no, please.

 Thanks,
 --Matt


 On Sat, Mar 2, 2013 at 7:32 PM, Konstantin Shvachko
 shv.had...@gmail.com
 wrote:

 Hi Matt,

 On Sat, Mar 2, 2013 at 12:32 PM, Matt Foley 
  mfo...@hortonworks.com
 wrote:
  Konstantin,
  I would like to explore what it would take to remove this
  perceived
  impediment --

 Glad you decided to explore. Thank you.

  although I reserve the right to argue that this is not
  pre-requisite to merging the cross-platform support patch.

 It's your right indeed. So as mine to question what the platform
 support means for you, which I believe remained unclear.
 I do not impede the change as you should have noticed. My
 requirement
 comes from my perception of the support, which means to me exactly
 two
 things:
 1. The ability to recognise the code is 

Re: [Vote] Merge branch-trunk-win to trunk

2013-03-25 Thread Konstantin Boudnik
Yes, you are right of course - the mis-merged commit is the cause. Thanks for
pointing this out!

I think it would be beneficial if we had branch-2 on going build in the
Jenkins. I will go ahead and create one tonight.

Cos

On Mon, Mar 25, 2013 at 05:09PM, Suresh Srinivas wrote:
 Adding other mailing lists I missed earlier.
 
 Cos,
 
 There is progress being made on that ticket. Also it has nothing to do with
 that.
 
 Please follow the discussion here and why this happened due to an invalid
 commit that was reverted -
 https://issues.apache.org/jira/browse/HDFS-4615?focusedCommentId=13612650page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13612650
 
 Regards,
 Suresh
 
 
 On Mon, Mar 25, 2013 at 1:17 PM, Konstantin Boudnik c...@apache.org wrote:
 
  It doesn't look like any progress has been done on the ticket below in the
  last 3 weeks. And now branch-2 can't be compiled because of
 
 
  hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java:[895,15]
  WINDOWS is not public in org.apache.hadoop.fs.Path; cannot be accessed from
  outside package
 
  That's exactly why I was -1'ing this...
Cos
 
  On Mon, Mar 04, 2013 at 05:41PM, Matt Foley wrote:
   Thanks, gentlemen.  I've opened and taken responsibility for
   https://issues.apache.org/jira/browse/HADOOP-9359.  Giri Kesavan has
  agreed
   to help with the parts that require Jenkins admin access.
  
   Thanks,
   --Matt
  
  
  
   On Mon, Mar 4, 2013 at 5:00 PM, Konstantin Shvachko 
  shv.had...@gmail.comwrote:
  
+1 on the merge.
   
I am glad we agreed.
Having Jira to track the CI effort is a good idea.
   
Thanks,
--Konstantin
   
On Mon, Mar 4, 2013 at 3:29 PM, Matt Foley mfo...@hortonworks.com
  wrote:
 Thanks.  I agree Windows -1's in test-patch should not block commits.

 --Matt



 On Mon, Mar 4, 2013 at 2:30 PM, Konstantin Shvachko 
shv.had...@gmail.com
 wrote:

 On Mon, Mar 4, 2013 at 12:22 PM, Matt Foley mfo...@hortonworks.com
  
 wrote:
  Konstantine, you have voted -1, and stated some requirements
  before
  you'll
  withdraw that -1.  As I plan to do work to fulfill those
requirements, I
  want to make sure that what I'm proposing will, in fact, satisfy
  you.
  That's why I'm asking, if we implement full test-patch
  integration
for
  Windows, does it seem to you that that would provide adequate
  support?

 Yes.

  I have learned not to presume that my interpretation is correct.
   My
  interpretation of item #1 is that test-patch provides pre-commit
build,
  so
  it would satisfy item #1.  But rather than assuming that I am
  interpreting
  it correctly, I simply want your agreement that it would, or if
  not,
  clarification why it won't.

 I agree it will satisfy my item #1.
 I did not agree in my previous email, but I changed my mind based on
 the latest discussion. I have to explain why now.
 I was proposing nightly build because I did not want pre-commit
  build
 for Windows block commits to Linux. But if people are fine just
  ignoring
 -1s for the Windows part of the build it should be good.

  Regarding item #2, it is also my interpretation that test-patch
provides
  an
  on-demand (perhaps 20-minutes deferred) Jenkins build and unit
  test,
  with
  logs available to the developer, so it would satisfy item #2.  But
  rather
  than assuming that I am interpreting it correctly, I simply want
  your
  agreement that it would, or if not, clarification why it won't.

 It will satisfy my item #2 in the following way:
 I can duplicate your pre-commit build for Windows and add an input
 parameter, which would let people run the build on their patches
 chosen from local machine rather than attaching them to Jiras.

 Thanks,
 --Konstantin

  In agile terms, you are the Owner of these requirements.  Please
  give
me
  owner feedback as to whether my proposed work sounds like it will
  satisfy
  the requirements.
 
  Thank you,
  --Matt
 
 
  On Sun, Mar 3, 2013 at 12:16 PM, Konstantin Shvachko
  shv.had...@gmail.com
  wrote:
 
  Didn't I explain in details what I am asking for?
 
  Thanks,
  --Konst
 
  On Sun, Mar 3, 2013 at 11:08 AM, Matt Foley 
  mfo...@hortonworks.com
  wrote:
   Hi Konstantin,
   I'd like to point out two things:
   First, I already committed in this thread (email of Thu, Feb
  28,
2013
   at
   6:01 PM) to providing CI for Windows builds.  So please stop
  acting
   like
   I'm
   resisting this idea or something.
   Second, you didn't answer my question, you just kvetched about
  the
   phrasing.
   So I ask again:
  
   Will providing full test-patch integration

Re: [Vote] Merge branch-trunk-win to trunk

2013-03-25 Thread Konstantin Boudnik
Here's a daily build for hadoop-2 branch
  https://builds.apache.org/job/Hadoop-branch2

It just builds the full Hadoop project without running any tests (for now).
Can be easily extended to do test runs/artifact deployment, if needed.

Cos

On Mon, Mar 25, 2013 at 07:14PM, Konstantin Boudnik wrote:
 Yes, you are right of course - the mis-merged commit is the cause. Thanks for
 pointing this out!
 
 I think it would be beneficial if we had branch-2 on going build in the
 Jenkins. I will go ahead and create one tonight.
 
 Cos
 
 On Mon, Mar 25, 2013 at 05:09PM, Suresh Srinivas wrote:
  Adding other mailing lists I missed earlier.
  
  Cos,
  
  There is progress being made on that ticket. Also it has nothing to do with
  that.
  
  Please follow the discussion here and why this happened due to an invalid
  commit that was reverted -
  https://issues.apache.org/jira/browse/HDFS-4615?focusedCommentId=13612650page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13612650
  
  Regards,
  Suresh
  
  
  On Mon, Mar 25, 2013 at 1:17 PM, Konstantin Boudnik c...@apache.org wrote:
  
   It doesn't look like any progress has been done on the ticket below in the
   last 3 weeks. And now branch-2 can't be compiled because of
  
  
   hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java:[895,15]
   WINDOWS is not public in org.apache.hadoop.fs.Path; cannot be accessed 
   from
   outside package
  
   That's exactly why I was -1'ing this...
 Cos
  
   On Mon, Mar 04, 2013 at 05:41PM, Matt Foley wrote:
Thanks, gentlemen.  I've opened and taken responsibility for
https://issues.apache.org/jira/browse/HADOOP-9359.  Giri Kesavan has
   agreed
to help with the parts that require Jenkins admin access.
   
Thanks,
--Matt
   
   
   
On Mon, Mar 4, 2013 at 5:00 PM, Konstantin Shvachko 
   shv.had...@gmail.comwrote:
   
 +1 on the merge.

 I am glad we agreed.
 Having Jira to track the CI effort is a good idea.

 Thanks,
 --Konstantin

 On Mon, Mar 4, 2013 at 3:29 PM, Matt Foley mfo...@hortonworks.com
   wrote:
  Thanks.  I agree Windows -1's in test-patch should not block 
  commits.
 
  --Matt
 
 
 
  On Mon, Mar 4, 2013 at 2:30 PM, Konstantin Shvachko 
 shv.had...@gmail.com
  wrote:
 
  On Mon, Mar 4, 2013 at 12:22 PM, Matt Foley mfo...@hortonworks.com
   
  wrote:
   Konstantine, you have voted -1, and stated some requirements
   before
   you'll
   withdraw that -1.  As I plan to do work to fulfill those
 requirements, I
   want to make sure that what I'm proposing will, in fact, satisfy
   you.
   That's why I'm asking, if we implement full test-patch
   integration
 for
   Windows, does it seem to you that that would provide adequate
   support?
 
  Yes.
 
   I have learned not to presume that my interpretation is correct.
My
   interpretation of item #1 is that test-patch provides pre-commit
 build,
   so
   it would satisfy item #1.  But rather than assuming that I am
   interpreting
   it correctly, I simply want your agreement that it would, or if
   not,
   clarification why it won't.
 
  I agree it will satisfy my item #1.
  I did not agree in my previous email, but I changed my mind based 
  on
  the latest discussion. I have to explain why now.
  I was proposing nightly build because I did not want pre-commit
   build
  for Windows block commits to Linux. But if people are fine just
   ignoring
  -1s for the Windows part of the build it should be good.
 
   Regarding item #2, it is also my interpretation that test-patch
 provides
   an
   on-demand (perhaps 20-minutes deferred) Jenkins build and unit
   test,
   with
   logs available to the developer, so it would satisfy item #2.  
   But
   rather
   than assuming that I am interpreting it correctly, I simply want
   your
   agreement that it would, or if not, clarification why it won't.
 
  It will satisfy my item #2 in the following way:
  I can duplicate your pre-commit build for Windows and add an input
  parameter, which would let people run the build on their patches
  chosen from local machine rather than attaching them to Jiras.
 
  Thanks,
  --Konstantin
 
   In agile terms, you are the Owner of these requirements.  Please
   give
 me
   owner feedback as to whether my proposed work sounds like it will
   satisfy
   the requirements.
  
   Thank you,
   --Matt
  
  
   On Sun, Mar 3, 2013 at 12:16 PM, Konstantin Shvachko
   shv.had...@gmail.com
   wrote:
  
   Didn't I explain in details what I am asking for?
  
   Thanks,
   --Konst
  
   On Sun, Mar 3, 2013 at 11:08 AM, Matt Foley 
   mfo...@hortonworks.com

[jira] [Resolved] (HADOOP-9399) protoc maven plugin doesn't work on mvn 3.0.2

2013-03-22 Thread Konstantin Boudnik (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-9399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik resolved HADOOP-9399.


  Resolution: Fixed
Release Note: Committed to 2.0.4-alpha branch

 protoc maven plugin doesn't work on mvn 3.0.2
 -

 Key: HADOOP-9399
 URL: https://issues.apache.org/jira/browse/HADOOP-9399
 Project: Hadoop Common
  Issue Type: Bug
  Components: build
Affects Versions: 3.0.0
Reporter: Todd Lipcon
Assignee: Konstantin Boudnik
Priority: Minor
 Fix For: 3.0.0, 2.0.5-beta, 2.0.4-alpha

 Attachments: hadoop-9399.txt


 On my machine with mvn 3.0.2, I get a ClassCastException trying to use the 
 maven protoc plugin. The issue seems to be that mvn 3.0.2 sees the ListFile 
 parameter, and doesn't see the generic type argument, and stuffs Strings 
 inside instead. So, we get ClassCastException trying to use the objects as 
 Files.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Reopened] (HADOOP-9299) kerberos name resolution is kicking in even when kerberos is not configured

2013-03-18 Thread Konstantin Boudnik (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-9299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik reopened HADOOP-9299:



Backporting to 2.0.4-alpha

 kerberos name resolution is kicking in even when kerberos is not configured
 ---

 Key: HADOOP-9299
 URL: https://issues.apache.org/jira/browse/HADOOP-9299
 Project: Hadoop Common
  Issue Type: Bug
  Components: security
Affects Versions: 2.0.3-alpha
Reporter: Roman Shaposhnik
Assignee: Daryn Sharp
Priority: Blocker
 Fix For: 3.0.0, 2.0.5-beta

 Attachments: HADOOP-9299-branch2.0.4.patch, HADOOP-9299.patch, 
 HADOOP-9299.patch


 Here's what I'm observing on a fully distributed cluster deployed via Bigtop 
 from the RC0 2.0.3-alpha tarball:
 {noformat}
 528077-oozie-tucu-W@mr-node] Error starting action [mr-node]. ErrorType 
 [TRANSIENT], ErrorCode [JA009], Message [JA009: 
 org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: 
 No rules applied to yarn/localhost@LOCALREALM
 at 
 org.apache.hadoop.security.token.delegation.AbstractDelegationTokenIdentifier.init(AbstractDelegationTokenIdentifier.java:68)
 at 
 org.apache.hadoop.mapreduce.v2.api.MRDelegationTokenIdentifier.init(MRDelegationTokenIdentifier.java:51)
 at 
 org.apache.hadoop.mapreduce.v2.hs.HistoryClientService$HSClientProtocolHandler.getDelegationToken(HistoryClientService.java:336)
 at 
 org.apache.hadoop.mapreduce.v2.api.impl.pb.service.MRClientProtocolPBServiceImpl.getDelegationToken(MRClientProtocolPBServiceImpl.java:210)
 at 
 org.apache.hadoop.yarn.proto.MRClientProtocol$MRClientProtocolService$2.callBlockingMethod(MRClientProtocol.java:240)
 at 
 org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:454)
 at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1014)
 at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1735)
 at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1731)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:396)
 at 
 org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1441)
 at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1729)
 Caused by: 
 org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: 
 No rules applied to yarn/localhost@LOCALREALM
 at 
 org.apache.hadoop.security.authentication.util.KerberosName.getShortName(KerberosName.java:378)
 at 
 org.apache.hadoop.security.token.delegation.AbstractDelegationTokenIdentifier.init(AbstractDelegationTokenIdentifier.java:66)
 ... 12 more
 ]
 {noformat}
 This is submitting a mapreduce job via Oozie 3.3.1. The reason I think this 
 is a Hadoop issue rather than the oozie one is because when I hack 
 /etc/krb5.conf to be:
 {noformat}
 [libdefaults]
ticket_lifetime = 600
default_realm = LOCALHOST
default_tkt_enctypes = des3-hmac-sha1 des-cbc-crc
default_tgs_enctypes = des3-hmac-sha1 des-cbc-crc
 [realms]
LOCALHOST = {
kdc = localhost:88
default_domain = .local
}
 [domain_realm]
.local = LOCALHOST
 [logging]
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmin.log
default = FILE:/var/log/krb5lib.log
 {noformat}
 The issue goes away. 
 Now, once again -- the kerberos auth is NOT configured for Hadoop, hence it 
 should NOT pay attention to /etc/krb5.conf to begin with.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Resolved] (HADOOP-9299) kerberos name resolution is kicking in even when kerberos is not configured

2013-03-18 Thread Konstantin Boudnik (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-9299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik resolved HADOOP-9299.


   Resolution: Fixed
Fix Version/s: 2.0.4-alpha

Apparently it has been already merged into branch-2 earlier
r1457770 | daryn | 2013-03-18 07:05:09 -0700 (Mon, 18 Mar 2013) | 2 lines

svn merge -c 1457763 FIXES: HADOOP-9299.  kerberos name resolution is kicking 
in even when kerberos is not configured (daryn)

Thanks Daryn! The ticket hasn't been marked as such.


 kerberos name resolution is kicking in even when kerberos is not configured
 ---

 Key: HADOOP-9299
 URL: https://issues.apache.org/jira/browse/HADOOP-9299
 Project: Hadoop Common
  Issue Type: Bug
  Components: security
Affects Versions: 2.0.3-alpha
Reporter: Roman Shaposhnik
Assignee: Daryn Sharp
Priority: Blocker
 Fix For: 3.0.0, 2.0.5-beta, 2.0.4-alpha

 Attachments: HADOOP-9299-branch2.0.4.patch, HADOOP-9299.patch, 
 HADOOP-9299.patch


 Here's what I'm observing on a fully distributed cluster deployed via Bigtop 
 from the RC0 2.0.3-alpha tarball:
 {noformat}
 528077-oozie-tucu-W@mr-node] Error starting action [mr-node]. ErrorType 
 [TRANSIENT], ErrorCode [JA009], Message [JA009: 
 org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: 
 No rules applied to yarn/localhost@LOCALREALM
 at 
 org.apache.hadoop.security.token.delegation.AbstractDelegationTokenIdentifier.init(AbstractDelegationTokenIdentifier.java:68)
 at 
 org.apache.hadoop.mapreduce.v2.api.MRDelegationTokenIdentifier.init(MRDelegationTokenIdentifier.java:51)
 at 
 org.apache.hadoop.mapreduce.v2.hs.HistoryClientService$HSClientProtocolHandler.getDelegationToken(HistoryClientService.java:336)
 at 
 org.apache.hadoop.mapreduce.v2.api.impl.pb.service.MRClientProtocolPBServiceImpl.getDelegationToken(MRClientProtocolPBServiceImpl.java:210)
 at 
 org.apache.hadoop.yarn.proto.MRClientProtocol$MRClientProtocolService$2.callBlockingMethod(MRClientProtocol.java:240)
 at 
 org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:454)
 at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1014)
 at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1735)
 at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1731)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:396)
 at 
 org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1441)
 at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1729)
 Caused by: 
 org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: 
 No rules applied to yarn/localhost@LOCALREALM
 at 
 org.apache.hadoop.security.authentication.util.KerberosName.getShortName(KerberosName.java:378)
 at 
 org.apache.hadoop.security.token.delegation.AbstractDelegationTokenIdentifier.init(AbstractDelegationTokenIdentifier.java:66)
 ... 12 more
 ]
 {noformat}
 This is submitting a mapreduce job via Oozie 3.3.1. The reason I think this 
 is a Hadoop issue rather than the oozie one is because when I hack 
 /etc/krb5.conf to be:
 {noformat}
 [libdefaults]
ticket_lifetime = 600
default_realm = LOCALHOST
default_tkt_enctypes = des3-hmac-sha1 des-cbc-crc
default_tgs_enctypes = des3-hmac-sha1 des-cbc-crc
 [realms]
LOCALHOST = {
kdc = localhost:88
default_domain = .local
}
 [domain_realm]
.local = LOCALHOST
 [logging]
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmin.log
default = FILE:/var/log/krb5lib.log
 {noformat}
 The issue goes away. 
 Now, once again -- the kerberos auth is NOT configured for Hadoop, hence it 
 should NOT pay attention to /etc/krb5.conf to begin with.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


Re: [Vote] Merge branch-trunk-win to trunk

2013-03-04 Thread Konstantin Boudnik
Ok, looks like we are converging on this across a few hundred emails ;)

So, as has been stated elsewhere: test-patch will be improved to fully support
Windows; furthermore -1 from Windows' test-patch won't block Linux commits.
This is ok with me.

Can we have a JIRA ticket for that test-patch work assigned to the real owner,
so it can be tracked? I am +1 in this case.

Cos

On Fri, Mar 01, 2013 at 04:57PM, Chris Douglas wrote:
 On Fri, Mar 1, 2013 at 1:57 PM, Konstantin Shvachko
 shv.had...@gmail.com wrote:
  Commitment is a good thing.
  I think the two builds that I proposed are a prerequisite for Win support.
  If we commit windows patch people will start breaking it the next day.
  Which we wont know without the nightly build and wont be able to fix
  without the on-demand one.
 
 As several people have pointed out already, the surface of possible
 conflicts is relatively limited, and- as you did in 2007- the devs on
 Windows will report and fix bugs in that platform as they find them.
 CI is important for detecting and preventing bugs, but this isn't
 software we're launching into orbit.
 
  Making two builds is less than 2 days work, imho, given that there is
  a Windows node available and that mvn targets are in place. Correct me
  if I missed any complications in the process.
 
 On Fri, Mar 1, 2013 at 3:47 PM, Konstantin Boudnik c...@apache.org wrote:
  It seems that with the HW in place, the matter of setting at least nightly
  build is trivial for anyone with up to date Windows knowledge. I wish I 
  could
  help. Going without a validation is a recipe for a disaster IMO.
 
 Fair enough, though that also implies that the window for regressions
 is small, and it leaves little room to doubt that this will receive
 priority. Until it's merged, spurious notifications that the current
 trunk breaks Windows are an awkward introduction to devs' workflow.
 The order of merge/CI is a choice between mild annoyances, really.
 
 But it might be moot. Giri: you're doing the work on this. When do you
 think it can be complete? -C


Re: [Vote] Merge branch-trunk-win to trunk

2013-03-01 Thread Konstantin Boudnik
It seems that with the HW in place, the matter of setting at least nightly
build is trivial for anyone with up to date Windows knowledge. I wish I could
help. Going without a validation is a recipe for a disaster IMO.

-1 until some reasonable solution is implemented.
  Cos

On Fri, Mar 01, 2013 at 01:57PM, Konstantin Shvachko wrote:
 Commitment is a good thing.
 I think the two builds that I proposed are a prerequisite for Win support.
 If we commit windows patch people will start breaking it the next day.
 Which we wont know without the nightly build and wont be able to fix
 without the on-demand one.
 
 Making two builds is less than 2 days work, imho, given that there is
 a Windows node available and that mvn targets are in place. Correct me
 if I missed any complications in the process.
 
 Thanks,
 --Konst
 
 On Fri, Mar 1, 2013 at 1:28 PM, Chris Douglas cdoug...@apache.org wrote:
  Konstantin-
 
  There's no debate on the necessity of CI and related infrastructure to
  support the platform well. Suresh outlined the support to effect this
  here: http://s.apache.org/s1
 
  Is the commitment to establish this infrastructure after the merge
  sufficient? -C
 
  On Fri, Mar 1, 2013 at 12:18 PM, Konstantin Shvachko
  shv.had...@gmail.com wrote:
  -1
  We should have a CI infrastructure in place before we can commit to
  supporting Windows platform.
 
  Eric is right Win/Cygwin was supported since day one.
  I had a Windows box under my desk running nightly builds back in 2006-07.
  People were irritated but I was filing windows bugs until 0.22 release.
  Times changing and I am glad to see wider support for Win platform.
 
  But in order to make it work you guys need to put the CI process in place
 
  1. windows jenkins build: could be nightly or PreCommit.
  - Nightly would mean that changes can be committed to trunk based on
  linux PreCommit build. And people will file bugs if the change broke
  Windows nightly build.
  - PreCommit-win build will mean automatic reporting failed tests to
  respective jira blocking commits the same way as it is now with linux
  PreCommit builds.
  We should discuss which way is more efficient for developers.
 
  2. On-demand-windows Jenkins build.
  I see it as a build to which I can attach my patch and the build will
  run my changes on a dedicated windows box.
  That way people can test their changes without having personal windows 
  nodes.
 
  I think this is the minimal set of requirement for us to be able to
  commit to the new platform.
  Right now I see only one windows related build
  https://builds.apache.org/view/Hadoop/job/Hadoop-1-win/
  Which was failing since Sept 8, 2012 and did not run in the last month.
 
  Thanks,
  --Konst
 
  On Thu, Feb 28, 2013 at 8:47 PM, Eric Baldeschwieler
  eri...@hortonworks.com wrote:
  +1 (non-binding)
 
  A few of observations:
 
  - Windows has actually been a supported platform for Hadoop since 0.1 .  
  Doug championed supporting windows then and we've continued to do it with 
  varying vigor over time.  To my knowledge we've never made a decision to 
  drop windows support.  The change here is improving our support and 
  dropping the requirement of cigwin.  We had Nutch windows users on the 
  list in 2006 and we've been supporting windows FS requirements since 
  inception.
 
  - A little pragmatism will go a long way.  As a community we've got to 
  stay committed to keeping hadoop simple (so it does work on many 
  platforms) and extending it to take advantage of key emerging OS/hardware 
  features, such as containers, new FSs, virtualization, flash ...  We 
  should all plan to let new features  optimizations emerge that don't 
  work everywhere, if they are compelling and central to hadoop's mission 
  of being THE best fabric for storing and processing big data.
 
  - A UI project like KDE has to deal with the MANY differences between 
  windows and linux UI APIs.  Hadoop faces no such complex challenge and 
  hence can be maintained from a single codeline IMO.  It is mostly 
  abstracted from the OS APIs via Java and our design choices.  Where it is 
  not we can continue to add plugable abstractions.
 


Re: [Vote] Merge branch-trunk-win to trunk

2013-03-01 Thread Konstantin Boudnik
Suresh, I appreciate all the troubles you're going through wrt syncing up the
huge patch for a long time - I really do.

I am not asking to have full test-patch process in place. But I think it is a
real good idea to have a way to run the full test suite once in a while - or
as Konstantin proposing - for a given patch, to make sure that codebase
doesn't bitrot at the edges.

Official support has nothing to do with the issue - you're trying to build a
straw man argument around this. What is relevant, on the other hand, is that
Windows is so different from _any_ Unix or pseudo-Unix flavors, including
Windows with Cygwin - that even multi-platform Java has hard hard time
dealing with it. This is enough, IMO, to warrant a separate checkpoint.

I hope I have explained myself better.
  Cos

On Fri, Mar 01, 2013 at 05:55PM, Suresh Srinivas wrote:
  It seems that with the HW in place, the matter of setting at least nightly
  build is trivial for anyone with up to date Windows knowledge. I wish I
  could
  help. Going without a validation is a recipe for a disaster IMO.
 
  -1 until some reasonable solution is implemented.
Cos
 
 
 Cos, I have hard time understanding your veto?
 
 Here is my rationale for merge:
 Currently with all the cross platform support, the merge patch has +1 from
 Jenkins on Linux. So no regression has been introduced in Hadoop on Linux.
 
 As regards to Windows support I want to make two points:
 1. Until Jenkins is setup and we are passing all the tests, I am okay, as
 Konstntin proposed, if we do not officially declare Windows as supported. I
 do not want to tie the patch merge with setting up Windows Jenkins. I have
 been maintaining this branch for a long time and keeping it in sync with
 trunk is non-trivial.
 2. After Jenkins is setup, there is a concern in the community about -1
 from Windows hindering patch commits. As others have already suggested in
 the thread, I am okay committing new patches even if -1 is posted by
 Jenkins on Windows. The team that worked on Windows will fix the issues. I
 do not see many such issues cropping up.


signature.asc
Description: Digital signature


Re: [Vote] Merge branch-trunk-win to trunk

2013-02-28 Thread Konstantin Boudnik
On Thu, Feb 28, 2013 at 03:08PM, sanjay Radia wrote:
 +1
 Java has done the bulk of the work in making Hadoop multi-platform.
 Windows specific code is a tiny percentage of the code.
 Jeninks support for windows is going help us keep the platform portable going 
 forward.
 I expect that the vast majority of new commits have  no problems. I propose
 that we start by fixing problems that Jenkins raises but not block new
 commits for too long if the author does not have a windows box or if a
 volunteer does not step up.

Considering a typical set of software most of the people here work with it
would be completely inappropriate to block commits for failing Windows
specific features. After all, Microsoft never did bother to check what
features or compatibilty matters they have broke in Java and elsewhere, so why
should we?

I believe this kind of rules have to be set and discussed before the merge is
done.

Cheers,
  Cos


signature.asc
Description: Digital signature


Re: ANNOUNCEMENT: Project Rhino: Enhanced Data Protection for the Apache Hadoop Ecosystem

2013-02-25 Thread Konstantin Boudnik
[yanking away most of the cross-posts...]

An interesting cross component project Avik. Any plans to incubate it in Apache?

Cos

On Mon, Feb 25, 2013 at 11:46PM, Dey, Avik wrote:
 Project Rhino
 
 As the Apache Hadoop ecosystem extends into new markets and sees new use
 cases with security and compliance challenges, the benefits of processing
 sensitive and legally protected data with Hadoop must be coupled with
 protection for private information that limits performance impact. Project
 Rhinohttps://github.com/intel-hadoop/project-rhino/ is our open source
 effort to enhance the existing data protection capabilities of the Hadoop
 ecosystem to address these challenges, and contribute the code back to
 Apache.
 
 The core of the Apache Hadoop ecosystem as it is commonly understood is:
 
 - Core: A set of shared libraries
 - HDFS: The Hadoop filesystem
 - MapReduce: Parallel computation framework
 - ZooKeeper: Configuration management and coordination
 - HBase: Column-oriented database on HDFS
 - Hive: Data warehouse on HDFS with SQL-like access
 - Pig: Higher-level programming language for Hadoop computations
 - Oozie: Orchestration and workflow management
 - Mahout: A library of machine learning and data mining algorithms
 - Flume: Collection and import of log and event data
 - Sqoop: Imports data from relational databases
 
 These components are all separate projects and therefore cross cutting 
 concerns like authN, authZ, a consistent security policy framework, 
 consistent authorization model and audit coverage are loosely coordinated. 
 Some security features expected by our customers, such as encryption, are 
 simply missing. Our aim is to take a full stack view and work with the 
 individual projects toward consistent concepts and capabilities, filling gaps 
 as we go.
 
 Our initial goals are:
 
 1) Framework support for encryption and key management
 
 There is currently no framework support for encryption or key management. We 
 will add this support into Hadoop Core and integrate it across the ecosystem.
 
 2) A common authorization framework for the Hadoop ecosystem
 
 Each component currently has its own authorization engine. We will abstract 
 the common functions into a reusable authorization framework with a 
 consistent interface. Where appropriate we will either modify an existing 
 engine to work within this framework, or we will plug in a common default 
 engine. Therefore we also must normalize how security policy is expressed and 
 applied by each component. Core, HDFS, ZooKeeper, and HBase currently support 
 simple access control lists (ACLs) composed of users and groups. We see this 
 as a good starting point. Where necessary we will modify components so they 
 each offer equivalent functionality, and build support into others.
 
 3) Token based authentication and single sign on
 
 Core, HDFS, ZooKeeper, and HBase currently support Kerberos authentication at 
 the RPC layer, via SASL. However this does not provide valuable attributes 
 such as group membership, classification level, organizational identity, or 
 support for user defined attributes. Hadoop components must interrogate 
 external resources for discovering these attributes and at scale this is 
 problematic. There is also no consistent delegation model. HDFS has a simple 
 delegation capability, and only Oozie can take limited advantage of it. We 
 will implement a common token based authentication framework to decouple 
 internal user and service authentication from external mechanisms used to 
 support it (like Kerberos).
 
 4) Extend HBase support for ACLs to the cell level
 
 Currently HBase supports setting access controls at the table or column 
 family level. However, many use cases would benefit from the additional 
 capability to do this on a per cell basis. In fact for many users dealing 
 with sensitive information the ability to do this is crucial.
 
 5) Improve audit logging
 
 Audit messages from various Hadoop components do not use a unified or even 
 consistently formatted format. This makes analysis of logs for verifying 
 compliance or taking corrective action difficult. We will build a common 
 audit logging facility as part of the common authorization framework work. We 
 will also build a set of common audit log processing tools for transforming 
 them to different industry standard formats, for supporting compliance 
 verification, and for triggering responses to policy violations.
 
 Current JIRAs:
 
 As part of this ongoing effort we are contributing our work to-date against 
 the JIRAs listed below. As you may appreciate, the goals for Project Rhino 
 covers a number of different Apache projects, the scope of work is 
 significant and likely to only increase as we get additional community input. 
 We also appreciate that there may be others in the Apache community that may 
 be working on some of this or are interested in contributing to it. If so, we 
 look forward to partnering with you in 

Re: [VOTE] Release hadoop-2.0.3-alpha

2013-02-12 Thread Konstantin Boudnik
We've created BigTop stack with 2.0.3 as the base. Ran YCSB, slive, and some
other loads on up to 20 nodes clusters as a part of the release validation.
Two issues were noted:

- due to the known issue with Jetty we are seeing MR jobs hanging here and
  there
- without properly configured capacity-scheduler.xml resource manager throws
  exception on the startup (BIGTOP-841).

+1

Cos

On Wed, Feb 06, 2013 at 07:59PM, Arun C Murthy wrote:
 Folks,
 
 I've created a release candidate (rc0) for hadoop-2.0.3-alpha that I would 
 like to release.
 
 This release contains several major enhancements such as QJM for HDFS HA,
 multi-resource scheduling for YARN, YARN ResourceManager restart etc.  Also
 YARN has achieved significant stability at scale (more details from Y! folks
 here: http://s.apache.org/VYO).
 
 The RC is available at: 
 http://people.apache.org/~acmurthy/hadoop-2.0.3-alpha-rc0/
 The RC tag in svn is here: 
 http://svn.apache.org/viewvc/hadoop/common/tags/release-2.0.3-alpha-rc0/
 
 The maven artifacts are available via repository.apache.org.
 
 Please try the release and vote; the vote will run for the usual 7 days.
 
 thanks,
 Arun
 
 
 
 --
 Arun C. Murthy
 Hortonworks Inc.
 http://hortonworks.com/
 
 


signature.asc
Description: Digital signature


Re: [VOTE] Release hadoop-2.0.3-alpha

2013-02-12 Thread Konstantin Boudnik
On Tue, Feb 12, 2013 at 07:44PM, Konstantin Boudnik wrote:
 We've created BigTop stack with 2.0.3 as the base. Ran YCSB, slive, and some
 other loads on up to 20 nodes clusters as a part of the release validation.
 Two issues were noted:
 
 - due to the known issue with Jetty we are seeing MR jobs hanging here and
   there

The issue referred here is reported as MAPREDUCE-2980

 - without properly configured capacity-scheduler.xml resource manager throws
   exception on the startup (BIGTOP-841).
 
 +1
 
 Cos
 
 On Wed, Feb 06, 2013 at 07:59PM, Arun C Murthy wrote:
  Folks,
  
  I've created a release candidate (rc0) for hadoop-2.0.3-alpha that I would 
  like to release.
  
  This release contains several major enhancements such as QJM for HDFS HA,
  multi-resource scheduling for YARN, YARN ResourceManager restart etc.  Also
  YARN has achieved significant stability at scale (more details from Y! folks
  here: http://s.apache.org/VYO).
  
  The RC is available at: 
  http://people.apache.org/~acmurthy/hadoop-2.0.3-alpha-rc0/
  The RC tag in svn is here: 
  http://svn.apache.org/viewvc/hadoop/common/tags/release-2.0.3-alpha-rc0/
  
  The maven artifacts are available via repository.apache.org.
  
  Please try the release and vote; the vote will run for the usual 7 days.
  
  thanks,
  Arun
  
  
  
  --
  Arun C. Murthy
  Hortonworks Inc.
  http://hortonworks.com/
  
  




signature.asc
Description: Digital signature


Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-12-12 Thread Konstantin Boudnik
On Sat, Dec 01, 2012 at 10:44AM, Steve Loughran wrote:
 On 1 December 2012 01:08, Eli Collins e...@cloudera.com wrote:
 
  -1, 0, -1
 
  IIUC the only platform we plan to add support for that we can't easily
  support today (w/o an emulation layer like cygwin) is Windows, and it
  seems like making the bash scripts simpler and having parallel bat
  files is IMO a better approach.
 
 
 WinNT Bat/CMD files are the worst possible scripting language invented. At
 the very least, .py should be the language of choice there

Compare to the OS in question - it isn't _that_ bad ;)



Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-12-12 Thread Konstantin Boudnik
On Sat, Dec 01, 2012 at 10:07PM, Eric Yang wrote:
 -1, +1, -1
 
 Python has fairly inconsistent support across all major OS vendors.  It is
 hard to get it right unless the scripts are all designed to make use of
 Python 2.4.  However, Python 2.4 doesn't have necessary OS features to make
 Python useful in runtime or build environment unless you write a lot of
 custom modules.  Which defeats the purpose to use python as intermediate
 layer to do OS dependent work.  Jruby may be a better choice.

JRuby? Really? Groovy is already there and it is really a Java dialect unlike
JRuby. And yes - it is quite suitable for build things, considering the use of
it in BigTop

Cos

 On Sat, Dec 1, 2012 at 12:28 PM, Joep Rottinghuis 
 jrottingh...@gmail.comwrote:
 
  0, 0, -1 (non-binding)
 
  Joep
 
  On Nov 24, 2012, at 12:13 PM, Matt Foley ma...@apache.org wrote:
 
   For discussion, please see previous thread [PROPOSAL] introduce Python
  as
   build-time and run-time dependency for Hadoop and throughout Hadoop
  stack.
  
   This vote consists of three separate items:
  
   1. Contributors shall be allowed to use Python as a platform-independent
   scripting language for build-time tasks, and add Python as a build-time
   dependency.
   Please vote +1, 0, -1.
  
   2. Contributors shall be encouraged to use Maven tasks in combination
  with
   either plug-ins or Groovy scripts to do cross-platform build-time tasks,
   even under ant in Hadoop-1.
   Please vote +1, 0, -1.
  
   3. Contributors shall be allowed to use Python as a platform-independent
   scripting language for run-time tasks, and add Python as a run-time
   dependency.
   Please vote +1, 0, -1.
  
   Note that voting -1 on #1 and +1 on #2 essentially REQUIRES contributors
  to
   use Maven plug-ins or Groovy as the only means of cross-platform
  build-time
   tasks, or to simply continue using platform-dependent scripts as is being
   done today.
  
   Vote closes at 12:30pm PST on Saturday 1 December.
   -
   Personally, my vote is +1, +1, +1.
   I think #2 is preferable to #1, but still has many unknowns in it, and
   until those are worked out I don't want to delay moving to cross-platform
   scripts for build-time tasks.
  
   Best regards,
   --Matt
 


Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-11-26 Thread Konstantin Boudnik
-1, +1, -1

Thanks

On Sat, Nov 24, 2012 at 12:13PM, Matt Foley wrote:
 For discussion, please see previous thread [PROPOSAL] introduce Python as
 build-time and run-time dependency for Hadoop and throughout Hadoop stack.
 
 This vote consists of three separate items:
 
 1. Contributors shall be allowed to use Python as a platform-independent
 scripting language for build-time tasks, and add Python as a build-time
 dependency.
 Please vote +1, 0, -1.
 
 2. Contributors shall be encouraged to use Maven tasks in combination with
 either plug-ins or Groovy scripts to do cross-platform build-time tasks,
 even under ant in Hadoop-1.
 Please vote +1, 0, -1.
 
 3. Contributors shall be allowed to use Python as a platform-independent
 scripting language for run-time tasks, and add Python as a run-time
 dependency.
 Please vote +1, 0, -1.
 
 Note that voting -1 on #1 and +1 on #2 essentially REQUIRES contributors to
 use Maven plug-ins or Groovy as the only means of cross-platform build-time
 tasks, or to simply continue using platform-dependent scripts as is being
 done today.
 
 Vote closes at 12:30pm PST on Saturday 1 December.
 -
 Personally, my vote is +1, +1, +1.
 I think #2 is preferable to #1, but still has many unknowns in it, and
 until those are worked out I don't want to delay moving to cross-platform
 scripts for build-time tasks.
 
 Best regards,
 --Matt


Re: [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

2012-11-24 Thread Konstantin Boudnik
If we decide to go with Maven then there's no point to complicate the
picture with jython. This time I will keep the offensive about *yton to myself
;)

Cos

On Sat, Nov 24, 2012 at 10:26PM, Radim Kolar wrote:
 we have not discussed advantages of stand alone python vs
 jython-in-maven pom
 
 http://code.google.com/p/jy-maven-plugin/
 
 language is about same, and it does not needs to have installed,
 which is advantage on windows.


Re: [ANNOUNCE] Hadoop branch-1.1 and Release Plan for Hadoop-1.1.0

2012-07-07 Thread Konstantin Boudnik
Also, I have updated HADOOP-8417 against 1.1.0 and we need to include it.
Otherwise, 1.1 will have the same issues for the downstream projects as 1.0.3
had.

Cos

On Sat, Jul 07, 2012 at 05:52PM, Konstantin Boudnik wrote:
 Matt,
 
 Thanks for the update.
 
 HADOOP-8399 would be beneficial for BigTop release and it is marked for 1.1.0
 release. The patch is available for a while now and if someone can review I'd
 go ahead and commit it today. 
 
 I am working on the content of 0.3.1 BigTop release and will shortly post the
 vote for it. Once Hadoop 1.1 rc is cut we'll start testing it with the rest of
 the stack.
 
 Cos
 
 On Fri, Jul 06, 2012 at 02:24PM, Matt Foley wrote:
  Hi Cos,
  the query string didn't come thru on the link you sent, but the jira query
  I use is:
  project in (HADOOP,HDFS,MAPREDUCE) and ((Target Version/s = '1.1.0'
  and (fixVersion != '1.1.0' or fixVersion is EMPTY)) or (fixVersion =
  '1.1.0' and Target Version/s is EMPTY)) and (status != Closed and status
  != Resolved) ORDER BY KEY
  
  You're correct that there are quite a few, currently 107, open jiras
  originally targeted for 1.1.0 that do not have committed fixes.  Many of
  these are just the inherited backlog of previously identified work.  I need
  to move them to Target Version/s = 1.1.1.
  
  Folks have requested that the following currently open jiras be included in
  1.1.0:
  
  HADOOP-8417 - HADOOP-6963 didn't update hadoop-core-pom-template.xml
  HADOOP-8445 - Token should not print the password in toString
  HDFS-96 - HDFS does not support blocks greater than 2GB
  HDFS-3596 - Improve FSEditLog pre-allocation in branch-1
  MAPREDUCE-2903 - Map Tasks graph is throwing XML Parse error when Job is
  executed with 0 maps
  MAPREDUCE-2129 - Job may hang if
  mapreduce.job.committer.setup.cleanup.needed=false and
  mapreduce.map/reduce.failures.maxpercent0
  ---
  MAPREDUCE-4049 - plugin for generic shuffle service
  HADOOP-7823 - port HADOOP-4012 to branch-1 (splitting support for bzip2)
  
  The first six are simple patches that I am comfortable including.
  The last two are complex patches that have not yet been committed.
  I am planning to defer those two to 1.1.1.
  
  Beyond that, I'm going to cut 1.1.0-rc0 from the current state of
  branch-1.1.
  I'm planning to do that this weekend.  This is obviously delayed from the
  previous plan, for which I apologize.
  
  Comments welcome.
  --Matt
  
  
  On Tue, Jul 3, 2012 at 8:32 PM, Konstantin Boudnik c...@apache.org wrote:
  
   Hi Matt.
  
   I am picking up the hat of BigTop's maintainer for Hadoop 1.x line. And I
   wanted to sync up with about the Hadoop 1.1 release outlook, progress,
   what help
   you might need, etc.
  
   I see a few jiras left open in the release
   http://is.gd/OyuaNQ
   Is this the correct representation of the current status?
   How I can help from BigTop side (I haven't yet finalized the stack's
   versions), etc. Looking forward for your input. Thanks.
  
 Cos
  
   On Fri, May 25, 2012 at 02:49PM, Matt Foley wrote:
Greetings.  With the approval of a public vote on common-dev@, I have
branched Hadoop branch-1 to create branch-1.1.  From this, I will create
   a
release candidate RC-0 for Hadoop-1.1.0, hopefully to be available
   shortly
after this weekend.
   
There are over 80 patches in branch-1, over and above the contents of
hadoop-1.0.3.  So I anticipate that some stabilization will be needed,
before the RC can be approved as a 1.1.0 release.  Your participation in
assuring a stable RC is very important.  When it becomes available,
   please
download it and work with it to determine whether it is stable enough to
release, and report issues found.  My colleagues and I will do likewise,
   of
course, but no one company can adequately exercise a new release with
   this
many new contributions.
   
There are two outstanding issue that are not yet committed, but I know
   the
contributors hope to see in 1.1.0:
MAPREDUCE-4049 https://issues.apache.org/jira/browse/MAPREDUCE-4049
   
HADOOP-4012 https://issues.apache.org/jira/browse/HADOOP-4012
Assuming there is an RC-1, and that these two patches can be committed
during stabilization of RC-0, I will plan to incorporate these 
additional
items in RC-1.
   
Best regards,
--Matt
Release Manager
  




signature.asc
Description: Digital signature


[jira] [Resolved] (HADOOP-8417) HADOOP-6963 didn't update hadoop-core-pom-template.xml

2012-07-07 Thread Konstantin Boudnik (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-8417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik resolved HADOOP-8417.


Resolution: Fixed

Committed to branch-1.1. Thank you Zhihong.

 HADOOP-6963 didn't update hadoop-core-pom-template.xml
 --

 Key: HADOOP-8417
 URL: https://issues.apache.org/jira/browse/HADOOP-8417
 Project: Hadoop Common
  Issue Type: Bug
Affects Versions: 1.0.3
Reporter: Zhihong Ted Yu
Assignee: Zhihong Ted Yu
 Fix For: 1.1.0

 Attachments: hadoop-8417-v2.txt, hadoop-8417.txt


 HADOOP-6963 introduced commons-io 2.1 in ivy.xml but forgot to update the 
 hadoop-core-pom-template.xml.
 This has caused map reduce jobs in downstream projects to fail with:
 {code}
 Caused by: java.lang.ClassNotFoundException: org.apache.commons.io.FileUtils
   at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
   at java.security.AccessController.doPrivileged(Native Method)
   at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
   at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
   at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
   at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
   ... 15 more
 {code}
 This caused a regression for 1.0.3 because downstream projects used to not 
 directly depend on commons-io

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




Re: Modification of wiki page Distributions and Commercial Support

2012-06-28 Thread Konstantin Boudnik
I will, I have the access.

Cos

On Thu, Jun 28, 2012 at 09:52AM, Roman Shaposhnik wrote:
 On Thu, Jun 28, 2012 at 3:27 AM, Olivier Lamy ol...@apache.org wrote:
  Hi,
  What is the process to modify the page
  http://wiki.apache.org/hadoop/Distributions%20and%20Commercial%20Support
  ?
 
 On a related note, I would like to add Apache Bigtop (incubating)
 Hadoop distribution
 to that wiki page.
 
 Can anybody help me with that? The wiki seems to be locked.
 
 Thanks,
 Roman.


Re: Modification of wiki page Distributions and Commercial Support

2012-06-28 Thread Konstantin Boudnik
Sorry, did what? The page doesn't have any references to the BigTop as of now.

Cos

On Thu, Jun 28, 2012 at 07:11PM, Olivier Lamy wrote:
 Thanks!
 Note: Harsh J  did it and give me karma to modify the page.
 
 --
 Olivier
 
 
 2012/6/28 Konstantin Boudnik c...@apache.org:
  I will, I have the access.
 
  Cos
 
  On Thu, Jun 28, 2012 at 09:52AM, Roman Shaposhnik wrote:
  On Thu, Jun 28, 2012 at 3:27 AM, Olivier Lamy ol...@apache.org wrote:
   Hi,
   What is the process to modify the page
   http://wiki.apache.org/hadoop/Distributions%20and%20Commercial%20Support
   ?
 
  On a related note, I would like to add Apache Bigtop (incubating)
  Hadoop distribution
  to that wiki page.
 
  Can anybody help me with that? The wiki seems to be locked.
 
  Thanks,
  Roman.
 
 
 
 -- 
 Olivier Lamy
 Talend: http://coders.talend.com
 http://twitter.com/olamy | http://linkedin.com/in/olamy


[jira] [Resolved] (HADOOP-7084) Remove java5 dependencies from site's build

2012-06-26 Thread Konstantin Boudnik (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-7084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik resolved HADOOP-7084.


Resolution: Duplicate

This is a dup of Hadoop-7072

 Remove java5 dependencies from site's build
 ---

 Key: HADOOP-7084
 URL: https://issues.apache.org/jira/browse/HADOOP-7084
 Project: Hadoop Common
  Issue Type: Improvement
  Components: build
Reporter: Konstantin Boudnik
Assignee: Konstantin Boudnik

 Java5 dependency needs to be removed from 
 http://svn.apache.org/repos/asf/hadoop/site/build.xml

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




Re: [VOTE - RELEASE PLAN] create branch-1.1 and make release candidate 1.1.0

2012-05-17 Thread Konstantin Boudnik
+1

On Thu, May 17, 2012 at 06:52PM, Matt Foley wrote:
 Hi all,
 I would like to branch branch-1.1 from the current HEAD of branch-1,
 and proceed to make a release candidate for hadoop-1.1.0 from it.
 I'm also hereby offering to be Release Manager for branch-1.1, as I have
 been
 for branch-1.0.
 
 There's a LOT of stuff in branch-1, over 80 patches not in the branch-1.0
 line, so I anticipate
 the need for some stabilization.  So I'll float the RC0 for a week or so,
 before possibly calling
 a vote on it.  Sound okay?
 
 Somewhat to my surprise, I found that our bylaws require a seven-day formal
 vote
 on this proposal.  So, please vote.  Voting will close on Thursday 24 May,
 7pm PDT.
 
 Best regards,
 --Matt


signature.asc
Description: Digital signature


Re: [VOTE] Hadoop-1.0.3-rc1

2012-05-14 Thread Konstantin Boudnik
Can we include HADOOP-8399 that takes care about annoyance of Java5 need?

Cos

On Mon, May 14, 2012 at 07:11PM, Tsz Wo (Nicholas), Sze wrote:
 Hi,
 
 
 I have verified all signatures and message digests.═ I also have
 - tested WebHDFS with curl;
 - run pi with 3 maps and 100 samples per map, the estimated value of Pi 
 is 3.14156800; and
 - run distbbp to compute the billionth bits of pi.
 All results look good.
 
 +1 on rc1.
 
 Regards,
 Tsz-Wo
 --
 $hadoop-1.0.3$./bin/hadoop jar ../distbbp.jar distbbp 10 1024 10 1 
 x-m1t2-r1t2-0 /distbbp /10e9
 STARTUP Mon May 14 18:39:02 PDT 2012
 ═ Started at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
 ═ Printed at org.apache.hadoop.examples.pi.DistSum.init(DistSum.java:428)
 DistBbp.VERSION = 20100731
 
 ...
 
 b = 1,000,000,000 (bits skipped)
 [32:
 ═ 3E08FF2B 03F1829E 05C038A3 2884A9C4 E7DEF417 875B1C22 D2DDDA11 D99573D8 
 43F80107 AB3A56CC 
 ═ C975C849 2DC5AD43 1D191704 9064DE41 67DE5C6E 0F264D33 D903CE49 F2324781 
 D9D7ED45 FAA9272D 
 ═ CAA42278 E8EF2FFF 450E2183 25FA9AB3 459D01AF AEF8AF79 DD6CC766 12152C31 
 2F31ABCD DCF01DEC 
 ═ 67332105 643A438D 
 ]
 
 CPU time = 1012611ms = 16:52.611
 END Mon May 14 18:43:31 PDT 2012
 
 
 - Original Message -
 From: Matt Foley ma...@apache.org
 Date: Tue, May 8, 2012 at 6:25 PM
 Subject: [VOTE] Hadoop-1.0.3-rc1
 To: common-dev@hadoop.apache.org
 
 
 
 Hi all,release Candidate for Hadoop-1.0.3 is now available for vote, at
 ═ ══http://people.apache.org/~mattf/hadoop-1.0.3-rc1/
 
 There are 29 bug fixes and enhancements in this release, including:
 
 4 patches in support of non-Oracle JDKs═
 several patches to clean up error handling and log═messages
 
 various production issues
 Please see the Release Notes for the full list, at:
 ═ ══http://people.apache.org/~mattf/hadoop-1.0.3-rc1/releasenotes.html
 
 Vote will run the statutory 7 days, and close on Tuesday 15 May, at 6:30pm 
 PDT.
 
 Thank you,
 --Matt
 
 Release Manager, Hadoop 1.0
 


signature.asc
Description: Digital signature


[jira] [Created] (HADOOP-8399) Remove JDK5 dependency from Hadoop 1.0+ line

2012-05-13 Thread Konstantin Boudnik (JIRA)
Konstantin Boudnik created HADOOP-8399:
--

 Summary: Remove JDK5 dependency from Hadoop 1.0+ line
 Key: HADOOP-8399
 URL: https://issues.apache.org/jira/browse/HADOOP-8399
 Project: Hadoop Common
  Issue Type: Bug
  Components: build
Affects Versions: 1.0.2
Reporter: Konstantin Boudnik


This issues has been fixed in Hadoop starting from 0.21 (see HDFS-1552).
I propose to make the same fix for 1.0 line and get rid of JDK5 dependency all 
together.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




Re: About Herriot

2012-04-27 Thread Konstantin Boudnik
Yeah, this is pretty much everything you need to know. A coupla additional
points:
  - Herriot hasn't been integrated to current trunk - the mavenization is
still due 
  - you apparently need to create instrumented bits of your cluster before you
can use Herriot 

Cos

On Fri, Apr 27, 2012 at 02:29PM, Nitin Pawar wrote:
 you can take a look at
 
 http://wiki.apache.org/hadoop/HowToUseSystemTestFramework
 
 its neatly explained
 
 On Fri, Apr 27, 2012 at 2:07 PM, 软件技术评估组_陈大雅 chend...@inspur.com wrote:
 
  Hi, I don't know how to use  Herriot !
  Could you like to give some procedure fo using Herriot to test my cluster?
 
  Thanks!!
 
 
 
   陈大雅
软件技术评估组
浪潮(北京)电子信息产业有限公司
 
地址:浪潮国家重点实验室(济南高新区新泺大街1766号齐鲁软件园大厦B座3层)
邮编:250101
电话(济南):86-531-85106428
手机:18954527190
Mail: chend...@inspur.com
 
 
 
 
 -- 
 Nitin Pawar


signature.asc
Description: Digital signature


[jira] [Resolved] (HADOOP-6205) Project java code has a significant number of declaration javadoc warnings and errors

2012-03-19 Thread Konstantin Boudnik (Resolved) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik resolved HADOOP-6205.


Resolution: Won't Fix

It seems to non-relevant anymore.

 Project java code has a significant number of declaration javadoc warnings 
 and errors
 -

 Key: HADOOP-6205
 URL: https://issues.apache.org/jira/browse/HADOOP-6205
 Project: Hadoop Common
  Issue Type: Improvement
Reporter: Konstantin Boudnik
Assignee: Konstantin Boudnik
 Attachments: Eclipse-JavaDoc-errors.png


 The issue here is that code inspections (say by IntelliJ IDEA) show that a 
 number of classes have javadoc problems in declarations. Some of those are 
 minor, e.g. missing @return, @param, @throws tags.
 Some of them are more severe, e.g. using non-existing parameter names in a 
 method javadoc (see HDFS' BlockManager.removeFromInvalidates(..) for an 
 example)
 I'd like to add more sophisticated code analysis to be able to fix at least 
 javadoc declaration errors in order to produce better code quality.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Resolved] (HADOOP-7157) Create build hosts configurations to prevent degradation of patch validation and snapshot build environment

2012-03-19 Thread Konstantin Boudnik (Resolved) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-7157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik resolved HADOOP-7157.


Resolution: Won't Fix

Seeminly not of interest to anyone.

 Create build hosts configurations to prevent degradation of patch validation 
 and snapshot build environment
 ---

 Key: HADOOP-7157
 URL: https://issues.apache.org/jira/browse/HADOOP-7157
 Project: Hadoop Common
  Issue Type: Improvement
  Components: build
 Environment: Apache Hadoop build hosts
Reporter: Konstantin Boudnik
Assignee: Konstantin Boudnik
 Attachments: HADOOP-7157.patch, HADOOP-7157.patch


 Build hosts are being re-jumped, in need for configuration updates, etc. It 
 is hard to maintain the same configuration across a 10+ hosts. A specialized 
 service such as Puppet can be used to maintain build machines software and 
 configurations at a desired level.
 Such configs should be checked into SCM system along with the of sources code.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Resolved] (HADOOP-7278) Automatic Hadoop cluster deployment for build validation

2012-03-19 Thread Konstantin Boudnik (Resolved) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-7278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik resolved HADOOP-7278.


Resolution: Won't Fix

Seeminly not of interest to anyone. Closing.

 Automatic Hadoop cluster deployment for build validation
 

 Key: HADOOP-7278
 URL: https://issues.apache.org/jira/browse/HADOOP-7278
 Project: Hadoop Common
  Issue Type: Improvement
  Components: build
Affects Versions: 0.22.0, 0.23.0
 Environment: Apache Jenkins
Reporter: Konstantin Boudnik
Assignee: Konstantin Boudnik
  Labels: newbie
 Attachments: HADOOP-7278.patch


 It'd be great to have a way of automatically deploying Hadoop cluster to a 
 set of machine once all components are successfully built (in the form or tar 
 or whatever). Deployed cluster then can be used to run a set of validation 
 jobs to make sure that produced artifacts are viable. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




Re: [VOTE] Hadoop-1.0.0 release candidate 3

2011-12-16 Thread Konstantin Boudnik
On Fri, Dec 16, 2011 at 12:10PM, Matt Foley wrote:
 Hello all,
 I have posted a new release candidate for Hadoop 1.0.0 at
 http://people.apache.org/~mattf/hadoop-1.0.0-rc3/
 
 Please download, evaluate, and vote on this list.
 The artifacts have also been posted to the maven repo.
 The vote will close at 12:30pm PST on Friday 13 Dec.

So, the vote has been closed for three days? Ok then ;)

 There had been an issue raised regarding
 HADOOP-7929https://issues.apache.org/jira/browse/HADOOP-7929.
  This has been evaluated and the jira closed as invalid.  Thanks to Andrew
 and others involved for helping assure the quality of this release.
 
 As previously mentioned, use of JDK 1.6.0_26 or better is recommended for
 this release.
 
 Thank you,
 
  --Matt (Release Manager for 1.0.0, formerly known as 0.20.205.1)
 
 
 On Fri, Dec 16, 2011 at 12:32 AM, Matt Foley mfo...@hortonworks.com wrote:
 
  Dear all,
  I have a clean build of the below, that passes junit testing, and was
  about to post it, when I got email about the newly opened 
  HADOOP-7929https://issues.apache.org/jira/browse/HADOOP-7929 Port
  HADOOP-7070 to branch-1.  It appears this prevents secure HBase from
  working with secure Hadoop.  Since HBase support is a key element of 1.0.0,
  and it seems this will be fixable fairly promptly, I am going to wait
  another day for 1.0.0-rc3.
 
  Thanks for your patience,
  --Matt
 
 
  On Tue, Dec 13, 2011 at 6:17 PM, Matt Foley mfo...@hortonworks.comwrote:
 
  In various emails and jiras, I have been asked to incorporate the
  following additional patches in 1.0.0:
 
 - MAPREDUCE-3319 (Roman)
 - HADOOP-7903 (Arpit)
 - MAPREDUCE-3475 (Daryn)
 - HDFS-2589 (Daryn)
 - HADOOP jira about to be opened (Chris W.) for missing jackson
 dependency in hadoop pom.
 
  I have not heard of any other significant issues in this build.
  I will re-spin the release candidate tomorrow.
 
  Thanks,
  --Matt
 
  On Thu, Dec 8, 2011 at 11:57 AM, Matt Foley ma...@apache.org wrote:
 
  Hello all,
  I have posted a release candidate for Hadoop 1.0.0 at
  http://people.apache.org/~mattf/hadoop-1.0.0-rc2/
 
  Please download, evaluate, and vote on this list.
  The artifacts will be posted to the maven repo shortly.
  The vote will close at noon PST on Thursday 15 Dec.
 
  It may be of interest that it is important to use an appropriate version
  of Java.
  I lost several days due to instabilities apparently in Oracle JDK
  1.6.0_23, which
  caused about 30 junit test failures in contrib (streaming, schedulers,
  and gridmix).
  Switching to JDK 1.6.0_26 made the problems go away.
 
  The release artifacts in this hadoop-1.0.0-rc2 were build with
  JDK 1.6.0_26.
 
  Thank you,
  --Matt (Release Manager for 1.0.0, formerly known as 0.20.205.1)
 
 
 
 


Re: how to write hadoop client to test HDFS with fault injection

2011-11-29 Thread Konstantin Boudnik
Once you have an instrumented cluster you can run some end-to-end tests 
(TeraSort
has been suggested and it would do fine). The tricky part is to assemble all
moving parts of the Hadoop cluster together from three different projects (if
you are on .21+ branch) and will be a way easier for 0.20.+ based versions.

However, one fine point here is that your faults (implemented as aspects)
should be triggered some how. You can do it logically (say every 3rd attempt
to get new block) or via environment, or from the test application. 

The latter might be a most tricky part because you'll have to essentially
communicate between different JVM processes perhaps located on different
physical hosts if you're running in a real cluster environment.

To address these issues we have project Herriot which is based on byte code
injection (read aspectj) and allows for distributed fault injections in a real
cluster. System test framework adds a few APIs to the Hadoop daemons and allow
to call them via standard Hadoop RPC. 

You can find framework sources and tests code in src/aop/system.
There's a separate set of build targets which will help you instrument
everything you need (jar-test-system should do). More info is here 
http://is.gd/SBkjrC

One last point: Herriot isn't fully integrated in trunk's maven build as of 
today.

Hope it helps,
  Cos

 From: Hao Yang [mailto:hao.yang0...@gmail.com]
 Sent: Monday, November 28, 2011 9:44 PM
 To: common-dev@hadoop.apache.org
 Subject: how to write hadoop client to test HDFS with fault injection

 Hi, all:

 I am a graduate student, now working on HDFS fault injection course
 project. I used FI framework to create aspects, and hdfs-fi.jar was
 generated. How can I write Hadoop Client code to test the injection? Thank
 you.


 Thank you very much for your time.


 Best regards
 Hao Yang




Re: Need help to get subscribe

2011-11-20 Thread Konstantin Boudnik
You can start at wiki.apache.org/hadoop/HowToContribute

On Thu, Nov 17, 2011 at 10:58PM, SATYAJIT RAKSHIT wrote:
 Hi,
   I am a java developer and want to contribute to Hadoop Common project.
 Please let me if anything needed to get into the project as developer role.
 
 Thanks
 Satyajit


[jira] [Reopened] (HADOOP-7450) Bump jetty to 6.1.26

2011-10-29 Thread Konstantin Boudnik (Reopened) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-7450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik reopened HADOOP-7450:


  Assignee: Konstantin Boudnik  (was: Jitendra Nath Pandey)

This issue needs to be fixed in 0.22 separately.

 Bump jetty to 6.1.26
 

 Key: HADOOP-7450
 URL: https://issues.apache.org/jira/browse/HADOOP-7450
 Project: Hadoop Common
  Issue Type: Bug
Reporter: Jitendra Nath Pandey
Assignee: Konstantin Boudnik
 Fix For: 0.23.0

 Attachments: HADOOP-7450.1.patch, HADOOP-7450.2.patch, 
 HADOOP-7450._0.22.patch


 Bump the jetty version, as previous version has an issue that can cause it to 
 hang at startup.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Resolved] (HADOOP-7730) Allow TestCLI to be run against a cluster

2011-10-11 Thread Konstantin Boudnik (Resolved) (JIRA)

 [ 
https://issues.apache.org/jira/browse/HADOOP-7730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Boudnik resolved HADOOP-7730.


   Resolution: Fixed
Fix Version/s: 0.22.0
 Hadoop Flags: Reviewed

I have just committed it.

 Allow TestCLI to be run against a cluster
 -

 Key: HADOOP-7730
 URL: https://issues.apache.org/jira/browse/HADOOP-7730
 Project: Hadoop Common
  Issue Type: Test
  Components: test
Affects Versions: 0.22.0
Reporter: Konstantin Boudnik
Assignee: Konstantin Boudnik
 Fix For: 0.22.0

 Attachments: HADOOP-7730.patch


 Use the same CLI test to test cluster bits (see HDFS-1762 for more info)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




Re: kfs and hdfs

2011-10-10 Thread Konstantin Boudnik
Mike,

I am not speaking for Owen in any way and appreciate your sense of humor, but
the position he has expressed was (and hopefully remain) a stance of these
lists: they aren't a marketing materials distribution channels. These are dev.
mailing lists.

Cos

On Mon, Oct 10, 2011 at 01:59PM, Segel, Mike wrote:
 Owen,
 Are you still a bit touchy over Mike Olson's rebuttal to your blog? :-P
 triumph_insult_comic_dog(I kee-id, I kee-id) /triumph_insult_comic_dog
 
 -Original Message-
 From: Owen O'Malley [mailto:o...@hortonworks.com]
 Sent: Monday, October 10, 2011 1:14 PM
 To: common-dev@hadoop.apache.org
 Subject: Re: kfs and hdfs
 
 Ted,
Please keep the marketing of closed-source insecure filesystems such as 
 MapR (and FAT32 *laugh*) off of the Apache Hadoop lists.
 
 -- Owen
 


  1   2   >