Hello Jim,
Yes, I'll start a new discussion about branching. Could you please share
the complete address of dev@? Is it different than
"dev@impala.incubator.apache.org"?
Updates about upstream activity -
1. We are porting native-toolchain on Power as building dependencies on
Power was a major problem when we ported 2.6 Impala. Currently we are stuck
at building gcc of kudu/thirdparty. Valencia has a different thread running
for this problem.
2. For Contribution approval, we are in discussion with our legal team.
Thanks,
Nishidha
From: Jim Apple <jbap...@cloudera.com>
To: "dev@impala" <dev@impala.incubator.apache.org>, Nishidha
Panpaliya/Austin/Contr/IBM@IBMUS
Cc: Silvius Rus <s...@cloudera.com>, Sudarshan
Jagadale/Austin/Contr/IBM@IBMUS, Manish
Patil/Austin/Contr/IBM@IBMUS, Valencia
Serrao/Austin/Contr/IBM@IBMUS
Date: 09/30/2016 09:55 PM
Subject: Re: Contributions to Cloudera Impala
Any updates on this, Nishidha?
On Fri, Sep 16, 2016 at 10:43 AM, Jim Apple <jbap...@cloudera.com> wrote:
> Nishidha, have you thought about starting that discussion on dev@ about
> branching?
>
> On Wed, Aug 17, 2016 at 9:12 AM, Jim Apple <jbap...@cloudera.com> wrote:
>>
>> > I'm glad to tell you that we are able to build and test Impala on
Ubuntu
>> > linux ppc64le with the great support from the Cloudera Community.
>>
>> Excellent!
>>
>> > Our next action is to upstream all our changes to Cloudera Impala.
>>
>> Great!
>>
>> Cloudera has donated Impala to the Apache Software Foundation (aka
>> "ASF"). Cloudera now contributes to the project, and the project is
>> managed by the Impala community.
>>
>> > With
>> > this, our plan is to start building latest Impala on Power8 as we'd
been
>> > porting quite an old version (code from cdh5-trunk branch till 23rd
>> > March,
>> > 2016). Since then, I know there have been many many changes happened
>> > which
>> > are yet to be ported, specially kudu stuff.
>>
>> Yes, there have been many changes. One is that Impala is now hosted on
>> ASF-owned git. Please see
>>
>> https://cwiki.apache.org/confluence/display/IMPALA/How+to+switch+to
+Apache-hosted+git
>>
>> > We know we need CLA to be signed to start contributions. We have
>> > already
>> > initiated the process and hoping to get it done soon.
>>
>> I think the right thing to do here is use the Apache CLAs. See
>>
>> https://www.apache.org/licenses/cla-corporate.txt
>>
>> and
>>
>> https://www.apache.org/licenses/icla.txt
>>
>> > By the time we get CLA signed, we would start porting the changes
>> > done
>> > in last 5 months. So, I wanted to know which tag/branch should we
>> > take
>> > up for this.
>>
>> This is a question we could all discuss together, and it might end up
>> being a decision made by the Project Management Committee (PMC).
>>
>> This is a big question about how Apache Impala will evolve. Our bylaws
>> state:
>>
>> "Significant, pervasive features may be developed in a speculative
>> branch of the repository. The PMC may grant commit rights on the
>> branch to its consistent contributors for the duration of the
>> initiative. Branch committers are responsible for shepherding their
>> feature into an active release and do not cast binding votes or vetoes
>> in the project."
>>
>> So perhaps this should happen on a separate branch?
>>
>> One question the community should also consider, IMHO, is whether the
>> community will have sufficient resources to maintain a working ppc64le
>> codebase indefinitely into the future.
>>
>>
>> > Working on cdh5-trunk will put us into an unending loop of
>> > porting as it is being modified everyday. We are thinking to create
a
>> > branch from cdh5.8.0-release tag and start working on it. Please
>> > suggest
>> > us the best way to do this.
>>
>> Since Impala is now developed on Apache infrastructure, we have
>> switched branching schemas. Our main branch is now "master". We do not
>> have any release branches yet.
>>
>> > Verifying all the changes on x86 platforms ourself here will also
be
>> > time consuming and add potential delays in upstreaming. So, we were
>> > thinking if we can get a job on Cloudera's Continuous integration
>> > server
>> > which would simply fetch our branch and verify it on all the
>> > supported
>> > platforms and do all the required checks. I'm not sure if this is
>> > feasible but just a thought. Any other suggestions to foster this
>> > activity would be appreciated.
>>
>> We are working on making a publicly-available CI setup, but we aren't
done
>> yet.
>>
>> Do you have a CI setup and x86-64 machines that your CI workers can run
>> on?
>>
>> > For every Pull Request, what are the basic sanity tests required to
>> > be
>> > ensured? Do you test all BE, FE, End-to-End tests, Custom cluster
>> > tests?
>>
>> Patches are sent to gerrit for review. Before they are merged, all
>> tests must pass in "core" (but not "exhaustive") mode.
>>
>> ==========
>>
>> If I were in your shoes, I might take the following steps:
>>
>> 1. Start a discussion on dev@ about whether a new branch is the right
>> way to develop.
>>
>> 2. Work out long-term maintenance plans and commitments and CI plans
>>
>> 3. Do the arduous work of rebasing on a recent HEAD.
>
>
