proposal for a Spark 2.x/scala 2.11.x release. was: [Spark 2.x/scala 2.11.x release]

[Andrew Palumbo]

wrote this up very quickly last Fri as a proposal.  Forgot to add it to the 
title.  Have updated it a bit.


From: Pat Ferrel 
Sent: Saturday, March 3, 2018 11:51 AM
To: dev@mahout.apache.org
Subject: Re: Spark 2.x/scala 2.11.x release

LGTM


From: Andrew Palumbo  
Reply: dev@mahout.apache.org  
Date: March 2, 2018 at 3:38:38 PM
To: dev@mahout.apache.org  
Subject:  Re: Spark 2.x/scala 2.11.x release

I've started a gdoc for a plan.


https://docs.google.com/document/d/1A8aqUORPp83vWa6fSqhC2jxUKEbDWWQ2lzqZ1V0xHS0/edit?usp=sharing


Please add comments, criticision, and alternate plans on on the doc.


--andy


From: Andrew Palumbo 
Sent: Friday, March 2, 2018 6:11:53 PM
To: dev@mahout.apache.org
Subject: Re: Spark 2.x/scala 2.11.x release

Pat, could you explain what you mean by the "Real Problem"? I know that we
have a lot of problems, but in terms of this release, what is the major
blocker?

From: Pat Ferrel 
Sent: Friday, March 2, 2018 5:32:58 PM
To: Trevor Grant; dev@mahout.apache.org
Subject: Re: Spark 2.x/scala 2.11.x release

Scopt is so not an issue. None whatsoever. The problem is that drivers have
unmet runtime needs that are different than libs. Scopt has absolutely
nothing to do with this. It was from a false theory that there was no 2.11
version but it actually has 2.11, 2.12, 2.09, and according to D a native
version too.

Get on to the real problems and drop this non-problem. Anything that driver
needs but is not on the classpath will stop them at runtime.

Better to say that we would be closer to release if we dropped drivers.


From: Trevor Grant  
Reply: dev@mahout.apache.org  

Date: March 2, 2018 at 2:26:13 PM
To: Mahout Dev List  
Subject: Re: Spark 2.x/scala 2.11.x release

Supposedly. I hard coded all of the poms to Scala 2.11 (closed PR unmerged)
Pat was still having issues w sbt- but the only dependency that was on 2.10
according to maven was scopt. /shrug



On Mar 2, 2018 4:20 PM, "Andrew Palumbo"  wrote:

> So We could release as is if we can get the scopt issue out? Thats our
> final blocker?
>
> 
> From: Trevor Grant 
> Sent: Friday, March 2, 2018 5:15:35 PM
> To: Mahout Dev List
> Subject: Re: Spark 2.x/scala 2.11.x release
>
> The only "mess" is in the cli spark drivers, namely scopt.
>
> Get rid of the drivers/fix the scopt issue- we have no mess.
>
>
>
> On Mar 2, 2018 4:09 PM, "Pat Ferrel"  wrote:
>
> > BTW the mess master is in is why git flow was invented and why I asked
> that
> > the site be in a new repo so it could be on a separate release cycle.
We
> > perpetuate the mess because it’s always to hard to fix.
> >
> >
> > From: Andrew Palumbo  
> > Reply: dev@mahout.apache.org  <
> > dev@mahout.apache.org>
> > Date: March 2, 2018 at 1:54:51 PM
> > To: dev@mahout.apache.org   >
> > Subject: Re: Spark 2.x/scala 2.11.x release
> >
> > re: reverting master, shit. I forgot that the website is not on
> `asf-site`
> > anymore. Well we could just re-jigger it, and check out `website` from
> > features/multi-artifact-build-MAHOUT-20xx after we revert the rest of
> > master.
> >
> >
> > You're right, Trevor- I 'm just going through the commits, and there
are
> > things like
> > https://github.com/apache/mahout/commit/c17bee3c2705495b638d81ae2ad374
> > bf7494c3f3
> >
> >
> >
> > [https://avatars3.githubusercontent.com/u/5852441?s=200=4]<
> > https://github.com/apache/mahout/commit/c17bee3c2705495b638d81ae2ad374
> > bf7494c3f3>
> >
> >
> > MAHOUT-1988 Make Native Solvers Scala 2.11 Complient closes apache/ma…
·
> > apache/mahout@c17bee3<
> > https://github.com/apache/mahout/commit/c17bee3c2705495b638d81ae2ad374
> > bf7494c3f3>
> >
> > github.com
> > …hout#326
> >
> >
> >
> > (make Native Solvers Scala 2.11 compliant) and others peppered in, Post
> > 0.13.0. It still may be possible and not that hard, to cherrypick
> > everything after 0.13.0 that we want. But I see what you're saying
about
> it
> > not being completely simple.
> >
> >
> > As for Git-Flow. I dont really care. I use it in some projects and in
> > others i use GitHub-flow. (basically what we've been doing with merging
> > everything to master).
> >
> >
> > Though this exact problem that we have right now is why git-flow is
nice.
> > Lets separate the question of how we go forward, with what commit/repo
> > style, and First figure out how to back out what we have now, 

Re: Backlog - Reordered Top->Down

[Andrew Palumbo]

+1 thank you



From: Trevor Grant 
Sent: Tuesday, March 6, 2018 2:13:14 PM
To: Mahout Dev List
Subject: Re: Backlog - Reordered Top->Down

Sorry- forgot to respond when i saw this- yes thanks!!



On Tue, Mar 6, 2018 at 12:51 PM, Dmitriy Lyubimov  wrote:

> Thank you.
>
> On Sat, Mar 3, 2018 at 1:45 PM, André Santi Oliveira 
> wrote:
>
> > Things which were in the backlog were organized (top->down) using
> criteria:
> >
> >   - Fix Version(s)
> >   - Priority
> >   - Type
> >
> >
> > If you don't agree with the order of some things which are there, feel
> free
> > to move around but I would like to understand the why to myself be able
> to
> > organize better next time. We have a lot of things "not tagged" which I
> put
> > on the bottom of the backlog, but it does not means are less important.
> > just means I was not able to classify still.
> >
> > I still need to understand how do you classify things in terms of
> versions.
> > In my head I have the workflow which is things are being done then we tag
> > to say "it will be delivered in this version for sure", but it's just a
> > detail for now as I said I'm still understanding the whole process and
> how
> > do you work.
> >
> >
> > Cheers,
> > André
> >
>

Re: Backlog - Reordered Top->Down

[Trevor Grant]

Sorry- forgot to respond when i saw this- yes thanks!!



On Tue, Mar 6, 2018 at 12:51 PM, Dmitriy Lyubimov  wrote:

> Thank you.
>
> On Sat, Mar 3, 2018 at 1:45 PM, André Santi Oliveira 
> wrote:
>
> > Things which were in the backlog were organized (top->down) using
> criteria:
> >
> >   - Fix Version(s)
> >   - Priority
> >   - Type
> >
> >
> > If you don't agree with the order of some things which are there, feel
> free
> > to move around but I would like to understand the why to myself be able
> to
> > organize better next time. We have a lot of things "not tagged" which I
> put
> > on the bottom of the backlog, but it does not means are less important.
> > just means I was not able to classify still.
> >
> > I still need to understand how do you classify things in terms of
> versions.
> > In my head I have the workflow which is things are being done then we tag
> > to say "it will be delivered in this version for sure", but it's just a
> > detail for now as I said I'm still understanding the whole process and
> how
> > do you work.
> >
> >
> > Cheers,
> > André
> >
>

Re: Backlog - Reordered Top->Down

[Dmitriy Lyubimov]

Thank you.

On Sat, Mar 3, 2018 at 1:45 PM, André Santi Oliveira 
wrote:

> Things which were in the backlog were organized (top->down) using criteria:
>
>   - Fix Version(s)
>   - Priority
>   - Type
>
>
> If you don't agree with the order of some things which are there, feel free
> to move around but I would like to understand the why to myself be able to
> organize better next time. We have a lot of things "not tagged" which I put
> on the bottom of the backlog, but it does not means are less important.
> just means I was not able to classify still.
>
> I still need to understand how do you classify things in terms of versions.
> In my head I have the workflow which is things are being done then we tag
> to say "it will be delivered in this version for sure", but it's just a
> detail for now as I said I'm still understanding the whole process and how
> do you work.
>
>
> Cheers,
> André
>