Re: Spark 2.x/scala 2.11.x release

2018-03-02 Thread Andrew Palumbo
I've started a gdoc for a plan. https://docs.google.com/document/d/1A8aqUORPp83vWa6fSqhC2jxUKEbDWWQ2lzqZ1V0xHS0/edit?usp=sharing Please add comments, criticision, and alternate plans on on the doc. --andy From: Andrew Palumbo Sent: Friday, March 2, 2018 6:11

Re: Spark 2.x/scala 2.11.x release

2018-03-02 Thread Andrew Palumbo
Pat, could you explain what you mean by the "Real Problem"? I know that we have a lot of problems, but in terms of this release, what is the major blocker? From: Pat Ferrel Sent: Friday, March 2, 2018 5:32:58 PM To: Trevor Grant; dev@mahout.apache.org Subject: Re

Re: Spark 2.x/scala 2.11.x release

2018-03-02 Thread Pat Ferrel
Scopt is so not an issue. None whatsoever. The problem is that drivers have unmet runtime needs that are different than libs. Scopt has absolutely nothing to do with this. It was from a false theory that there was no 2.11 version but it actually has 2.11, 2.12, 2.09, and according to D a native ver

Re: Spark 2.x/scala 2.11.x release

2018-03-02 Thread Trevor Grant
Supposedly. I hard coded all of the poms to Scala 2.11 (closed PR unmerged) Pat was still having issues w sbt- but the only dependency that was on 2.10 according to maven was scopt. /shrug On Mar 2, 2018 4:20 PM, "Andrew Palumbo" wrote: > So We could release as is if we can get the scopt issue

Re: Spark 2.x/scala 2.11.x release

2018-03-02 Thread Andrew Palumbo
So We could release as is if we can get the scopt issue out? Thats our final blocker? From: Trevor Grant Sent: Friday, March 2, 2018 5:15:35 PM To: Mahout Dev List Subject: Re: Spark 2.x/scala 2.11.x release The only "mess" is in the cli spark drivers, namely s

Re: Spark 2.x/scala 2.11.x release

2018-03-02 Thread Trevor Grant
The only "mess" is in the cli spark drivers, namely scopt. Get rid of the drivers/fix the scopt issue- we have no mess. On Mar 2, 2018 4:09 PM, "Pat Ferrel" wrote: > BTW the mess master is in is why git flow was invented and why I asked that > the site be in a new repo so it could be on a sep

Re: Spark 2.x/scala 2.11.x release

2018-03-02 Thread Pat Ferrel
BTW the mess master is in is why git flow was invented and why I asked that the site be in a new repo so it could be on a separate release cycle. We perpetuate the mess because it’s always to hard to fix. From: Andrew Palumbo Reply: dev@mahout.apache.org Date: March 2, 2018 at 1:54:51 PM To:

Re: Spark 2.x/scala 2.11.x release

2018-03-02 Thread Andrew Palumbo
re: reverting master, shit. I forgot that the website is not on `asf-site` anymore. Well we could just re-jigger it, and check out `website` from features/multi-artifact-build-MAHOUT-20xx after we revert the rest of master. You're right, Trevor- I 'm just going through the commits, and the

Re: Spark 2.x/scala 2.11.x release

2018-03-02 Thread Trevor Grant
In addition to numerous other problems- its not 'just that simple' Git has to be setup and replumbed to close PRs based on merge to develop, as well as making that the default for new PRs. IIRC we tried doing this last year- it was a mess, we went back. We're not doing _that_ much work on master

Re: Spark 2.x/scala 2.11.x release

2018-03-02 Thread Trevor Grant
If you revert master to the release tag you're going to destroy the website. The website pulls and rebuilds from mater whenever Jenkins detects a change. mahout-0.13.0 has no website. So it will pull nothing and there will be no site. tg On Fri, Mar 2, 2018 at 1:24 PM, Andrew Palumbo wrote:

Re: Spark 2.x/scala 2.11.x release

2018-03-02 Thread Pat Ferrel
Trevor said -1 to git flow. I agree this is a good example of one nice feature. Master is always pristine. From: Andrew Palumbo Reply: dev@mahout.apache.org Date: March 2, 2018 at 12:30:09 PM To: dev@mahout.apache.org Subject: Re: Spark 2.x/scala 2.11.x release Pat - Not sure if you thou

Re: Spark 2.x/scala 2.11.x release

2018-03-02 Thread Andrew Palumbo
oh I see that I missed Trevor's -1 on git-flow. From: Andrew Palumbo Sent: Friday, March 2, 2018 3:29:54 PM To: dev@mahout.apache.org Subject: Re: Spark 2.x/scala 2.11.x release Pat - Not sure if you thought I meant -1 or you're saying -1 to `git-flow`. I'm +1

Re: Spark 2.x/scala 2.11.x release

2018-03-02 Thread Andrew Palumbo
Pat - Not sure if you thought I meant -1 or you're saying -1 to `git-flow`. I'm +1 on it, and this is a perfect time to make the transition, since we don't have a `develop` branch, now we can revert `master` to its state at the 0.13.0 release, create a `develop` at the same time, branched from t

Re: Spark 2.x/scala 2.11.x release

2018-03-02 Thread Pat Ferrel
-1 on git flow? What are your reasons? I use it in 3-4 project, one of which is Apache PIO. I’d think this would be a good example of why it’s nice. right now our master is screwed up, with git flow it would still have 0.13.0, which is what we are talking about releasing with minor mods. From: T

Re: Spark 2.x/scala 2.11.x release

2018-03-02 Thread Trevor Grant
I'm +1 for a working scala-2.11, spark-2.x build. I'm -1 for previously stated reasons on git-flow. tg On Fri, Mar 2, 2018 at 1:24 PM, Andrew Palumbo wrote: > > Sounds Good. I'll put out a proposal for the release, and we can go Over > it and vote if we want to on releasing or on the scope.

Spark 2.x/scala 2.11.x release

2018-03-02 Thread Andrew Palumbo
Sounds Good. I'll put out a proposal for the release, and we can go Over it and vote if we want to on releasing or on the scope. I'm +1 on it. Broad strokes of what I'm thinking: - Checkout a new branch "features/multi-artifact-build-22xx" from master @ the `mahout-0.13.0` release tag. -