I am not this afternoon. This week is still crazy busy for me, but hopefully will ease up after next weekend.
On Sun, Apr 19, 2020, 12:51 PM Andrew Musselman <andrew.mussel...@gmail.com> wrote: > I think it's safe to move to 1.8; yeah what tune is good for you? I'm in > Pacific time zone and am flexible this afternoon. > > Trevor you free? > > On Sun, Apr 19, 2020 at 09:37 Christofer Dutz <christofer.d...@c-ware.de> > wrote: > > > Yikes! ... well guess then I can't help you folks as it's almost > > impossible to get my hands on a 1.7 version. > > > > What's preventing you from going to 1.8+? > > > > Any your build says 1.8 and above: > > <requireJavaVersion> > > <version>[1.8,)</version> > > </requireJavaVersion> > > > > Regarding the artifacts ... would it be ok to have the maven artifacts > > using classifiers and perhaps the files in the libs and distribution to > > follow the typical scala sheme? > > > > Chris > > > > Am 19.04.20, 17:56 schrieb "Trevor Grant" <trevor.d.gr...@gmail.com>: > > > > Yea, we have a requirement on 1.7. we need to get it up to 8, but > > considered that a different issue. > > > > Maven throws warning, sbt breaks down entirely (when importing)- > hence > > why > > we were using a script to replace 2.11 w 2.12 > > > > On Sun, Apr 19, 2020, 10:33 AM Christofer Dutz < > > christofer.d...@c-ware.de> > > wrote: > > > > > Well I have been compiling with various jdks from 8 to 14. However > I > > > noticed that if I select a jdk and start the build that the Scala > > parts > > > Seefahrt to the version in my systems path. Which then results in > > all sorts > > > of major version problems. > > > > > > For the Scalatest plugin I managed to have it use the maven > > java.home... > > > The Scala compiler is being a little trickyer. > > > > > > Will continue this evening. Keep your fingers crossed. > > > > > > Also does maven totally not like variables in artifactids. > > > > > > Chris > > > ________________________________ > > > Von: Trevor Grant <trevor.d.gr...@gmail.com> > > > Gesendet: Sonntag, 19. April 2020 15:41 > > > An: Mahout Dev List <dev@mahout.apache.org> > > > Betreff: Re: Hi ... need some help? > > > > > > Hey Chris, > > > > > > First of all- thank you so much for your work so far. > > > > > > The build 100% currently works, the only issue is with releasing > > binaries. > > > Are you compiling w 1.7? Not really sure why artifact IDs are > > changing? > > > That's very strange. > > > > > > To answer your question regarding the community block- no, it is > not > > as if > > > there is a company that pays us to work on Mahout and the rest is > > > community. What it means is that while we appreciate those code > > > contributions, no one still actively involved is maintaining them > > and we > > > advise people to use at their own risk. The refactor to a > `community` > > > folder was meant to reflect the reality of the situation. > > > > > > The only test issues I was aware of were with Javadocs when trying > > to do a > > > release with Java 1.8+ those would fail due to formatting. > > > > > > I feel like you're hitting some unique issues, and maybe we could > > help you > > > out / there may exist simple solutions. > > > > > > Thank you again for your help! > > > > > > tg > > > > > > > > > > > > > > > On Sun, Apr 19, 2020 at 2:17 AM Christofer Dutz < > > christofer.d...@c-ware.de > > > > > > > wrote: > > > > > > > Hi folks, > > > > > > > > So yesterday I invested several hours in cleaning up your build. > I > > got > > > > quite far, currently trying to get the tests to pass. For some > > > > Scala-Java-major-version problems, but working on fixing them. > > > > > > > > However sind things will be different. I hope that's OK. > > Especially with > > > > the artifact ids. > > > > > > > > Just as a question: does the build currently work at all? > > Especially in > > > > the community block I had to fix quite a few API changes where > the > > code > > > in > > > > the blocks were for older versions of libraries. > > > > > > > > I hope I get some more tests to pass today. > > > > > > > > Chris > > > > ________________________________ > > > > Von: Andrew Musselman <a...@apache.org> > > > > Gesendet: Donnerstag, 16. April 2020 20:35 > > > > An: Mahout Dev List <dev@mahout.apache.org> > > > > Betreff: Re: Hi ... need some help? > > > > > > > > Looking forward to working on this with you; thanks again! > > > > > > > > On Thu, Apr 16, 2020 at 11:09 AM Christofer Dutz < > > > > christofer.d...@c-ware.de> > > > > wrote: > > > > > > > > > Hi Andrew, > > > > > > > > > > guess I'll start with the fork and contact you folks on slack. > > > > > > > > > > Chris > > > > > > > > > > > > > > > > > > > > Am 16.04.20, 19:43 schrieb "Andrew Musselman" <a...@apache.org > >: > > > > > > > > > > Chris, thank you for your help.. > > > > > > > > > > Yeah if you fork what's in master you can see what state > > it's in; > > > we > > > > > are in > > > > > the #mahout channel in tge-asf slack and this is also a > fine > > way to > > > > > keep > > > > > track of discussion. > > > > > > > > > > We could file a JIRA ticket as well, however you prefer to > > work. > > > > > > > > > > Best > > > > > Andrew > > > > > > > > > > On Thu, Apr 16, 2020 at 06:59 Christofer Dutz < > > > > > christofer.d...@c-ware.de> > > > > > wrote: > > > > > > > > > > > Hi Trevor, > > > > > > > > > > > > ok ... first of all ... the Mahout PMC is defining a > > "community > > > > > > maintained" library which is not maintained by the mahout > > PMC?!?! > > > > > > I thought at Apache everything is about Community over > > code. So > > > is > > > > a > > > > > > company driving the non-community stuff? > > > > > > > > > > > > But back to your build issues: > > > > > > I had a look and I too encountered these comments and > > remarks and > > > > > > sometimes patterns I recognized and could imagine why > they > > were > > > > > created. > > > > > > Yes quite a bit of the build could be cleaned up and > > simplified a > > > > > lot. > > > > > > > > > > > > So how about I create a fork and try to do a cleanup of > the > > > build. > > > > > > Usually I also leave comments about what I do as I hope > > I'll not > > > be > > > > > the > > > > > > only one maintaining a build and documenting things helps > > people > > > > > feel more > > > > > > confident. > > > > > > > > > > > > However in some cases I will have questions ... so would > > someone > > > be > > > > > > available on Slack for quick questions? > > > > > > > > > > > > Usually switching to another build system does solve some > > > problems > > > > > ... > > > > > > mostly the reason to switch is that it solved the main > > problem > > > that > > > > > you are > > > > > > having with the old. > > > > > > However you usually notice too late that you get yourself > > a lot > > > of > > > > > new > > > > > > problems. I remember doing some contract work for an > > insurance > > > > > company and > > > > > > they were totally down Maven-road but then had to build > > something > > > > > with SBT > > > > > > ... in the end I compiled the thing on my laptop, copied > > it to a > > > > USB > > > > > stick > > > > > > and told the people what was on the stick and that I'll > be > > > having a > > > > > coffee > > > > > > and will be back in 30 minutes. When I came back the sick > > wasn't > > > at > > > > > the > > > > > > same place and the build problem was "solved" ;-) > > > > > > > > > > > > So I think it's quite good to stick to maven ... that is > > very > > > > > mature, you > > > > > > can do almost everything you want with it and it > integrates > > > > > perfectly into > > > > > > the Apache infrastructure. > > > > > > > > > > > > But that's just my opinion. > > > > > > > > > > > > So if you want me to help, I'll be happy to be of > > assistance. > > > > > > > > > > > > > > > > > > Chris > > > > > > > > > > > > > > > > > > > > > > > > Am 16.04.20, 15:28 schrieb "Trevor Grant" < > > > > trevor.d.gr...@gmail.com > > > > > >: > > > > > > > > > > > > Hey Christopher, > > > > > > > > > > > > I would agree with what Stevo outlined but add some > > more > > > > context > > > > > and a > > > > > > couple related JIRA issues. > > > > > > > > > > > > For 0.14.0 We did a big refactor and finally moved > the > > > > MapReduce > > > > > based > > > > > > Mahout all into what we called "community/" that is > > community > > > > > > maintained, > > > > > > which is to say, we're not maintaining it anymore > > (sunset > > > > began I > > > > > > think in > > > > > > 2015). > > > > > > > > > > > > But all of our POMs were so huge and fat because > > they'd been > > > > > layered up > > > > > > over the years by people coming and going and > dropping > > in > > > > code. I > > > > > > wouldn't > > > > > > call these drive- bys, its just been over 10 years > and > > people > > > > > come and > > > > > > go. > > > > > > Such is the life of Apache Projects. So we had a > > situation > > > > where > > > > > a lot > > > > > > of > > > > > > the old Map Reduce stuff and the POMs were considered > > > > > "old-magic" no > > > > > > one > > > > > > really knew how it was all tied together, but we > > didn't want > > > to > > > > > mess > > > > > > with > > > > > > it for fear of breaking something in the "new" Mahout > > (aka > > > > > Samsar) > > > > > > which is > > > > > > the Scala/Spark based library that it is now* (to > > others in > > > the > > > > > > community: > > > > > > I know it runs on other engines, but for simplicity, > > I'm just > > > > > calling > > > > > > it > > > > > > "runs-on-spark"). > > > > > > > > > > > > For 0.14.0 We decided to trim out as much of that > > which was > > > > > possible. > > > > > > We > > > > > > did some major liposuction on POMs, re organized > > things, etc. > > > > > This was > > > > > > done > > > > > > by commenting out a section, then seeing if it would > > still > > > > > build. So > > > > > > the > > > > > > current release > > > > > > _does_ build. And aside for some CLI driver issues > > which are > > > > > outlined > > > > > > in > > > > > > [1], the project runs fairly smooth. (An SBT would > > probably > > > > > solve [1], > > > > > > I > > > > > > believe Pat Ferrel has made his own SBT script to > > compile > > > > > Mahout, which > > > > > > solved that problem for them). > > > > > > > > > > > > The issue we ran into with the releases (and the > > reason I > > > think > > > > > you're > > > > > > here), is that we also somewhere along the line > > commented out > > > > > something > > > > > > that was important to the release process. Hence why > > 0.14.0 > > > > > released > > > > > > source > > > > > > only. > > > > > > > > > > > > Since 2008, there has been a lot of great work on > > generating > > > > > plugins > > > > > > for > > > > > > doing Apache releases. Instead of the awkward hacks > > that made > > > > up > > > > > the > > > > > > old > > > > > > poms (literally comments that said, "this is a hack, > > there's > > > > > supposedly > > > > > > something better coming from ..." dated like 2012), > we > > would > > > > > like to > > > > > > do it > > > > > > the "right way" and incorporate the appropriate > > plugins. > > > > > > > > > > > > Refactoring to SBT was _one_ proposed solution. We're > > also OK > > > > > > continuing to > > > > > > use Maven, and I agree with what you said about the > > cross > > > > > compiling. We > > > > > > actually have a script that just changes the scala > > version. > > > We > > > > > tried > > > > > > using > > > > > > the classifiers but there were issues in SBT, but the > > way > > > > you're > > > > > > proposing > > > > > > sounds a lot more pro than the route we were trying > > for. > > > > > > > > > > > > That said- we'd be OK just releasing one scala/spark > > version > > > at > > > > > a time. > > > > > > But getting the convenience binaries to > > release/publish would > > > > be > > > > > a > > > > > > major > > > > > > first step. > > > > > > > > > > > > Also, we really appreciate the help, > > > > > > > > > > > > tg > > > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > https://issues.apache.org/jira/projects/MAHOUT/issues/MAHOUT-2093?filter=allopenissues > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Apr 16, 2020 at 4:50 AM Christofer Dutz < > > > > > > christofer.d...@c-ware.de> > > > > > > wrote: > > > > > > > > > > > > > Hi Stevo, > > > > > > > > > > > > > > so let me summarize what I understood: > > > > > > > > > > > > > > - There are some modules in mahout that are built > > with > > > Scala, > > > > > some > > > > > > with > > > > > > > java and some with both (At least that's what I see > > when > > > > > checking > > > > > > out the > > > > > > > project) > > > > > > > - The current build uses Scala 2.11 to build the > > Scala > > > code. > > > > > > > - The resulting libraries are only compatible with > > Scala > > > 2.11 > > > > > > > > > > > > > > Now you want to also publish versions compatible > > with Scala > > > > > 2.12? > > > > > > > > > > > > > > If that's the case I think Maven could easily add > > multiple > > > > > executions > > > > > > > where each compile compiles to different output > > > directories: > > > > > > > - Java --> target/classes > > > > > > > - Scala 2.11 --> target/classes-2.11 > > > > > > > - Scala 2.12 --> target/classes-2.12 > > > > > > > > > > > > > > Then the packaging would also need a second > > execution ... > > > > each > > > > > of the > > > > > > > executions bundling the classes and the > > corresponding scala > > > > > output. > > > > > > > Ideally I would probably use maven classifiers to > > > distinguish > > > > > the > > > > > > > artifacts. > > > > > > > > > > > > > > <dependency> > > > > > > > <groupId>org.apache.mahout</groupId> > > > > > > > <artifactId>mahout-spark</artifactId> > > > > > > > <version>14.1-SNAPSHOT</version> > > > > > > > <classifier>2.11</classifier> > > > > > > > </dependency> > > > > > > > > > > > > > > Then it should all work in a normal maven build. In > > the > > > > > > distributions you > > > > > > > could also filter the versions according to their > > > > classifiers. > > > > > > > > > > > > > > So if this is the case, I could help you with this. > > > > > > > > > > > > > > Chris > > > > > > > > > > > > > > > > > > > > > Am 16.04.20, 09:39 schrieb "Stevo Slavić" < > > > ssla...@gmail.com > > > > >: > > > > > > > > > > > > > > Disclaimer: I'm not active Mahout maintainer > for > > quite > > > a > > > > > while, > > > > > > have > > > > > > > some > > > > > > > historical perspective, take it with a grain of > > salt, > > > > > could be > > > > > > I'm > > > > > > > missing > > > > > > > the whole point you were approached for by a > wide > > > margin > > > > of > > > > > > error. > > > > > > > > > > > > > > At a point Mahout, some of its modules, have > > turned > > > into > > > > a > > > > > scala > > > > > > > library, and there was need to cross publish > > those > > > > modules, > > > > > > across > > > > > > > different scala versions. Back than Maven scala > > plugin > > > > > didn't > > > > > > support > > > > > > > cross > > > > > > > publishing, it doesn't fit well with Maven's > > build > > > > > lifecycle > > > > > > concept > > > > > > > (multiple compile phases - one for each scala > > version, > > > > and > > > > > what > > > > > > not > > > > > > > would > > > > > > > be needed). Switching to sbt could have solved > > the > > > > problem. > > > > > > Switch was > > > > > > > deemed to be too big task, even though ages > have > > been > > > > > spent on > > > > > > trying > > > > > > > to > > > > > > > apply Maven (profiles) + bash scripts and what > > not to > > > > > solve the > > > > > > > problem. > > > > > > > Trying to apply same approach over and over > > again and > > > > > expecting > > > > > > > different > > > > > > > results is not smart, no expert can help there. > > Mahout > > > > > > maintainers and > > > > > > > contributors, should consider alternative > > approach, one > > > > of > > > > > them > > > > > > being > > > > > > > switching to sbt - it's scala native, supports > > scala > > > > cross > > > > > > publishing, > > > > > > > supports publishing Maven compatible release > > metadata > > > and > > > > > > binaries. > > > > > > > > > > > > > > Kind regards, > > > > > > > Stevo Slavic. > > > > > > > > > > > > > > On Thu, Apr 16, 2020 at 9:15 AM Christofer > Dutz < > > > > > > > christofer.d...@c-ware.de> > > > > > > > wrote: > > > > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > > > my name is Chris and I’m involved in quite a > > lot of > > > > > Apache > > > > > > projects. > > > > > > > > Justin approached me this morning, asking me > > if I > > > could > > > > > > perhaps help > > > > > > > you. > > > > > > > > He told me you were having trouble with doing > > Maven > > > > > releases. > > > > > > > > > > > > > > > > As Maven releases are my specialty, could you > > please > > > > > summarize > > > > > > the > > > > > > > issues > > > > > > > > you are having? > > > > > > > > > > > > > > > > Chris > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
