*what time On Sun, Apr 19, 2020 at 10:51 Andrew Musselman <andrew.mussel...@gmail.com> wrote:
> I think it's safe to move to 1.8; yeah what tune is good for you? I'm in > Pacific time zone and am flexible this afternoon. > > Trevor you free? > > On Sun, Apr 19, 2020 at 09:37 Christofer Dutz <christofer.d...@c-ware.de> > wrote: > >> Yikes! ... well guess then I can't help you folks as it's almost >> impossible to get my hands on a 1.7 version. >> >> What's preventing you from going to 1.8+? >> >> Any your build says 1.8 and above: >> <requireJavaVersion> >> <version>[1.8,)</version> >> </requireJavaVersion> >> >> Regarding the artifacts ... would it be ok to have the maven artifacts >> using classifiers and perhaps the files in the libs and distribution to >> follow the typical scala sheme? >> >> Chris >> >> Am 19.04.20, 17:56 schrieb "Trevor Grant" <trevor.d.gr...@gmail.com>: >> >> Yea, we have a requirement on 1.7. we need to get it up to 8, but >> considered that a different issue. >> >> Maven throws warning, sbt breaks down entirely (when importing)- >> hence why >> we were using a script to replace 2.11 w 2.12 >> >> On Sun, Apr 19, 2020, 10:33 AM Christofer Dutz < >> christofer.d...@c-ware.de> >> wrote: >> >> > Well I have been compiling with various jdks from 8 to 14. However I >> > noticed that if I select a jdk and start the build that the Scala >> parts >> > Seefahrt to the version in my systems path. Which then results in >> all sorts >> > of major version problems. >> > >> > For the Scalatest plugin I managed to have it use the maven >> java.home... >> > The Scala compiler is being a little trickyer. >> > >> > Will continue this evening. Keep your fingers crossed. >> > >> > Also does maven totally not like variables in artifactids. >> > >> > Chris >> > ________________________________ >> > Von: Trevor Grant <trevor.d.gr...@gmail.com> >> > Gesendet: Sonntag, 19. April 2020 15:41 >> > An: Mahout Dev List <dev@mahout.apache.org> >> > Betreff: Re: Hi ... need some help? >> > >> > Hey Chris, >> > >> > First of all- thank you so much for your work so far. >> > >> > The build 100% currently works, the only issue is with releasing >> binaries. >> > Are you compiling w 1.7? Not really sure why artifact IDs are >> changing? >> > That's very strange. >> > >> > To answer your question regarding the community block- no, it is >> not as if >> > there is a company that pays us to work on Mahout and the rest is >> > community. What it means is that while we appreciate those code >> > contributions, no one still actively involved is maintaining them >> and we >> > advise people to use at their own risk. The refactor to a >> `community` >> > folder was meant to reflect the reality of the situation. >> > >> > The only test issues I was aware of were with Javadocs when trying >> to do a >> > release with Java 1.8+ those would fail due to formatting. >> > >> > I feel like you're hitting some unique issues, and maybe we could >> help you >> > out / there may exist simple solutions. >> > >> > Thank you again for your help! >> > >> > tg >> > >> > >> > >> > >> > On Sun, Apr 19, 2020 at 2:17 AM Christofer Dutz < >> christofer.d...@c-ware.de >> > > >> > wrote: >> > >> > > Hi folks, >> > > >> > > So yesterday I invested several hours in cleaning up your build. >> I got >> > > quite far, currently trying to get the tests to pass. For some >> > > Scala-Java-major-version problems, but working on fixing them. >> > > >> > > However sind things will be different. I hope that's OK. >> Especially with >> > > the artifact ids. >> > > >> > > Just as a question: does the build currently work at all? >> Especially in >> > > the community block I had to fix quite a few API changes where >> the code >> > in >> > > the blocks were for older versions of libraries. >> > > >> > > I hope I get some more tests to pass today. >> > > >> > > Chris >> > > ________________________________ >> > > Von: Andrew Musselman <a...@apache.org> >> > > Gesendet: Donnerstag, 16. April 2020 20:35 >> > > An: Mahout Dev List <dev@mahout.apache.org> >> > > Betreff: Re: Hi ... need some help? >> > > >> > > Looking forward to working on this with you; thanks again! >> > > >> > > On Thu, Apr 16, 2020 at 11:09 AM Christofer Dutz < >> > > christofer.d...@c-ware.de> >> > > wrote: >> > > >> > > > Hi Andrew, >> > > > >> > > > guess I'll start with the fork and contact you folks on slack. >> > > > >> > > > Chris >> > > > >> > > > >> > > > >> > > > Am 16.04.20, 19:43 schrieb "Andrew Musselman" <a...@apache.org>: >> > > > >> > > > Chris, thank you for your help.. >> > > > >> > > > Yeah if you fork what's in master you can see what state >> it's in; >> > we >> > > > are in >> > > > the #mahout channel in tge-asf slack and this is also a >> fine way to >> > > > keep >> > > > track of discussion. >> > > > >> > > > We could file a JIRA ticket as well, however you prefer to >> work. >> > > > >> > > > Best >> > > > Andrew >> > > > >> > > > On Thu, Apr 16, 2020 at 06:59 Christofer Dutz < >> > > > christofer.d...@c-ware.de> >> > > > wrote: >> > > > >> > > > > Hi Trevor, >> > > > > >> > > > > ok ... first of all ... the Mahout PMC is defining a >> "community >> > > > > maintained" library which is not maintained by the mahout >> PMC?!?! >> > > > > I thought at Apache everything is about Community over >> code. So >> > is >> > > a >> > > > > company driving the non-community stuff? >> > > > > >> > > > > But back to your build issues: >> > > > > I had a look and I too encountered these comments and >> remarks and >> > > > > sometimes patterns I recognized and could imagine why >> they were >> > > > created. >> > > > > Yes quite a bit of the build could be cleaned up and >> simplified a >> > > > lot. >> > > > > >> > > > > So how about I create a fork and try to do a cleanup of >> the >> > build. >> > > > > Usually I also leave comments about what I do as I hope >> I'll not >> > be >> > > > the >> > > > > only one maintaining a build and documenting things helps >> people >> > > > feel more >> > > > > confident. >> > > > > >> > > > > However in some cases I will have questions ... so would >> someone >> > be >> > > > > available on Slack for quick questions? >> > > > > >> > > > > Usually switching to another build system does solve some >> > problems >> > > > ... >> > > > > mostly the reason to switch is that it solved the main >> problem >> > that >> > > > you are >> > > > > having with the old. >> > > > > However you usually notice too late that you get yourself >> a lot >> > of >> > > > new >> > > > > problems. I remember doing some contract work for an >> insurance >> > > > company and >> > > > > they were totally down Maven-road but then had to build >> something >> > > > with SBT >> > > > > ... in the end I compiled the thing on my laptop, copied >> it to a >> > > USB >> > > > stick >> > > > > and told the people what was on the stick and that I'll be >> > having a >> > > > coffee >> > > > > and will be back in 30 minutes. When I came back the sick >> wasn't >> > at >> > > > the >> > > > > same place and the build problem was "solved" ;-) >> > > > > >> > > > > So I think it's quite good to stick to maven ... that is >> very >> > > > mature, you >> > > > > can do almost everything you want with it and it >> integrates >> > > > perfectly into >> > > > > the Apache infrastructure. >> > > > > >> > > > > But that's just my opinion. >> > > > > >> > > > > So if you want me to help, I'll be happy to be of >> assistance. >> > > > > >> > > > > >> > > > > Chris >> > > > > >> > > > > >> > > > > >> > > > > Am 16.04.20, 15:28 schrieb "Trevor Grant" < >> > > trevor.d.gr...@gmail.com >> > > > >: >> > > > > >> > > > > Hey Christopher, >> > > > > >> > > > > I would agree with what Stevo outlined but add some >> more >> > > context >> > > > and a >> > > > > couple related JIRA issues. >> > > > > >> > > > > For 0.14.0 We did a big refactor and finally moved the >> > > MapReduce >> > > > based >> > > > > Mahout all into what we called "community/" that is >> community >> > > > > maintained, >> > > > > which is to say, we're not maintaining it anymore >> (sunset >> > > began I >> > > > > think in >> > > > > 2015). >> > > > > >> > > > > But all of our POMs were so huge and fat because >> they'd been >> > > > layered up >> > > > > over the years by people coming and going and >> dropping in >> > > code. I >> > > > > wouldn't >> > > > > call these drive- bys, its just been over 10 years >> and people >> > > > come and >> > > > > go. >> > > > > Such is the life of Apache Projects. So we had a >> situation >> > > where >> > > > a lot >> > > > > of >> > > > > the old Map Reduce stuff and the POMs were considered >> > > > "old-magic" no >> > > > > one >> > > > > really knew how it was all tied together, but we >> didn't want >> > to >> > > > mess >> > > > > with >> > > > > it for fear of breaking something in the "new" Mahout >> (aka >> > > > Samsar) >> > > > > which is >> > > > > the Scala/Spark based library that it is now* (to >> others in >> > the >> > > > > community: >> > > > > I know it runs on other engines, but for simplicity, >> I'm just >> > > > calling >> > > > > it >> > > > > "runs-on-spark"). >> > > > > >> > > > > For 0.14.0 We decided to trim out as much of that >> which was >> > > > possible. >> > > > > We >> > > > > did some major liposuction on POMs, re organized >> things, etc. >> > > > This was >> > > > > done >> > > > > by commenting out a section, then seeing if it would >> still >> > > > build. So >> > > > > the >> > > > > current release >> > > > > _does_ build. And aside for some CLI driver issues >> which are >> > > > outlined >> > > > > in >> > > > > [1], the project runs fairly smooth. (An SBT would >> probably >> > > > solve [1], >> > > > > I >> > > > > believe Pat Ferrel has made his own SBT script to >> compile >> > > > Mahout, which >> > > > > solved that problem for them). >> > > > > >> > > > > The issue we ran into with the releases (and the >> reason I >> > think >> > > > you're >> > > > > here), is that we also somewhere along the line >> commented out >> > > > something >> > > > > that was important to the release process. Hence why >> 0.14.0 >> > > > released >> > > > > source >> > > > > only. >> > > > > >> > > > > Since 2008, there has been a lot of great work on >> generating >> > > > plugins >> > > > > for >> > > > > doing Apache releases. Instead of the awkward hacks >> that made >> > > up >> > > > the >> > > > > old >> > > > > poms (literally comments that said, "this is a hack, >> there's >> > > > supposedly >> > > > > something better coming from ..." dated like 2012), >> we would >> > > > like to >> > > > > do it >> > > > > the "right way" and incorporate the appropriate >> plugins. >> > > > > >> > > > > Refactoring to SBT was _one_ proposed solution. We're >> also OK >> > > > > continuing to >> > > > > use Maven, and I agree with what you said about the >> cross >> > > > compiling. We >> > > > > actually have a script that just changes the scala >> version. >> > We >> > > > tried >> > > > > using >> > > > > the classifiers but there were issues in SBT, but the >> way >> > > you're >> > > > > proposing >> > > > > sounds a lot more pro than the route we were trying >> for. >> > > > > >> > > > > That said- we'd be OK just releasing one scala/spark >> version >> > at >> > > > a time. >> > > > > But getting the convenience binaries to >> release/publish would >> > > be >> > > > a >> > > > > major >> > > > > first step. >> > > > > >> > > > > Also, we really appreciate the help, >> > > > > >> > > > > tg >> > > > > >> > > > > >> > > > > [1] >> > > > > >> > > > > >> > > > >> > > >> > >> https://issues.apache.org/jira/projects/MAHOUT/issues/MAHOUT-2093?filter=allopenissues >> > > > > >> > > > > >> > > > > >> > > > > On Thu, Apr 16, 2020 at 4:50 AM Christofer Dutz < >> > > > > christofer.d...@c-ware.de> >> > > > > wrote: >> > > > > >> > > > > > Hi Stevo, >> > > > > > >> > > > > > so let me summarize what I understood: >> > > > > > >> > > > > > - There are some modules in mahout that are built >> with >> > Scala, >> > > > some >> > > > > with >> > > > > > java and some with both (At least that's what I see >> when >> > > > checking >> > > > > out the >> > > > > > project) >> > > > > > - The current build uses Scala 2.11 to build the >> Scala >> > code. >> > > > > > - The resulting libraries are only compatible with >> Scala >> > 2.11 >> > > > > > >> > > > > > Now you want to also publish versions compatible >> with Scala >> > > > 2.12? >> > > > > > >> > > > > > If that's the case I think Maven could easily add >> multiple >> > > > executions >> > > > > > where each compile compiles to different output >> > directories: >> > > > > > - Java --> target/classes >> > > > > > - Scala 2.11 --> target/classes-2.11 >> > > > > > - Scala 2.12 --> target/classes-2.12 >> > > > > > >> > > > > > Then the packaging would also need a second >> execution ... >> > > each >> > > > of the >> > > > > > executions bundling the classes and the >> corresponding scala >> > > > output. >> > > > > > Ideally I would probably use maven classifiers to >> > distinguish >> > > > the >> > > > > > artifacts. >> > > > > > >> > > > > > <dependency> >> > > > > > <groupId>org.apache.mahout</groupId> >> > > > > > <artifactId>mahout-spark</artifactId> >> > > > > > <version>14.1-SNAPSHOT</version> >> > > > > > <classifier>2.11</classifier> >> > > > > > </dependency> >> > > > > > >> > > > > > Then it should all work in a normal maven build. In >> the >> > > > > distributions you >> > > > > > could also filter the versions according to their >> > > classifiers. >> > > > > > >> > > > > > So if this is the case, I could help you with this. >> > > > > > >> > > > > > Chris >> > > > > > >> > > > > > >> > > > > > Am 16.04.20, 09:39 schrieb "Stevo Slavić" < >> > ssla...@gmail.com >> > > >: >> > > > > > >> > > > > > Disclaimer: I'm not active Mahout maintainer >> for quite >> > a >> > > > while, >> > > > > have >> > > > > > some >> > > > > > historical perspective, take it with a grain of >> salt, >> > > > could be >> > > > > I'm >> > > > > > missing >> > > > > > the whole point you were approached for by a >> wide >> > margin >> > > of >> > > > > error. >> > > > > > >> > > > > > At a point Mahout, some of its modules, have >> turned >> > into >> > > a >> > > > scala >> > > > > > library, and there was need to cross publish >> those >> > > modules, >> > > > > across >> > > > > > different scala versions. Back than Maven scala >> plugin >> > > > didn't >> > > > > support >> > > > > > cross >> > > > > > publishing, it doesn't fit well with Maven's >> build >> > > > lifecycle >> > > > > concept >> > > > > > (multiple compile phases - one for each scala >> version, >> > > and >> > > > what >> > > > > not >> > > > > > would >> > > > > > be needed). Switching to sbt could have solved >> the >> > > problem. >> > > > > Switch was >> > > > > > deemed to be too big task, even though ages >> have been >> > > > spent on >> > > > > trying >> > > > > > to >> > > > > > apply Maven (profiles) + bash scripts and what >> not to >> > > > solve the >> > > > > > problem. >> > > > > > Trying to apply same approach over and over >> again and >> > > > expecting >> > > > > > different >> > > > > > results is not smart, no expert can help there. >> Mahout >> > > > > maintainers and >> > > > > > contributors, should consider alternative >> approach, one >> > > of >> > > > them >> > > > > being >> > > > > > switching to sbt - it's scala native, supports >> scala >> > > cross >> > > > > publishing, >> > > > > > supports publishing Maven compatible release >> metadata >> > and >> > > > > binaries. >> > > > > > >> > > > > > Kind regards, >> > > > > > Stevo Slavic. >> > > > > > >> > > > > > On Thu, Apr 16, 2020 at 9:15 AM Christofer Dutz >> < >> > > > > > christofer.d...@c-ware.de> >> > > > > > wrote: >> > > > > > >> > > > > > > Hi folks, >> > > > > > > >> > > > > > > my name is Chris and I’m involved in quite a >> lot of >> > > > Apache >> > > > > projects. >> > > > > > > Justin approached me this morning, asking me >> if I >> > could >> > > > > perhaps help >> > > > > > you. >> > > > > > > He told me you were having trouble with doing >> Maven >> > > > releases. >> > > > > > > >> > > > > > > As Maven releases are my specialty, could you >> please >> > > > summarize >> > > > > the >> > > > > > issues >> > > > > > > you are having? >> > > > > > > >> > > > > > > Chris >> > > > > > > >> > > > > > >> > > > > > >> > > > > >> > > > > >> > > > >> > > > >> > > >> > >> >>
