Thank you Chris! Will take a look. On Mon, Apr 27, 2020 at 01:12 Christofer Dutz <christofer.d...@c-ware.de> wrote:
> Hi Folks, > > so I just pushed some changes I started on the weekend regarding the > assembly and distribution building. > > So all static resources config files and scripts I moved to the > distribution resources. > > The distribution now builds 3 artifacts: > - archive: apache-mahout-14.1-SNAPSHOT.tar.gz > - archive: apache-mahout-14.1-SNAPSHOT.tar.bz2 > - directory: apache-mahout-14.1-SNAPSHOT > > So if you build and have a look at the directory: > distribution/target/apache-mahout-14.1-SNAPSHOT > > That should be the runnable thing ... I tried to create the same structure > as with the last binary release (12.x). > However I didn't test the scripts or anything at all ... so if you could > please check if it's working and perhaps send a list of changes that need > to be done, that would be helpful. I'd make sure to implement those changes > ASAP. > > Chris > > > Am 24.04.20, 15:31 schrieb "Andrew Musselman" <andrew.mussel...@gmail.com > >: > > Alright, great; here's the link to join: > > https://hangouts.google.com/hangouts/_/calendar/YW5kcmV3Lm11c3NlbG1hbkBnbWFpbC5jb20.4tg9vg3t51gjlco0ebl5siim1s > > On Fri, Apr 24, 2020 at 02:19 Christofer Dutz < > christofer.d...@c-ware.de> > wrote: > > > Hi all, > > > > just found the email with the time for the call today ... that's > 19:00 / > > 7pm here ... perfect. > > Would just need some information on how to join. > > > > Chris > > > > > > Am 20.04.20, 17:31 schrieb "Andrew Musselman" < > andrew.mussel...@gmail.com > > >: > > > > Thanks Chris, will take a look at your PR. > > > > I think we would be fine upgrading anything that is still making > > improvements, probably makes sense to discuss Friday on our call > if > > you can > > make it. > > > > Best > > Andrew > > > > On Mon, Apr 20, 2020 at 1:53 AM Christofer Dutz < > > christofer.d...@c-ware.de> > > wrote: > > > > > Hi Folks, > > > > > > so I've now tested the build with java 1.8, 9, 10 and they > work ... > > > 11 I'm getting errors about unsupported java major versions > again ... > > > guess there's some old library version in there somewhere. > > > > > > But at least I managed to get you out of this 1.7 trap. > > > > > > Chris > > > > > > > > > Am 20.04.20, 09:46 schrieb "Christofer Dutz" < > > christofer.d...@c-ware.de>: > > > > > > Hi folks, > > > > > > so I was now able to build (including all tests) with Java > 8 and > > 9 ... > > > currently trying 10 ... > > > > > > Are there any objection that some maven dependencies get > updated > > to > > > more recent versions? I mean ... the hbase-client you're using > is > > more than > > > 5 years old ... > > > > > > Chris > > > > > > > > > Am 20.04.20, 00:29 schrieb "Andrew Musselman" < > > > andrew.mussel...@gmail.com>: > > > > > > No problem; would 10:00 a.m. Pacific next Friday the > 24th > > work for > > > you time > > > zone-wise? > > > > > > On Sun, Apr 19, 2020 at 12:15 PM Christofer Dutz < > > > christofer.d...@c-ware.de> > > > wrote: > > > > > > > Sorry ... > > > > > > > > didn't see your response ... into Mahout too deep ;-) > > > > Guess we have to postpone this to sometime over the > week. > > > > > > > > Chris > > > > > > > > > > > > Am 19.04.20, 19:56 schrieb "Andrew Musselman" < > > > andrew.mussel...@gmail.com > > > > >: > > > > > > > > *what time > > > > > > > > On Sun, Apr 19, 2020 at 10:51 Andrew Musselman < > > > > andrew.mussel...@gmail.com> > > > > wrote: > > > > > > > > > I think it's safe to move to 1.8; yeah what > tune is > > good > > > for you? > > > > I'm in > > > > > Pacific time zone and am flexible this > afternoon. > > > > > > > > > > Trevor you free? > > > > > > > > > > On Sun, Apr 19, 2020 at 09:37 Christofer Dutz < > > > > christofer.d...@c-ware.de> > > > > > wrote: > > > > > > > > > >> Yikes! ... well guess then I can't help you > folks > > as it's > > > almost > > > > >> impossible to get my hands on a 1.7 version. > > > > >> > > > > >> What's preventing you from going to 1.8+? > > > > >> > > > > >> Any your build says 1.8 and above: > > > > >> <requireJavaVersion> > > > > >> <version>[1.8,)</version> > > > > >> </requireJavaVersion> > > > > >> > > > > >> Regarding the artifacts ... would it be ok to > have > > the > > > maven > > > > artifacts > > > > >> using classifiers and perhaps the files in > the libs > > and > > > > distribution to > > > > >> follow the typical scala sheme? > > > > >> > > > > >> Chris > > > > >> > > > > >> Am 19.04.20, 17:56 schrieb "Trevor Grant" < > > > trevor.d.gr...@gmail.com > > > > >: > > > > >> > > > > >> Yea, we have a requirement on 1.7. we > need to > > get it > > > up to 8, > > > > but > > > > >> considered that a different issue. > > > > >> > > > > >> Maven throws warning, sbt breaks down > entirely > > (when > > > importing)- > > > > >> hence why > > > > >> we were using a script to replace 2.11 w > 2.12 > > > > >> > > > > >> On Sun, Apr 19, 2020, 10:33 AM Christofer > Dutz < > > > > >> christofer.d...@c-ware.de> > > > > >> wrote: > > > > >> > > > > >> > Well I have been compiling with various > jdks > > from 8 > > > to 14. > > > > However I > > > > >> > noticed that if I select a jdk and > start the > > build > > > that the > > > > Scala > > > > >> parts > > > > >> > Seefahrt to the version in my systems > path. > > Which > > > then > > > > results in > > > > >> all sorts > > > > >> > of major version problems. > > > > >> > > > > > >> > For the Scalatest plugin I managed to > have it > > use > > > the maven > > > > >> java.home... > > > > >> > The Scala compiler is being a little > trickyer. > > > > >> > > > > > >> > Will continue this evening. Keep your > fingers > > > crossed. > > > > >> > > > > > >> > Also does maven totally not like > variables in > > > artifactids. > > > > >> > > > > > >> > Chris > > > > >> > ________________________________ > > > > >> > Von: Trevor Grant < > trevor.d.gr...@gmail.com> > > > > >> > Gesendet: Sonntag, 19. April 2020 15:41 > > > > >> > An: Mahout Dev List < > dev@mahout.apache.org> > > > > >> > Betreff: Re: Hi ... need some help? > > > > >> > > > > > >> > Hey Chris, > > > > >> > > > > > >> > First of all- thank you so much for > your work > > so > > > far. > > > > >> > > > > > >> > The build 100% currently works, the only > > issue is > > > with > > > > releasing > > > > >> binaries. > > > > >> > Are you compiling w 1.7? Not really > sure why > > > artifact IDs are > > > > >> changing? > > > > >> > That's very strange. > > > > >> > > > > > >> > To answer your question regarding the > > community > > > block- no, it > > > > is > > > > >> not as if > > > > >> > there is a company that pays us to work > on > > Mahout > > > and the > > > > rest is > > > > >> > community. What it means is that while > we > > > appreciate those > > > > code > > > > >> > contributions, no one still actively > involved > > is > > > maintaining > > > > them > > > > >> and we > > > > >> > advise people to use at their own risk. > The > > > refactor to a > > > > >> `community` > > > > >> > folder was meant to reflect the reality > of the > > > situation. > > > > >> > > > > > >> > The only test issues I was aware of > were with > > > Javadocs when > > > > trying > > > > >> to do a > > > > >> > release with Java 1.8+ those would fail > due to > > > formatting. > > > > >> > > > > > >> > I feel like you're hitting some unique > > issues, and > > > maybe we > > > > could > > > > >> help you > > > > >> > out / there may exist simple solutions. > > > > >> > > > > > >> > Thank you again for your help! > > > > >> > > > > > >> > tg > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > On Sun, Apr 19, 2020 at 2:17 AM > Christofer > > Dutz < > > > > >> christofer.d...@c-ware.de > > > > >> > > > > > > >> > wrote: > > > > >> > > > > > >> > > Hi folks, > > > > >> > > > > > > >> > > So yesterday I invested several hours > in > > cleaning > > > up your > > > > build. > > > > >> I got > > > > >> > > quite far, currently trying to get the > > tests to > > > pass. For > > > > some > > > > >> > > Scala-Java-major-version problems, but > > working on > > > fixing > > > > them. > > > > >> > > > > > > >> > > However sind things will be > different. I > > hope > > > that's OK. > > > > >> Especially with > > > > >> > > the artifact ids. > > > > >> > > > > > > >> > > Just as a question: does the build > > currently work > > > at all? > > > > >> Especially in > > > > >> > > the community block I had to fix > quite a > > few API > > > changes > > > > where > > > > >> the code > > > > >> > in > > > > >> > > the blocks were for older versions of > > libraries. > > > > >> > > > > > > >> > > I hope I get some more tests to pass > today. > > > > >> > > > > > > >> > > Chris > > > > >> > > ________________________________ > > > > >> > > Von: Andrew Musselman <a...@apache.org > > > > > > >> > > Gesendet: Donnerstag, 16. April 2020 > 20:35 > > > > >> > > An: Mahout Dev List < > dev@mahout.apache.org> > > > > >> > > Betreff: Re: Hi ... need some help? > > > > >> > > > > > > >> > > Looking forward to working on this > with you; > > > thanks again! > > > > >> > > > > > > >> > > On Thu, Apr 16, 2020 at 11:09 AM > Christofer > > Dutz < > > > > >> > > christofer.d...@c-ware.de> > > > > >> > > wrote: > > > > >> > > > > > > >> > > > Hi Andrew, > > > > >> > > > > > > > >> > > > guess I'll start with the fork and > > contact you > > > folks on > > > > slack. > > > > >> > > > > > > > >> > > > Chris > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > Am 16.04.20, 19:43 schrieb "Andrew > > Musselman" < > > > > a...@apache.org>: > > > > >> > > > > > > > >> > > > Chris, thank you for your help.. > > > > >> > > > > > > > >> > > > Yeah if you fork what's in > master you > > can > > > see what > > > > state > > > > >> it's in; > > > > >> > we > > > > >> > > > are in > > > > >> > > > the #mahout channel in tge-asf > slack > > and > > > this is also > > > > a > > > > >> fine way to > > > > >> > > > keep > > > > >> > > > track of discussion. > > > > >> > > > > > > > >> > > > We could file a JIRA ticket as > well, > > > however you > > > > prefer to > > > > >> work. > > > > >> > > > > > > > >> > > > Best > > > > >> > > > Andrew > > > > >> > > > > > > > >> > > > On Thu, Apr 16, 2020 at 06:59 > > Christofer > > > Dutz < > > > > >> > > > christofer.d...@c-ware.de> > > > > >> > > > wrote: > > > > >> > > > > > > > >> > > > > Hi Trevor, > > > > >> > > > > > > > > >> > > > > ok ... first of all ... the > Mahout > > PMC is > > > defining a > > > > >> "community > > > > >> > > > > maintained" library which is > not > > > maintained by the > > > > mahout > > > > >> PMC?!?! > > > > >> > > > > I thought at Apache > everything is > > about > > > Community > > > > over > > > > >> code. So > > > > >> > is > > > > >> > > a > > > > >> > > > > company driving the > non-community > > stuff? > > > > >> > > > > > > > > >> > > > > But back to your build issues: > > > > >> > > > > I had a look and I too > encountered > > these > > > comments > > > > and > > > > >> remarks and > > > > >> > > > > sometimes patterns I > recognized and > > could > > > imagine > > > > why > > > > >> they were > > > > >> > > > created. > > > > >> > > > > Yes quite a bit of the build > could > > be > > > cleaned up and > > > > >> simplified a > > > > >> > > > lot. > > > > >> > > > > > > > > >> > > > > So how about I create a fork > and > > try to > > > do a > > > > cleanup of > > > > >> the > > > > >> > build. > > > > >> > > > > Usually I also leave comments > about > > what > > > I do as I > > > > hope > > > > >> I'll not > > > > >> > be > > > > >> > > > the > > > > >> > > > > only one maintaining a build > and > > > documenting things > > > > helps > > > > >> people > > > > >> > > > feel more > > > > >> > > > > confident. > > > > >> > > > > > > > > >> > > > > However in some cases I will > have > > > questions ... so > > > > would > > > > >> someone > > > > >> > be > > > > >> > > > > available on Slack for quick > > questions? > > > > >> > > > > > > > > >> > > > > Usually switching to another > build > > system > > > does > > > > solve some > > > > >> > problems > > > > >> > > > ... > > > > >> > > > > mostly the reason to switch > is that > > it > > > solved the > > > > main > > > > >> problem > > > > >> > that > > > > >> > > > you are > > > > >> > > > > having with the old. > > > > >> > > > > However you usually notice > too late > > that > > > you get > > > > yourself > > > > >> a lot > > > > >> > of > > > > >> > > > new > > > > >> > > > > problems. I remember doing > some > > contract > > > work for an > > > > >> insurance > > > > >> > > > company and > > > > >> > > > > they were totally down > Maven-road > > but > > > then had to > > > > build > > > > >> something > > > > >> > > > with SBT > > > > >> > > > > ... in the end I compiled the > thing > > on my > > > laptop, > > > > copied > > > > >> it to a > > > > >> > > USB > > > > >> > > > stick > > > > >> > > > > and told the people what was > on the > > stick > > > and that > > > > I'll be > > > > >> > having a > > > > >> > > > coffee > > > > >> > > > > and will be back in 30 > minutes. > > When I > > > came back > > > > the sick > > > > >> wasn't > > > > >> > at > > > > >> > > > the > > > > >> > > > > same place and the build > problem was > > > "solved" ;-) > > > > >> > > > > > > > > >> > > > > So I think it's quite good to > stick > > to > > > maven ... > > > > that is > > > > >> very > > > > >> > > > mature, you > > > > >> > > > > can do almost everything you > want > > with it > > > and it > > > > >> integrates > > > > >> > > > perfectly into > > > > >> > > > > the Apache infrastructure. > > > > >> > > > > > > > > >> > > > > But that's just my opinion. > > > > >> > > > > > > > > >> > > > > So if you want me to help, > I'll be > > happy > > > to be of > > > > >> assistance. > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > Chris > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > Am 16.04.20, 15:28 schrieb > "Trevor > > Grant" > > > < > > > > >> > > trevor.d.gr...@gmail.com > > > > >> > > > >: > > > > >> > > > > > > > > >> > > > > Hey Christopher, > > > > >> > > > > > > > > >> > > > > I would agree with what > Stevo > > > outlined but add > > > > some > > > > >> more > > > > >> > > context > > > > >> > > > and a > > > > >> > > > > couple related JIRA > issues. > > > > >> > > > > > > > > >> > > > > For 0.14.0 We did a big > > refactor and > > > finally > > > > moved the > > > > >> > > MapReduce > > > > >> > > > based > > > > >> > > > > Mahout all into what we > called > > > "community/" > > > > that is > > > > >> community > > > > >> > > > > maintained, > > > > >> > > > > which is to say, we're not > > > maintaining it > > > > anymore > > > > >> (sunset > > > > >> > > began I > > > > >> > > > > think in > > > > >> > > > > 2015). > > > > >> > > > > > > > > >> > > > > But all of our POMs were > so > > huge and > > > fat because > > > > >> they'd been > > > > >> > > > layered up > > > > >> > > > > over the years by people > coming > > and > > > going and > > > > >> dropping in > > > > >> > > code. I > > > > >> > > > > wouldn't > > > > >> > > > > call these drive- bys, > its just > > been > > > over 10 > > > > years > > > > >> and people > > > > >> > > > come and > > > > >> > > > > go. > > > > >> > > > > Such is the life of Apache > > Projects. > > > So we had a > > > > >> situation > > > > >> > > where > > > > >> > > > a lot > > > > >> > > > > of > > > > >> > > > > the old Map Reduce stuff > and > > the POMs > > > were > > > > considered > > > > >> > > > "old-magic" no > > > > >> > > > > one > > > > >> > > > > really knew how it was > all tied > > > together, but we > > > > >> didn't want > > > > >> > to > > > > >> > > > mess > > > > >> > > > > with > > > > >> > > > > it for fear of breaking > > something in > > > the "new" > > > > Mahout > > > > >> (aka > > > > >> > > > Samsar) > > > > >> > > > > which is > > > > >> > > > > the Scala/Spark based > library > > that it > > > is now* > > > > (to > > > > >> others in > > > > >> > the > > > > >> > > > > community: > > > > >> > > > > I know it runs on other > > engines, but > > > for > > > > simplicity, > > > > >> I'm just > > > > >> > > > calling > > > > >> > > > > it > > > > >> > > > > "runs-on-spark"). > > > > >> > > > > > > > > >> > > > > For 0.14.0 We decided to > trim > > out as > > > much of > > > > that > > > > >> which was > > > > >> > > > possible. > > > > >> > > > > We > > > > >> > > > > did some major > liposuction on > > POMs, > > > re organized > > > > >> things, etc. > > > > >> > > > This was > > > > >> > > > > done > > > > >> > > > > by commenting out a > section, > > then > > > seeing if it > > > > would > > > > >> still > > > > >> > > > build. So > > > > >> > > > > the > > > > >> > > > > current release > > > > >> > > > > _does_ build. And aside > for > > some CLI > > > driver > > > > issues > > > > >> which are > > > > >> > > > outlined > > > > >> > > > > in > > > > >> > > > > [1], the project runs > fairly > > smooth. > > > (An SBT > > > > would > > > > >> probably > > > > >> > > > solve [1], > > > > >> > > > > I > > > > >> > > > > believe Pat Ferrel has > made his > > own > > > SBT script > > > > to > > > > >> compile > > > > >> > > > Mahout, which > > > > >> > > > > solved that problem for > them). > > > > >> > > > > > > > > >> > > > > The issue we ran into > with the > > > releases (and the > > > > >> reason I > > > > >> > think > > > > >> > > > you're > > > > >> > > > > here), is that we also > somewhere > > > along the line > > > > >> commented out > > > > >> > > > something > > > > >> > > > > that was important to the > > release > > > process. > > > > Hence why > > > > >> 0.14.0 > > > > >> > > > released > > > > >> > > > > source > > > > >> > > > > only. > > > > >> > > > > > > > > >> > > > > Since 2008, there has > been a > > lot of > > > great work > > > > on > > > > >> generating > > > > >> > > > plugins > > > > >> > > > > for > > > > >> > > > > doing Apache releases. > Instead > > of the > > > awkward > > > > hacks > > > > >> that made > > > > >> > > up > > > > >> > > > the > > > > >> > > > > old > > > > >> > > > > poms (literally comments > that > > said, > > > "this is a > > > > hack, > > > > >> there's > > > > >> > > > supposedly > > > > >> > > > > something better coming > from > > ..." > > > dated like > > > > 2012), > > > > >> we would > > > > >> > > > like to > > > > >> > > > > do it > > > > >> > > > > the "right way" and > incorporate > > the > > > appropriate > > > > >> plugins. > > > > >> > > > > > > > > >> > > > > Refactoring to SBT was > _one_ > > proposed > > > solution. > > > > We're > > > > >> also OK > > > > >> > > > > continuing to > > > > >> > > > > use Maven, and I agree > with > > what you > > > said about > > > > the > > > > >> cross > > > > >> > > > compiling. We > > > > >> > > > > actually have a script > that just > > > changes the > > > > scala > > > > >> version. > > > > >> > We > > > > >> > > > tried > > > > >> > > > > using > > > > >> > > > > the classifiers but there > were > > issues > > > in SBT, > > > > but the > > > > >> way > > > > >> > > you're > > > > >> > > > > proposing > > > > >> > > > > sounds a lot more pro > than the > > route > > > we were > > > > trying > > > > >> for. > > > > >> > > > > > > > > >> > > > > That said- we'd be OK just > > releasing > > > one > > > > scala/spark > > > > >> version > > > > >> > at > > > > >> > > > a time. > > > > >> > > > > But getting the > convenience > > binaries > > > to > > > > >> release/publish would > > > > >> > > be > > > > >> > > > a > > > > >> > > > > major > > > > >> > > > > first step. > > > > >> > > > > > > > > >> > > > > Also, we really > appreciate the > > help, > > > > >> > > > > > > > > >> > > > > tg > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > [1] > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > > > > > > > https://issues.apache.org/jira/projects/MAHOUT/issues/MAHOUT-2093?filter=allopenissues > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > On Thu, Apr 16, 2020 at > 4:50 AM > > > Christofer Dutz > > > > < > > > > >> > > > > christofer.d...@c-ware.de> > > > > >> > > > > wrote: > > > > >> > > > > > > > > >> > > > > > Hi Stevo, > > > > >> > > > > > > > > > >> > > > > > so let me summarize > what I > > > understood: > > > > >> > > > > > > > > > >> > > > > > - There are some > modules in > > mahout > > > that are > > > > built > > > > >> with > > > > >> > Scala, > > > > >> > > > some > > > > >> > > > > with > > > > >> > > > > > java and some with both > (At > > least > > > that's what > > > > I see > > > > >> when > > > > >> > > > checking > > > > >> > > > > out the > > > > >> > > > > > project) > > > > >> > > > > > - The current build uses > > Scala 2.11 > > > to build > > > > the > > > > >> Scala > > > > >> > code. > > > > >> > > > > > - The resulting > libraries are > > only > > > compatible > > > > with > > > > >> Scala > > > > >> > 2.11 > > > > >> > > > > > > > > > >> > > > > > Now you want to also > publish > > > versions > > > > compatible > > > > >> with Scala > > > > >> > > > 2.12? > > > > >> > > > > > > > > > >> > > > > > If that's the case I > think > > Maven > > > could easily > > > > add > > > > >> multiple > > > > >> > > > executions > > > > >> > > > > > where each compile > compiles to > > > different > > > > output > > > > >> > directories: > > > > >> > > > > > - Java --> > target/classes > > > > >> > > > > > - Scala 2.11 --> > > target/classes-2.11 > > > > >> > > > > > - Scala 2.12 --> > > target/classes-2.12 > > > > >> > > > > > > > > > >> > > > > > Then the packaging > would also > > need > > > a second > > > > >> execution ... > > > > >> > > each > > > > >> > > > of the > > > > >> > > > > > executions bundling the > > classes and > > > the > > > > >> corresponding scala > > > > >> > > > output. > > > > >> > > > > > Ideally I would > probably use > > maven > >
