Hi all,
just found the email with the time for the call today ... that's 19:00 / 7pm
here ... perfect.
Would just need some information on how to join.
Chris
Am 20.04.20, 17:31 schrieb "Andrew Musselman" <andrew.mussel...@gmail.com>:
Thanks Chris, will take a look at your PR.
I think we would be fine upgrading anything that is still making
improvements, probably makes sense to discuss Friday on our call if you can
make it.
Best
Andrew
On Mon, Apr 20, 2020 at 1:53 AM Christofer Dutz <christofer.d...@c-ware.de>
wrote:
> Hi Folks,
>
> so I've now tested the build with java 1.8, 9, 10 and they work ...
> 11 I'm getting errors about unsupported java major versions again ...
> guess there's some old library version in there somewhere.
>
> But at least I managed to get you out of this 1.7 trap.
>
> Chris
>
>
> Am 20.04.20, 09:46 schrieb "Christofer Dutz" <christofer.d...@c-ware.de>:
>
> Hi folks,
>
> so I was now able to build (including all tests) with Java 8 and 9 ...
> currently trying 10 ...
>
> Are there any objection that some maven dependencies get updated to
> more recent versions? I mean ... the hbase-client you're using is more
than
> 5 years old ...
>
> Chris
>
>
> Am 20.04.20, 00:29 schrieb "Andrew Musselman" <
> andrew.mussel...@gmail.com>:
>
> No problem; would 10:00 a.m. Pacific next Friday the 24th work for
> you time
> zone-wise?
>
> On Sun, Apr 19, 2020 at 12:15 PM Christofer Dutz <
> christofer.d...@c-ware.de>
> wrote:
>
> > Sorry ...
> >
> > didn't see your response ... into Mahout too deep ;-)
> > Guess we have to postpone this to sometime over the week.
> >
> > Chris
> >
> >
> > Am 19.04.20, 19:56 schrieb "Andrew Musselman" <
> andrew.mussel...@gmail.com
> > >:
> >
> > *what time
> >
> > On Sun, Apr 19, 2020 at 10:51 Andrew Musselman <
> > andrew.mussel...@gmail.com>
> > wrote:
> >
> > > I think it's safe to move to 1.8; yeah what tune is good
> for you?
> > I'm in
> > > Pacific time zone and am flexible this afternoon.
> > >
> > > Trevor you free?
> > >
> > > On Sun, Apr 19, 2020 at 09:37 Christofer Dutz <
> > christofer.d...@c-ware.de>
> > > wrote:
> > >
> > >> Yikes! ... well guess then I can't help you folks as it's
> almost
> > >> impossible to get my hands on a 1.7 version.
> > >>
> > >> What's preventing you from going to 1.8+?
> > >>
> > >> Any your build says 1.8 and above:
> > >> <requireJavaVersion>
> > >> <version>[1.8,)</version>
> > >> </requireJavaVersion>
> > >>
> > >> Regarding the artifacts ... would it be ok to have the
> maven
> > artifacts
> > >> using classifiers and perhaps the files in the libs and
> > distribution to
> > >> follow the typical scala sheme?
> > >>
> > >> Chris
> > >>
> > >> Am 19.04.20, 17:56 schrieb "Trevor Grant" <
> trevor.d.gr...@gmail.com
> > >:
> > >>
> > >> Yea, we have a requirement on 1.7. we need to get it
> up to 8,
> > but
> > >> considered that a different issue.
> > >>
> > >> Maven throws warning, sbt breaks down entirely (when
> importing)-
> > >> hence why
> > >> we were using a script to replace 2.11 w 2.12
> > >>
> > >> On Sun, Apr 19, 2020, 10:33 AM Christofer Dutz <
> > >> christofer.d...@c-ware.de>
> > >> wrote:
> > >>
> > >> > Well I have been compiling with various jdks from 8
> to 14.
> > However I
> > >> > noticed that if I select a jdk and start the build
> that the
> > Scala
> > >> parts
> > >> > Seefahrt to the version in my systems path. Which
> then
> > results in
> > >> all sorts
> > >> > of major version problems.
> > >> >
> > >> > For the Scalatest plugin I managed to have it use
> the maven
> > >> java.home...
> > >> > The Scala compiler is being a little trickyer.
> > >> >
> > >> > Will continue this evening. Keep your fingers
> crossed.
> > >> >
> > >> > Also does maven totally not like variables in
> artifactids.
> > >> >
> > >> > Chris
> > >> > ________________________________
> > >> > Von: Trevor Grant <trevor.d.gr...@gmail.com>
> > >> > Gesendet: Sonntag, 19. April 2020 15:41
> > >> > An: Mahout Dev List <dev@mahout.apache.org>
> > >> > Betreff: Re: Hi ... need some help?
> > >> >
> > >> > Hey Chris,
> > >> >
> > >> > First of all- thank you so much for your work so
> far.
> > >> >
> > >> > The build 100% currently works, the only issue is
> with
> > releasing
> > >> binaries.
> > >> > Are you compiling w 1.7? Not really sure why
> artifact IDs are
> > >> changing?
> > >> > That's very strange.
> > >> >
> > >> > To answer your question regarding the community
> block- no, it
> > is
> > >> not as if
> > >> > there is a company that pays us to work on Mahout
> and the
> > rest is
> > >> > community. What it means is that while we
> appreciate those
> > code
> > >> > contributions, no one still actively involved is
> maintaining
> > them
> > >> and we
> > >> > advise people to use at their own risk. The
> refactor to a
> > >> `community`
> > >> > folder was meant to reflect the reality of the
> situation.
> > >> >
> > >> > The only test issues I was aware of were with
> Javadocs when
> > trying
> > >> to do a
> > >> > release with Java 1.8+ those would fail due to
> formatting.
> > >> >
> > >> > I feel like you're hitting some unique issues, and
> maybe we
> > could
> > >> help you
> > >> > out / there may exist simple solutions.
> > >> >
> > >> > Thank you again for your help!
> > >> >
> > >> > tg
> > >> >
> > >> >
> > >> >
> > >> >
> > >> > On Sun, Apr 19, 2020 at 2:17 AM Christofer Dutz <
> > >> christofer.d...@c-ware.de
> > >> > >
> > >> > wrote:
> > >> >
> > >> > > Hi folks,
> > >> > >
> > >> > > So yesterday I invested several hours in cleaning
> up your
> > build.
> > >> I got
> > >> > > quite far, currently trying to get the tests to
> pass. For
> > some
> > >> > > Scala-Java-major-version problems, but working on
> fixing
> > them.
> > >> > >
> > >> > > However sind things will be different. I hope
> that's OK.
> > >> Especially with
> > >> > > the artifact ids.
> > >> > >
> > >> > > Just as a question: does the build currently work
> at all?
> > >> Especially in
> > >> > > the community block I had to fix quite a few API
> changes
> > where
> > >> the code
> > >> > in
> > >> > > the blocks were for older versions of libraries.
> > >> > >
> > >> > > I hope I get some more tests to pass today.
> > >> > >
> > >> > > Chris
> > >> > > ________________________________
> > >> > > Von: Andrew Musselman <a...@apache.org>
> > >> > > Gesendet: Donnerstag, 16. April 2020 20:35
> > >> > > An: Mahout Dev List <dev@mahout.apache.org>
> > >> > > Betreff: Re: Hi ... need some help?
> > >> > >
> > >> > > Looking forward to working on this with you;
> thanks again!
> > >> > >
> > >> > > On Thu, Apr 16, 2020 at 11:09 AM Christofer Dutz
<
> > >> > > christofer.d...@c-ware.de>
> > >> > > wrote:
> > >> > >
> > >> > > > Hi Andrew,
> > >> > > >
> > >> > > > guess I'll start with the fork and contact you
> folks on
> > slack.
> > >> > > >
> > >> > > > Chris
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > Am 16.04.20, 19:43 schrieb "Andrew Musselman" <
> > a...@apache.org>:
> > >> > > >
> > >> > > > Chris, thank you for your help..
> > >> > > >
> > >> > > > Yeah if you fork what's in master you can
> see what
> > state
> > >> it's in;
> > >> > we
> > >> > > > are in
> > >> > > > the #mahout channel in tge-asf slack and
> this is also
> > a
> > >> fine way to
> > >> > > > keep
> > >> > > > track of discussion.
> > >> > > >
> > >> > > > We could file a JIRA ticket as well,
> however you
> > prefer to
> > >> work.
> > >> > > >
> > >> > > > Best
> > >> > > > Andrew
> > >> > > >
> > >> > > > On Thu, Apr 16, 2020 at 06:59 Christofer
> Dutz <
> > >> > > > christofer.d...@c-ware.de>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > Hi Trevor,
> > >> > > > >
> > >> > > > > ok ... first of all ... the Mahout PMC is
> defining a
> > >> "community
> > >> > > > > maintained" library which is not
> maintained by the
> > mahout
> > >> PMC?!?!
> > >> > > > > I thought at Apache everything is about
> Community
> > over
> > >> code. So
> > >> > is
> > >> > > a
> > >> > > > > company driving the non-community stuff?
> > >> > > > >
> > >> > > > > But back to your build issues:
> > >> > > > > I had a look and I too encountered these
> comments
> > and
> > >> remarks and
> > >> > > > > sometimes patterns I recognized and could
> imagine
> > why
> > >> they were
> > >> > > > created.
> > >> > > > > Yes quite a bit of the build could be
> cleaned up and
> > >> simplified a
> > >> > > > lot.
> > >> > > > >
> > >> > > > > So how about I create a fork and try to
> do a
> > cleanup of
> > >> the
> > >> > build.
> > >> > > > > Usually I also leave comments about what
> I do as I
> > hope
> > >> I'll not
> > >> > be
> > >> > > > the
> > >> > > > > only one maintaining a build and
> documenting things
> > helps
> > >> people
> > >> > > > feel more
> > >> > > > > confident.
> > >> > > > >
> > >> > > > > However in some cases I will have
> questions ... so
> > would
> > >> someone
> > >> > be
> > >> > > > > available on Slack for quick questions?
> > >> > > > >
> > >> > > > > Usually switching to another build system
> does
> > solve some
> > >> > problems
> > >> > > > ...
> > >> > > > > mostly the reason to switch is that it
> solved the
> > main
> > >> problem
> > >> > that
> > >> > > > you are
> > >> > > > > having with the old.
> > >> > > > > However you usually notice too late that
> you get
> > yourself
> > >> a lot
> > >> > of
> > >> > > > new
> > >> > > > > problems. I remember doing some contract
> work for an
> > >> insurance
> > >> > > > company and
> > >> > > > > they were totally down Maven-road but
> then had to
> > build
> > >> something
> > >> > > > with SBT
> > >> > > > > ... in the end I compiled the thing on my
> laptop,
> > copied
> > >> it to a
> > >> > > USB
> > >> > > > stick
> > >> > > > > and told the people what was on the stick
> and that
> > I'll be
> > >> > having a
> > >> > > > coffee
> > >> > > > > and will be back in 30 minutes. When I
> came back
> > the sick
> > >> wasn't
> > >> > at
> > >> > > > the
> > >> > > > > same place and the build problem was
> "solved" ;-)
> > >> > > > >
> > >> > > > > So I think it's quite good to stick to
> maven ...
> > that is
> > >> very
> > >> > > > mature, you
> > >> > > > > can do almost everything you want with it
> and it
> > >> integrates
> > >> > > > perfectly into
> > >> > > > > the Apache infrastructure.
> > >> > > > >
> > >> > > > > But that's just my opinion.
> > >> > > > >
> > >> > > > > So if you want me to help, I'll be happy
> to be of
> > >> assistance.
> > >> > > > >
> > >> > > > >
> > >> > > > > Chris
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > Am 16.04.20, 15:28 schrieb "Trevor Grant"
> <
> > >> > > trevor.d.gr...@gmail.com
> > >> > > > >:
> > >> > > > >
> > >> > > > > Hey Christopher,
> > >> > > > >
> > >> > > > > I would agree with what Stevo
> outlined but add
> > some
> > >> more
> > >> > > context
> > >> > > > and a
> > >> > > > > couple related JIRA issues.
> > >> > > > >
> > >> > > > > For 0.14.0 We did a big refactor and
> finally
> > moved the
> > >> > > MapReduce
> > >> > > > based
> > >> > > > > Mahout all into what we called
> "community/"
> > that is
> > >> community
> > >> > > > > maintained,
> > >> > > > > which is to say, we're not
> maintaining it
> > anymore
> > >> (sunset
> > >> > > began I
> > >> > > > > think in
> > >> > > > > 2015).
> > >> > > > >
> > >> > > > > But all of our POMs were so huge and
> fat because
> > >> they'd been
> > >> > > > layered up
> > >> > > > > over the years by people coming and
> going and
> > >> dropping in
> > >> > > code. I
> > >> > > > > wouldn't
> > >> > > > > call these drive- bys, its just been
> over 10
> > years
> > >> and people
> > >> > > > come and
> > >> > > > > go.
> > >> > > > > Such is the life of Apache Projects.
> So we had a
> > >> situation
> > >> > > where
> > >> > > > a lot
> > >> > > > > of
> > >> > > > > the old Map Reduce stuff and the POMs
> were
> > considered
> > >> > > > "old-magic" no
> > >> > > > > one
> > >> > > > > really knew how it was all tied
> together, but we
> > >> didn't want
> > >> > to
> > >> > > > mess
> > >> > > > > with
> > >> > > > > it for fear of breaking something in
> the "new"
> > Mahout
> > >> (aka
> > >> > > > Samsar)
> > >> > > > > which is
> > >> > > > > the Scala/Spark based library that it
> is now*
> > (to
> > >> others in
> > >> > the
> > >> > > > > community:
> > >> > > > > I know it runs on other engines, but
> for
> > simplicity,
> > >> I'm just
> > >> > > > calling
> > >> > > > > it
> > >> > > > > "runs-on-spark").
> > >> > > > >
> > >> > > > > For 0.14.0 We decided to trim out as
> much of
> > that
> > >> which was
> > >> > > > possible.
> > >> > > > > We
> > >> > > > > did some major liposuction on POMs,
> re organized
> > >> things, etc.
> > >> > > > This was
> > >> > > > > done
> > >> > > > > by commenting out a section, then
> seeing if it
> > would
> > >> still
> > >> > > > build. So
> > >> > > > > the
> > >> > > > > current release
> > >> > > > > _does_ build. And aside for some CLI
> driver
> > issues
> > >> which are
> > >> > > > outlined
> > >> > > > > in
> > >> > > > > [1], the project runs fairly smooth.
> (An SBT
> > would
> > >> probably
> > >> > > > solve [1],
> > >> > > > > I
> > >> > > > > believe Pat Ferrel has made his own
> SBT script
> > to
> > >> compile
> > >> > > > Mahout, which
> > >> > > > > solved that problem for them).
> > >> > > > >
> > >> > > > > The issue we ran into with the
> releases (and the
> > >> reason I
> > >> > think
> > >> > > > you're
> > >> > > > > here), is that we also somewhere
> along the line
> > >> commented out
> > >> > > > something
> > >> > > > > that was important to the release
> process.
> > Hence why
> > >> 0.14.0
> > >> > > > released
> > >> > > > > source
> > >> > > > > only.
> > >> > > > >
> > >> > > > > Since 2008, there has been a lot of
> great work
> > on
> > >> generating
> > >> > > > plugins
> > >> > > > > for
> > >> > > > > doing Apache releases. Instead of the
> awkward
> > hacks
> > >> that made
> > >> > > up
> > >> > > > the
> > >> > > > > old
> > >> > > > > poms (literally comments that said,
> "this is a
> > hack,
> > >> there's
> > >> > > > supposedly
> > >> > > > > something better coming from ..."
> dated like
> > 2012),
> > >> we would
> > >> > > > like to
> > >> > > > > do it
> > >> > > > > the "right way" and incorporate the
> appropriate
> > >> plugins.
> > >> > > > >
> > >> > > > > Refactoring to SBT was _one_ proposed
> solution.
> > We're
> > >> also OK
> > >> > > > > continuing to
> > >> > > > > use Maven, and I agree with what you
> said about
> > the
> > >> cross
> > >> > > > compiling. We
> > >> > > > > actually have a script that just
> changes the
> > scala
> > >> version.
> > >> > We
> > >> > > > tried
> > >> > > > > using
> > >> > > > > the classifiers but there were issues
> in SBT,
> > but the
> > >> way
> > >> > > you're
> > >> > > > > proposing
> > >> > > > > sounds a lot more pro than the route
> we were
> > trying
> > >> for.
> > >> > > > >
> > >> > > > > That said- we'd be OK just releasing
> one
> > scala/spark
> > >> version
> > >> > at
> > >> > > > a time.
> > >> > > > > But getting the convenience binaries
> to
> > >> release/publish would
> > >> > > be
> > >> > > > a
> > >> > > > > major
> > >> > > > > first step.
> > >> > > > >
> > >> > > > > Also, we really appreciate the help,
> > >> > > > >
> > >> > > > > tg
> > >> > > > >
> > >> > > > >
> > >> > > > > [1]
> > >> > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
>
https://issues.apache.org/jira/projects/MAHOUT/issues/MAHOUT-2093?filter=allopenissues
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > On Thu, Apr 16, 2020 at 4:50 AM
> Christofer Dutz
> > <
> > >> > > > > christofer.d...@c-ware.de>
> > >> > > > > wrote:
> > >> > > > >
> > >> > > > > > Hi Stevo,
> > >> > > > > >
> > >> > > > > > so let me summarize what I
> understood:
> > >> > > > > >
> > >> > > > > > - There are some modules in mahout
> that are
> > built
> > >> with
> > >> > Scala,
> > >> > > > some
> > >> > > > > with
> > >> > > > > > java and some with both (At least
> that's what
> > I see
> > >> when
> > >> > > > checking
> > >> > > > > out the
> > >> > > > > > project)
> > >> > > > > > - The current build uses Scala 2.11
> to build
> > the
> > >> Scala
> > >> > code.
> > >> > > > > > - The resulting libraries are only
> compatible
> > with
> > >> Scala
> > >> > 2.11
> > >> > > > > >
> > >> > > > > > Now you want to also publish
> versions
> > compatible
> > >> with Scala
> > >> > > > 2.12?
> > >> > > > > >
> > >> > > > > > If that's the case I think Maven
> could easily
> > add
> > >> multiple
> > >> > > > executions
> > >> > > > > > where each compile compiles to
> different
> > output
> > >> > directories:
> > >> > > > > > - Java --> target/classes
> > >> > > > > > - Scala 2.11 -->
target/classes-2.11
> > >> > > > > > - Scala 2.12 -->
target/classes-2.12
> > >> > > > > >
> > >> > > > > > Then the packaging would also need
> a second
> > >> execution ...
> > >> > > each
> > >> > > > of the
> > >> > > > > > executions bundling the classes and
> the
> > >> corresponding scala
> > >> > > > output.
> > >> > > > > > Ideally I would probably use maven
> > classifiers to
> > >> > distinguish
> > >> > > > the
> > >> > > > > > artifacts.
> > >> > > > > >
> > >> > > > > > <dependency>
> > >> > > > > >
> <groupId>org.apache.mahout</groupId>
> > >> > > > > >
> <artifactId>mahout-spark</artifactId>
> > >> > > > > >
<version>14.1-SNAPSHOT</version>
> > >> > > > > > <classifier>2.11</classifier>
> > >> > > > > > </dependency>
> > >> > > > > >
> > >> > > > > > Then it should all work in a normal
> maven
> > build. In
> > >> the
> > >> > > > > distributions you
> > >> > > > > > could also filter the versions
> according to
> > their
> > >> > > classifiers.
> > >> > > > > >
> > >> > > > > > So if this is the case, I could
> help you with
> > this.
> > >> > > > > >
> > >> > > > > > Chris
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > Am 16.04.20, 09:39 schrieb "Stevo
> Slavić" <
> > >> > ssla...@gmail.com
> > >> > > >:
> > >> > > > > >
> > >> > > > > > Disclaimer: I'm not active
> Mahout
> > maintainer
> > >> for quite
> > >> > a
> > >> > > > while,
> > >> > > > > have
> > >> > > > > > some
> > >> > > > > > historical perspective, take it
> with a
> > grain of
> > >> salt,
> > >> > > > could be
> > >> > > > > I'm
> > >> > > > > > missing
> > >> > > > > > the whole point you were
> approached for
> > by a
> > >> wide
> > >> > margin
> > >> > > of
> > >> > > > > error.
> > >> > > > > >
> > >> > > > > > At a point Mahout, some of its
> modules,
> > have
> > >> turned
> > >> > into
> > >> > > a
> > >> > > > scala
> > >> > > > > > library, and there was need to
> cross
> > publish
> > >> those
> > >> > > modules,
> > >> > > > > across
> > >> > > > > > different scala versions. Back
> than Maven
> > scala
> > >> plugin
> > >> > > > didn't
> > >> > > > > support
> > >> > > > > > cross
> > >> > > > > > publishing, it doesn't fit well
> with
> > Maven's
> > >> build
> > >> > > > lifecycle
> > >> > > > > concept
> > >> > > > > > (multiple compile phases - one
> for each
> > scala
> > >> version,
> > >> > > and
> > >> > > > what
> > >> > > > > not
> > >> > > > > > would
> > >> > > > > > be needed). Switching to sbt
> could have
> > solved
> > >> the
> > >> > > problem.
> > >> > > > > Switch was
> > >> > > > > > deemed to be too big task, even
> though
> > ages
> > >> have been
> > >> > > > spent on
> > >> > > > > trying
> > >> > > > > > to
> > >> > > > > > apply Maven (profiles) + bash
> scripts and
> > what
> > >> not to
> > >> > > > solve the
> > >> > > > > > problem.
> > >> > > > > > Trying to apply same approach
> over and
> > over
> > >> again and
> > >> > > > expecting
> > >> > > > > > different
> > >> > > > > > results is not smart, no expert
> can help
> > there.
> > >> Mahout
> > >> > > > > maintainers and
> > >> > > > > > contributors, should consider
> alternative
> > >> approach, one
> > >> > > of
> > >> > > > them
> > >> > > > > being
> > >> > > > > > switching to sbt - it's scala
> native,
> > supports
> > >> scala
> > >> > > cross
> > >> > > > > publishing,
> > >> > > > > > supports publishing Maven
> compatible
> > release
> > >> metadata
> > >> > and
> > >> > > > > binaries.
> > >> > > > > >
> > >> > > > > > Kind regards,
> > >> > > > > > Stevo Slavic.
> > >> > > > > >
> > >> > > > > > On Thu, Apr 16, 2020 at 9:15 AM
> > Christofer Dutz
> > >> <
> > >> > > > > > christofer.d...@c-ware.de>
> > >> > > > > > wrote:
> > >> > > > > >
> > >> > > > > > > Hi folks,
> > >> > > > > > >
> > >> > > > > > > my name is Chris and I’m
> involved in
> > quite a
> > >> lot of
> > >> > > > Apache
> > >> > > > > projects.
> > >> > > > > > > Justin approached me this
> morning,
> > asking me
> > >> if I
> > >> > could
> > >> > > > > perhaps help
> > >> > > > > > you.
> > >> > > > > > > He told me you were having
> trouble with
> > doing
> > >> Maven
> > >> > > > releases.
> > >> > > > > > >
> > >> > > > > > > As Maven releases are my
> specialty,
> > could you
> > >> please
> > >> > > > summarize
> > >> > > > > the
> > >> > > > > > issues
> > >> > > > > > > you are having?
> > >> > > > > > >
> > >> > > > > > > Chris
> > >> > > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > > >
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >>
> >
> >
>
>
>
