Hi Folks,
so I've now tested the build with java 1.8, 9, 10 and they work ...
11 I'm getting errors about unsupported java major versions again ...
guess there's some old library version in there somewhere.
But at least I managed to get you out of this 1.7 trap.
Chris
Am 20.04.20, 09:46 schrieb "Christofer Dutz" <christofer.d...@c-ware.de>:
Hi folks,
so I was now able to build (including all tests) with Java 8 and 9 ...
currently trying 10 ...
Are there any objection that some maven dependencies get updated to more
recent versions? I mean ... the hbase-client you're using is more than 5 years
old ...
Chris
Am 20.04.20, 00:29 schrieb "Andrew Musselman" <andrew.mussel...@gmail.com>:
No problem; would 10:00 a.m. Pacific next Friday the 24th work for you
time
zone-wise?
On Sun, Apr 19, 2020 at 12:15 PM Christofer Dutz
<christofer.d...@c-ware.de>
wrote:
> Sorry ...
>
> didn't see your response ... into Mahout too deep ;-)
> Guess we have to postpone this to sometime over the week.
>
> Chris
>
>
> Am 19.04.20, 19:56 schrieb "Andrew Musselman"
<andrew.mussel...@gmail.com
> >:
>
> *what time
>
> On Sun, Apr 19, 2020 at 10:51 Andrew Musselman <
> andrew.mussel...@gmail.com>
> wrote:
>
> > I think it's safe to move to 1.8; yeah what tune is good for
you?
> I'm in
> > Pacific time zone and am flexible this afternoon.
> >
> > Trevor you free?
> >
> > On Sun, Apr 19, 2020 at 09:37 Christofer Dutz <
> christofer.d...@c-ware.de>
> > wrote:
> >
> >> Yikes! ... well guess then I can't help you folks as it's
almost
> >> impossible to get my hands on a 1.7 version.
> >>
> >> What's preventing you from going to 1.8+?
> >>
> >> Any your build says 1.8 and above:
> >> <requireJavaVersion>
> >> <version>[1.8,)</version>
> >> </requireJavaVersion>
> >>
> >> Regarding the artifacts ... would it be ok to have the maven
> artifacts
> >> using classifiers and perhaps the files in the libs and
> distribution to
> >> follow the typical scala sheme?
> >>
> >> Chris
> >>
> >> Am 19.04.20, 17:56 schrieb "Trevor Grant"
<trevor.d.gr...@gmail.com
> >:
> >>
> >> Yea, we have a requirement on 1.7. we need to get it up to
8,
> but
> >> considered that a different issue.
> >>
> >> Maven throws warning, sbt breaks down entirely (when
importing)-
> >> hence why
> >> we were using a script to replace 2.11 w 2.12
> >>
> >> On Sun, Apr 19, 2020, 10:33 AM Christofer Dutz <
> >> christofer.d...@c-ware.de>
> >> wrote:
> >>
> >> > Well I have been compiling with various jdks from 8 to
14.
> However I
> >> > noticed that if I select a jdk and start the build that
the
> Scala
> >> parts
> >> > Seefahrt to the version in my systems path. Which then
> results in
> >> all sorts
> >> > of major version problems.
> >> >
> >> > For the Scalatest plugin I managed to have it use the
maven
> >> java.home...
> >> > The Scala compiler is being a little trickyer.
> >> >
> >> > Will continue this evening. Keep your fingers crossed.
> >> >
> >> > Also does maven totally not like variables in
artifactids.
> >> >
> >> > Chris
> >> > ________________________________
> >> > Von: Trevor Grant <trevor.d.gr...@gmail.com>
> >> > Gesendet: Sonntag, 19. April 2020 15:41
> >> > An: Mahout Dev List <dev@mahout.apache.org>
> >> > Betreff: Re: Hi ... need some help?
> >> >
> >> > Hey Chris,
> >> >
> >> > First of all- thank you so much for your work so far.
> >> >
> >> > The build 100% currently works, the only issue is with
> releasing
> >> binaries.
> >> > Are you compiling w 1.7? Not really sure why artifact
IDs are
> >> changing?
> >> > That's very strange.
> >> >
> >> > To answer your question regarding the community block-
no, it
> is
> >> not as if
> >> > there is a company that pays us to work on Mahout and the
> rest is
> >> > community. What it means is that while we appreciate
those
> code
> >> > contributions, no one still actively involved is
maintaining
> them
> >> and we
> >> > advise people to use at their own risk. The refactor to a
> >> `community`
> >> > folder was meant to reflect the reality of the situation.
> >> >
> >> > The only test issues I was aware of were with Javadocs
when
> trying
> >> to do a
> >> > release with Java 1.8+ those would fail due to
formatting.
> >> >
> >> > I feel like you're hitting some unique issues, and maybe
we
> could
> >> help you
> >> > out / there may exist simple solutions.
> >> >
> >> > Thank you again for your help!
> >> >
> >> > tg
> >> >
> >> >
> >> >
> >> >
> >> > On Sun, Apr 19, 2020 at 2:17 AM Christofer Dutz <
> >> christofer.d...@c-ware.de
> >> > >
> >> > wrote:
> >> >
> >> > > Hi folks,
> >> > >
> >> > > So yesterday I invested several hours in cleaning up
your
> build.
> >> I got
> >> > > quite far, currently trying to get the tests to pass.
For
> some
> >> > > Scala-Java-major-version problems, but working on
fixing
> them.
> >> > >
> >> > > However sind things will be different. I hope that's
OK.
> >> Especially with
> >> > > the artifact ids.
> >> > >
> >> > > Just as a question: does the build currently work at
all?
> >> Especially in
> >> > > the community block I had to fix quite a few API
changes
> where
> >> the code
> >> > in
> >> > > the blocks were for older versions of libraries.
> >> > >
> >> > > I hope I get some more tests to pass today.
> >> > >
> >> > > Chris
> >> > > ________________________________
> >> > > Von: Andrew Musselman <a...@apache.org>
> >> > > Gesendet: Donnerstag, 16. April 2020 20:35
> >> > > An: Mahout Dev List <dev@mahout.apache.org>
> >> > > Betreff: Re: Hi ... need some help?
> >> > >
> >> > > Looking forward to working on this with you; thanks
again!
> >> > >
> >> > > On Thu, Apr 16, 2020 at 11:09 AM Christofer Dutz <
> >> > > christofer.d...@c-ware.de>
> >> > > wrote:
> >> > >
> >> > > > Hi Andrew,
> >> > > >
> >> > > > guess I'll start with the fork and contact you folks
on
> slack.
> >> > > >
> >> > > > Chris
> >> > > >
> >> > > >
> >> > > >
> >> > > > Am 16.04.20, 19:43 schrieb "Andrew Musselman" <
> a...@apache.org>:
> >> > > >
> >> > > > Chris, thank you for your help..
> >> > > >
> >> > > > Yeah if you fork what's in master you can see
what
> state
> >> it's in;
> >> > we
> >> > > > are in
> >> > > > the #mahout channel in tge-asf slack and this is
also
> a
> >> fine way to
> >> > > > keep
> >> > > > track of discussion.
> >> > > >
> >> > > > We could file a JIRA ticket as well, however you
> prefer to
> >> work.
> >> > > >
> >> > > > Best
> >> > > > Andrew
> >> > > >
> >> > > > On Thu, Apr 16, 2020 at 06:59 Christofer Dutz <
> >> > > > christofer.d...@c-ware.de>
> >> > > > wrote:
> >> > > >
> >> > > > > Hi Trevor,
> >> > > > >
> >> > > > > ok ... first of all ... the Mahout PMC is
defining a
> >> "community
> >> > > > > maintained" library which is not maintained by
the
> mahout
> >> PMC?!?!
> >> > > > > I thought at Apache everything is about
Community
> over
> >> code. So
> >> > is
> >> > > a
> >> > > > > company driving the non-community stuff?
> >> > > > >
> >> > > > > But back to your build issues:
> >> > > > > I had a look and I too encountered these
comments
> and
> >> remarks and
> >> > > > > sometimes patterns I recognized and could
imagine
> why
> >> they were
> >> > > > created.
> >> > > > > Yes quite a bit of the build could be cleaned
up and
> >> simplified a
> >> > > > lot.
> >> > > > >
> >> > > > > So how about I create a fork and try to do a
> cleanup of
> >> the
> >> > build.
> >> > > > > Usually I also leave comments about what I do
as I
> hope
> >> I'll not
> >> > be
> >> > > > the
> >> > > > > only one maintaining a build and documenting
things
> helps
> >> people
> >> > > > feel more
> >> > > > > confident.
> >> > > > >
> >> > > > > However in some cases I will have questions
... so
> would
> >> someone
> >> > be
> >> > > > > available on Slack for quick questions?
> >> > > > >
> >> > > > > Usually switching to another build system does
> solve some
> >> > problems
> >> > > > ...
> >> > > > > mostly the reason to switch is that it solved
the
> main
> >> problem
> >> > that
> >> > > > you are
> >> > > > > having with the old.
> >> > > > > However you usually notice too late that you
get
> yourself
> >> a lot
> >> > of
> >> > > > new
> >> > > > > problems. I remember doing some contract work
for an
> >> insurance
> >> > > > company and
> >> > > > > they were totally down Maven-road but then had
to
> build
> >> something
> >> > > > with SBT
> >> > > > > ... in the end I compiled the thing on my
laptop,
> copied
> >> it to a
> >> > > USB
> >> > > > stick
> >> > > > > and told the people what was on the stick and
that
> I'll be
> >> > having a
> >> > > > coffee
> >> > > > > and will be back in 30 minutes. When I came
back
> the sick
> >> wasn't
> >> > at
> >> > > > the
> >> > > > > same place and the build problem was "solved"
;-)
> >> > > > >
> >> > > > > So I think it's quite good to stick to maven
...
> that is
> >> very
> >> > > > mature, you
> >> > > > > can do almost everything you want with it and
it
> >> integrates
> >> > > > perfectly into
> >> > > > > the Apache infrastructure.
> >> > > > >
> >> > > > > But that's just my opinion.
> >> > > > >
> >> > > > > So if you want me to help, I'll be happy to be
of
> >> assistance.
> >> > > > >
> >> > > > >
> >> > > > > Chris
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > Am 16.04.20, 15:28 schrieb "Trevor Grant" <
> >> > > trevor.d.gr...@gmail.com
> >> > > > >:
> >> > > > >
> >> > > > > Hey Christopher,
> >> > > > >
> >> > > > > I would agree with what Stevo outlined but
add
> some
> >> more
> >> > > context
> >> > > > and a
> >> > > > > couple related JIRA issues.
> >> > > > >
> >> > > > > For 0.14.0 We did a big refactor and
finally
> moved the
> >> > > MapReduce
> >> > > > based
> >> > > > > Mahout all into what we called "community/"
> that is
> >> community
> >> > > > > maintained,
> >> > > > > which is to say, we're not maintaining it
> anymore
> >> (sunset
> >> > > began I
> >> > > > > think in
> >> > > > > 2015).
> >> > > > >
> >> > > > > But all of our POMs were so huge and fat
because
> >> they'd been
> >> > > > layered up
> >> > > > > over the years by people coming and going
and
> >> dropping in
> >> > > code. I
> >> > > > > wouldn't
> >> > > > > call these drive- bys, its just been over
10
> years
> >> and people
> >> > > > come and
> >> > > > > go.
> >> > > > > Such is the life of Apache Projects. So we
had a
> >> situation
> >> > > where
> >> > > > a lot
> >> > > > > of
> >> > > > > the old Map Reduce stuff and the POMs were
> considered
> >> > > > "old-magic" no
> >> > > > > one
> >> > > > > really knew how it was all tied together,
but we
> >> didn't want
> >> > to
> >> > > > mess
> >> > > > > with
> >> > > > > it for fear of breaking something in the
"new"
> Mahout
> >> (aka
> >> > > > Samsar)
> >> > > > > which is
> >> > > > > the Scala/Spark based library that it is
now*
> (to
> >> others in
> >> > the
> >> > > > > community:
> >> > > > > I know it runs on other engines, but for
> simplicity,
> >> I'm just
> >> > > > calling
> >> > > > > it
> >> > > > > "runs-on-spark").
> >> > > > >
> >> > > > > For 0.14.0 We decided to trim out as much
of
> that
> >> which was
> >> > > > possible.
> >> > > > > We
> >> > > > > did some major liposuction on POMs, re
organized
> >> things, etc.
> >> > > > This was
> >> > > > > done
> >> > > > > by commenting out a section, then seeing
if it
> would
> >> still
> >> > > > build. So
> >> > > > > the
> >> > > > > current release
> >> > > > > _does_ build. And aside for some CLI driver
> issues
> >> which are
> >> > > > outlined
> >> > > > > in
> >> > > > > [1], the project runs fairly smooth. (An
SBT
> would
> >> probably
> >> > > > solve [1],
> >> > > > > I
> >> > > > > believe Pat Ferrel has made his own SBT
script
> to
> >> compile
> >> > > > Mahout, which
> >> > > > > solved that problem for them).
> >> > > > >
> >> > > > > The issue we ran into with the releases
(and the
> >> reason I
> >> > think
> >> > > > you're
> >> > > > > here), is that we also somewhere along the
line
> >> commented out
> >> > > > something
> >> > > > > that was important to the release process.
> Hence why
> >> 0.14.0
> >> > > > released
> >> > > > > source
> >> > > > > only.
> >> > > > >
> >> > > > > Since 2008, there has been a lot of great
work
> on
> >> generating
> >> > > > plugins
> >> > > > > for
> >> > > > > doing Apache releases. Instead of the
awkward
> hacks
> >> that made
> >> > > up
> >> > > > the
> >> > > > > old
> >> > > > > poms (literally comments that said, "this
is a
> hack,
> >> there's
> >> > > > supposedly
> >> > > > > something better coming from ..." dated
like
> 2012),
> >> we would
> >> > > > like to
> >> > > > > do it
> >> > > > > the "right way" and incorporate the
appropriate
> >> plugins.
> >> > > > >
> >> > > > > Refactoring to SBT was _one_ proposed
solution.
> We're
> >> also OK
> >> > > > > continuing to
> >> > > > > use Maven, and I agree with what you said
about
> the
> >> cross
> >> > > > compiling. We
> >> > > > > actually have a script that just changes
the
> scala
> >> version.
> >> > We
> >> > > > tried
> >> > > > > using
> >> > > > > the classifiers but there were issues in
SBT,
> but the
> >> way
> >> > > you're
> >> > > > > proposing
> >> > > > > sounds a lot more pro than the route we
were
> trying
> >> for.
> >> > > > >
> >> > > > > That said- we'd be OK just releasing one
> scala/spark
> >> version
> >> > at
> >> > > > a time.
> >> > > > > But getting the convenience binaries to
> >> release/publish would
> >> > > be
> >> > > > a
> >> > > > > major
> >> > > > > first step.
> >> > > > >
> >> > > > > Also, we really appreciate the help,
> >> > > > >
> >> > > > > tg
> >> > > > >
> >> > > > >
> >> > > > > [1]
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>
https://issues.apache.org/jira/projects/MAHOUT/issues/MAHOUT-2093?filter=allopenissues
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > On Thu, Apr 16, 2020 at 4:50 AM Christofer
Dutz
> <
> >> > > > > christofer.d...@c-ware.de>
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Hi Stevo,
> >> > > > > >
> >> > > > > > so let me summarize what I understood:
> >> > > > > >
> >> > > > > > - There are some modules in mahout that
are
> built
> >> with
> >> > Scala,
> >> > > > some
> >> > > > > with
> >> > > > > > java and some with both (At least that's
what
> I see
> >> when
> >> > > > checking
> >> > > > > out the
> >> > > > > > project)
> >> > > > > > - The current build uses Scala 2.11 to
build
> the
> >> Scala
> >> > code.
> >> > > > > > - The resulting libraries are only
compatible
> with
> >> Scala
> >> > 2.11
> >> > > > > >
> >> > > > > > Now you want to also publish versions
> compatible
> >> with Scala
> >> > > > 2.12?
> >> > > > > >
> >> > > > > > If that's the case I think Maven could
easily
> add
> >> multiple
> >> > > > executions
> >> > > > > > where each compile compiles to different
> output
> >> > directories:
> >> > > > > > - Java --> target/classes
> >> > > > > > - Scala 2.11 --> target/classes-2.11
> >> > > > > > - Scala 2.12 --> target/classes-2.12
> >> > > > > >
> >> > > > > > Then the packaging would also need a
second
> >> execution ...
> >> > > each
> >> > > > of the
> >> > > > > > executions bundling the classes and the
> >> corresponding scala
> >> > > > output.
> >> > > > > > Ideally I would probably use maven
> classifiers to
> >> > distinguish
> >> > > > the
> >> > > > > > artifacts.
> >> > > > > >
> >> > > > > > <dependency>
> >> > > > > > <groupId>org.apache.mahout</groupId>
> >> > > > > > <artifactId>mahout-spark</artifactId>
> >> > > > > > <version>14.1-SNAPSHOT</version>
> >> > > > > > <classifier>2.11</classifier>
> >> > > > > > </dependency>
> >> > > > > >
> >> > > > > > Then it should all work in a normal maven
> build. In
> >> the
> >> > > > > distributions you
> >> > > > > > could also filter the versions according
to
> their
> >> > > classifiers.
> >> > > > > >
> >> > > > > > So if this is the case, I could help you
with
> this.
> >> > > > > >
> >> > > > > > Chris
> >> > > > > >
> >> > > > > >
> >> > > > > > Am 16.04.20, 09:39 schrieb "Stevo
Slavić" <
> >> > ssla...@gmail.com
> >> > > >:
> >> > > > > >
> >> > > > > > Disclaimer: I'm not active Mahout
> maintainer
> >> for quite
> >> > a
> >> > > > while,
> >> > > > > have
> >> > > > > > some
> >> > > > > > historical perspective, take it with
a
> grain of
> >> salt,
> >> > > > could be
> >> > > > > I'm
> >> > > > > > missing
> >> > > > > > the whole point you were approached
for
> by a
> >> wide
> >> > margin
> >> > > of
> >> > > > > error.
> >> > > > > >
> >> > > > > > At a point Mahout, some of its
modules,
> have
> >> turned
> >> > into
> >> > > a
> >> > > > scala
> >> > > > > > library, and there was need to cross
> publish
> >> those
> >> > > modules,
> >> > > > > across
> >> > > > > > different scala versions. Back than
Maven
> scala
> >> plugin
> >> > > > didn't
> >> > > > > support
> >> > > > > > cross
> >> > > > > > publishing, it doesn't fit well with
> Maven's
> >> build
> >> > > > lifecycle
> >> > > > > concept
> >> > > > > > (multiple compile phases - one for
each
> scala
> >> version,
> >> > > and
> >> > > > what
> >> > > > > not
> >> > > > > > would
> >> > > > > > be needed). Switching to sbt could
have
> solved
> >> the
> >> > > problem.
> >> > > > > Switch was
> >> > > > > > deemed to be too big task, even
though
> ages
> >> have been
> >> > > > spent on
> >> > > > > trying
> >> > > > > > to
> >> > > > > > apply Maven (profiles) + bash
scripts and
> what
> >> not to
> >> > > > solve the
> >> > > > > > problem.
> >> > > > > > Trying to apply same approach over
and
> over
> >> again and
> >> > > > expecting
> >> > > > > > different
> >> > > > > > results is not smart, no expert can
help
> there.
> >> Mahout
> >> > > > > maintainers and
> >> > > > > > contributors, should consider
alternative
> >> approach, one
> >> > > of
> >> > > > them
> >> > > > > being
> >> > > > > > switching to sbt - it's scala native,
> supports
> >> scala
> >> > > cross
> >> > > > > publishing,
> >> > > > > > supports publishing Maven compatible
> release
> >> metadata
> >> > and
> >> > > > > binaries.
> >> > > > > >
> >> > > > > > Kind regards,
> >> > > > > > Stevo Slavic.
> >> > > > > >
> >> > > > > > On Thu, Apr 16, 2020 at 9:15 AM
> Christofer Dutz
> >> <
> >> > > > > > christofer.d...@c-ware.de>
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > > Hi folks,
> >> > > > > > >
> >> > > > > > > my name is Chris and I’m involved
in
> quite a
> >> lot of
> >> > > > Apache
> >> > > > > projects.
> >> > > > > > > Justin approached me this morning,
> asking me
> >> if I
> >> > could
> >> > > > > perhaps help
> >> > > > > > you.
> >> > > > > > > He told me you were having trouble
with
> doing
> >> Maven
> >> > > > releases.
> >> > > > > > >
> >> > > > > > > As Maven releases are my specialty,
> could you
> >> please
> >> > > > summarize
> >> > > > > the
> >> > > > > > issues
> >> > > > > > > you are having?
> >> > > > > > >
> >> > > > > > > Chris
> >> > > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > > >
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> >>
>
>
