Chris, thank you for your help.. Yeah if you fork what's in master you can see what state it's in; we are in the #mahout channel in tge-asf slack and this is also a fine way to keep track of discussion.
We could file a JIRA ticket as well, however you prefer to work. Best Andrew On Thu, Apr 16, 2020 at 06:59 Christofer Dutz <christofer.d...@c-ware.de> wrote: > Hi Trevor, > > ok ... first of all ... the Mahout PMC is defining a "community > maintained" library which is not maintained by the mahout PMC?!?! > I thought at Apache everything is about Community over code. So is a > company driving the non-community stuff? > > But back to your build issues: > I had a look and I too encountered these comments and remarks and > sometimes patterns I recognized and could imagine why they were created. > Yes quite a bit of the build could be cleaned up and simplified a lot. > > So how about I create a fork and try to do a cleanup of the build. > Usually I also leave comments about what I do as I hope I'll not be the > only one maintaining a build and documenting things helps people feel more > confident. > > However in some cases I will have questions ... so would someone be > available on Slack for quick questions? > > Usually switching to another build system does solve some problems ... > mostly the reason to switch is that it solved the main problem that you are > having with the old. > However you usually notice too late that you get yourself a lot of new > problems. I remember doing some contract work for an insurance company and > they were totally down Maven-road but then had to build something with SBT > ... in the end I compiled the thing on my laptop, copied it to a USB stick > and told the people what was on the stick and that I'll be having a coffee > and will be back in 30 minutes. When I came back the sick wasn't at the > same place and the build problem was "solved" ;-) > > So I think it's quite good to stick to maven ... that is very mature, you > can do almost everything you want with it and it integrates perfectly into > the Apache infrastructure. > > But that's just my opinion. > > So if you want me to help, I'll be happy to be of assistance. > > > Chris > > > > Am 16.04.20, 15:28 schrieb "Trevor Grant" <trevor.d.gr...@gmail.com>: > > Hey Christopher, > > I would agree with what Stevo outlined but add some more context and a > couple related JIRA issues. > > For 0.14.0 We did a big refactor and finally moved the MapReduce based > Mahout all into what we called "community/" that is community > maintained, > which is to say, we're not maintaining it anymore (sunset began I > think in > 2015). > > But all of our POMs were so huge and fat because they'd been layered up > over the years by people coming and going and dropping in code. I > wouldn't > call these drive- bys, its just been over 10 years and people come and > go. > Such is the life of Apache Projects. So we had a situation where a lot > of > the old Map Reduce stuff and the POMs were considered "old-magic" no > one > really knew how it was all tied together, but we didn't want to mess > with > it for fear of breaking something in the "new" Mahout (aka Samsar) > which is > the Scala/Spark based library that it is now* (to others in the > community: > I know it runs on other engines, but for simplicity, I'm just calling > it > "runs-on-spark"). > > For 0.14.0 We decided to trim out as much of that which was possible. > We > did some major liposuction on POMs, re organized things, etc. This was > done > by commenting out a section, then seeing if it would still build. So > the > current release > _does_ build. And aside for some CLI driver issues which are outlined > in > [1], the project runs fairly smooth. (An SBT would probably solve [1], > I > believe Pat Ferrel has made his own SBT script to compile Mahout, which > solved that problem for them). > > The issue we ran into with the releases (and the reason I think you're > here), is that we also somewhere along the line commented out something > that was important to the release process. Hence why 0.14.0 released > source > only. > > Since 2008, there has been a lot of great work on generating plugins > for > doing Apache releases. Instead of the awkward hacks that made up the > old > poms (literally comments that said, "this is a hack, there's supposedly > something better coming from ..." dated like 2012), we would like to > do it > the "right way" and incorporate the appropriate plugins. > > Refactoring to SBT was _one_ proposed solution. We're also OK > continuing to > use Maven, and I agree with what you said about the cross compiling. We > actually have a script that just changes the scala version. We tried > using > the classifiers but there were issues in SBT, but the way you're > proposing > sounds a lot more pro than the route we were trying for. > > That said- we'd be OK just releasing one scala/spark version at a time. > But getting the convenience binaries to release/publish would be a > major > first step. > > Also, we really appreciate the help, > > tg > > > [1] > > https://issues.apache.org/jira/projects/MAHOUT/issues/MAHOUT-2093?filter=allopenissues > > > > On Thu, Apr 16, 2020 at 4:50 AM Christofer Dutz < > christofer.d...@c-ware.de> > wrote: > > > Hi Stevo, > > > > so let me summarize what I understood: > > > > - There are some modules in mahout that are built with Scala, some > with > > java and some with both (At least that's what I see when checking > out the > > project) > > - The current build uses Scala 2.11 to build the Scala code. > > - The resulting libraries are only compatible with Scala 2.11 > > > > Now you want to also publish versions compatible with Scala 2.12? > > > > If that's the case I think Maven could easily add multiple executions > > where each compile compiles to different output directories: > > - Java --> target/classes > > - Scala 2.11 --> target/classes-2.11 > > - Scala 2.12 --> target/classes-2.12 > > > > Then the packaging would also need a second execution ... each of the > > executions bundling the classes and the corresponding scala output. > > Ideally I would probably use maven classifiers to distinguish the > > artifacts. > > > > <dependency> > > <groupId>org.apache.mahout</groupId> > > <artifactId>mahout-spark</artifactId> > > <version>14.1-SNAPSHOT</version> > > <classifier>2.11</classifier> > > </dependency> > > > > Then it should all work in a normal maven build. In the > distributions you > > could also filter the versions according to their classifiers. > > > > So if this is the case, I could help you with this. > > > > Chris > > > > > > Am 16.04.20, 09:39 schrieb "Stevo Slavić" <ssla...@gmail.com>: > > > > Disclaimer: I'm not active Mahout maintainer for quite a while, > have > > some > > historical perspective, take it with a grain of salt, could be > I'm > > missing > > the whole point you were approached for by a wide margin of > error. > > > > At a point Mahout, some of its modules, have turned into a scala > > library, and there was need to cross publish those modules, > across > > different scala versions. Back than Maven scala plugin didn't > support > > cross > > publishing, it doesn't fit well with Maven's build lifecycle > concept > > (multiple compile phases - one for each scala version, and what > not > > would > > be needed). Switching to sbt could have solved the problem. > Switch was > > deemed to be too big task, even though ages have been spent on > trying > > to > > apply Maven (profiles) + bash scripts and what not to solve the > > problem. > > Trying to apply same approach over and over again and expecting > > different > > results is not smart, no expert can help there. Mahout > maintainers and > > contributors, should consider alternative approach, one of them > being > > switching to sbt - it's scala native, supports scala cross > publishing, > > supports publishing Maven compatible release metadata and > binaries. > > > > Kind regards, > > Stevo Slavic. > > > > On Thu, Apr 16, 2020 at 9:15 AM Christofer Dutz < > > christofer.d...@c-ware.de> > > wrote: > > > > > Hi folks, > > > > > > my name is Chris and I’m involved in quite a lot of Apache > projects. > > > Justin approached me this morning, asking me if I could > perhaps help > > you. > > > He told me you were having trouble with doing Maven releases. > > > > > > As Maven releases are my specialty, could you please summarize > the > > issues > > > you are having? > > > > > > Chris > > > > > > > > >
