Thanks Raviteja,
Someone will review the PR shortly. And these ran w/o issue for you in cluster mode, correct? Andy ________________________________ From: Raviteja Lokineni <raviteja.lokin...@gmail.com> Sent: Sunday, August 7, 2016 9:01:08 PM To: mahout Subject: Re: MAHOUT-1876 - Lucene compatibility Submitted another PR with Lucene 5.5.2 and Java 7 compatibility. Based on the devs preference we can choose one of patches. https://github.com/apache/mahout/pull/248 I did all the necessary tests specified above and all are successful. Thanks, Raviteja On Sat, Aug 6, 2016 at 7:20 PM, Raviteja Lokineni < raviteja.lokin...@gmail.com> wrote: > *correction: wasn't working with Java 7 > > On Sat, Aug 6, 2016 at 10:17 PM, Raviteja Lokineni < > raviteja.lokin...@gmail.com> wrote: > >> Hi Andy, >> >> I ran the following tests as you have specified: >> >> - classify-wikipedia.sh >> - Option 2 >> - cluster-reuters.sh >> - Option 1,2 >> - classify-20newsgroups.sh >> - Option 1 >> >> All these examples *ran successfully* on a cloudera quickstart vm 5.7. I >> had to change the cluster JVM to 1.8 to make it work otherwise lucene was >> failing with incompatible class major/minor version error (because lucene >> 6.1.0 was built for JVM 1.8). >> >> On seeing that this patch wasn't working with Java 8, I was like why, >> why, why? >> >> Thanks, >> Raviteja >> >> On Sat, Aug 6, 2016 at 5:15 PM, Raviteja Lokineni < >> raviteja.lokin...@gmail.com> wrote: >> >>> I will let you know by tomorrow. Will run them now. >>> >>> On Aug 6, 2016 5:13 PM, "Andrew Palumbo" <ap....@outlook.com> wrote: >>> >>>> We will likely move to Java 8 at some point of course, but I personally >>>> would not be inclined to enforce it right now as most of our current new >>>> work is Scala-based, and this (the lucene dep.) is only used in legacy >>>> components. Admittedly though, one useful legacy component. Were you >>>> able to get the examples to run in pseudo-cluster mode with lucene 6? >>>> >>>> >>>> Thanks, >>>> >>>> >>>> Andy >>>> >>>> >>>> ________________________________ >>>> From: Andrew Palumbo <ap....@outlook.com> >>>> Sent: Saturday, August 6, 2016 5:03:45 PM >>>> To: dev@mahout.apache.org >>>> Subject: Re: MAHOUT-1876 - Lucene compatibility >>>> >>>> Thank you Raviteja, this is something that we will have to discuss. >>>> >>>> ________________________________ >>>> From: Raviteja Lokineni <raviteja.lokin...@gmail.com> >>>> Sent: Friday, August 5, 2016 11:41:09 PM >>>> To: mahout >>>> Subject: Re: MAHOUT-1876 - Lucene compatibility >>>> >>>> Guys, found an issue lucene 6.x is compatible only with Java 8. What's >>>> the >>>> plan for mahout compatibility? Do you guys want to call in a vote for >>>> Java >>>> compatibility? >>>> >>>> On Aug 5, 2016 4:58 PM, "Andrew Palumbo" <ap....@outlook.com> wrote: >>>> >>>> > Hi Raviteja, >>>> > >>>> > >>>> > Since this upgrade affects the entire Mahout MapReduce text processing >>>> > pipeline it is important to make sure that it is working in the end >>>> to end >>>> > examples. >>>> > >>>> > >>>> > Could you please set up a Hadoop 2.4.1 pseudo cluster and run through >>>> the >>>> > previously mentioned examples? >>>> > >>>> > >>>> > The instructions are here (this is from 2.7.1 but should be the same >>>> for >>>> > 2.4.1) : >>>> > >>>> > >>>> > <https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/h >>>> adoop-common/ >>>> > SingleCluster.html>https://hadoop.apache.org/docs/r2.7.2/ >>>> > hadoop-project-dist/hadoop-common/SingleCluster.html# >>>> > Pseudo-Distributed_Operation >>>> > >>>> > >>>> > Thanks very much, >>>> > >>>> > >>>> > Andy >>>> > >>>> > >>>> > >>>> > ________________________________ >>>> > From: Andrew Palumbo <ap....@outlook.com> >>>> > Sent: Friday, August 5, 2016 2:38 PM >>>> > To: dev@mahout.apache.org >>>> > Subject: Re: MAHOUT-1876 - Lucene compatibility >>>> > >>>> > Ahh- yes I think we started removing MAHOUT_LOCAL capability I see the >>>> > check for MAHOUT_LOCAL was removed in this commit: >>>> > >>>> > >>>> > https://github.com/apache/mahout/commit/daad3a4ce618cbd05be4 >>>> 68c4ce6e45 >>>> > 1618f3a028 >>>> > [https://avatars3.githubusercontent.com/u/692523?v=3&s=200]<https:// >>>> > github.com/apache/mahout/commit/daad3a4ce618cbd05be468c4ce6e >>>> 451618f3a028> >>>> > >>>> > MAHOUT-1665: Update hadoop commands in example scripts (akm) closes >>>> a… · >>>> > apache/mahout@daad3a4<https://github.com/apache/mahout/commit/ >>>> > daad3a4ce618cbd05be468c4ce6e451618f3a028> >>>> > github.com >>>> > …pache/mahout#98 >>>> > >>>> > >>>> > >>>> > >>>> > So it would make sense that you are seeing that Error in local mode. >>>> > >>>> > ________________________________ >>>> > From: Raviteja Lokineni <raviteja.lokin...@gmail.com> >>>> > Sent: Friday, August 5, 2016 2:28:08 PM >>>> > To: mahout >>>> > Subject: Re: MAHOUT-1876 - Lucene compatibility >>>> > >>>> > Nope in a Linux environment. >>>> > >>>> > On Aug 5, 2016 2:21 PM, "Suneel Marthi" <smar...@apache.org> wrote: >>>> > >>>> > > r u running this on windows prompt or in Cygwin. >>>> > > >>>> > > Suggest use Cygwin. >>>> > > >>>> > > On Fri, Aug 5, 2016 at 2:15 PM, Raviteja Lokineni < >>>> > > raviteja.lokin...@gmail.com> wrote: >>>> > > >>>> > > > This is what I get. >>>> > > > >>>> > > > $ ./classify-20newsgroups.sh >>>> > > > /home/lok268/projects/mahout/examples/bin/set-dfs-commands.sh: >>>> line >>>> > > > 36: /bin/hadoop: No such file or directory >>>> > > > /home/lok268/projects/mahout/examples/bin/set-dfs-commands.sh: >>>> line >>>> > > > 38: [: too many arguments >>>> > > > /home/lok268/projects/mahout/examples/bin/set-dfs-commands.sh: >>>> line >>>> > > > 43: [: -eq: unary operator expected >>>> > > > Can't determine Hadoop version. >>>> > > > >>>> > > > >>>> > > > On Fri, Aug 5, 2016 at 2:08 PM, Suneel Marthi <smar...@apache.org >>>> > >>>> > > wrote: >>>> > > > >>>> > > > > u don't need a hadoop cluster for that, >>>> > > > > >>>> > > > > set MAHOUT_LOCAL=true >>>> > > > > and u shuld be able to run locally >>>> > > > > >>>> > > > > On Fri, Aug 5, 2016 at 1:57 PM, Raviteja Lokineni < >>>> > > > > raviteja.lokin...@gmail.com> wrote: >>>> > > > > >>>> > > > > > Hi Andrew, >>>> > > > > > >>>> > > > > > Looks like the examples don't seem to work unless on a hadoop >>>> > > cluster. >>>> > > > > If I >>>> > > > > > get some time I will download a cloudera quickstart vm and >>>> test it >>>> > > out. >>>> > > > > > >>>> > > > > > Thanks, >>>> > > > > > Raviteja >>>> > > > > > >>>> > > > > > On Fri, Aug 5, 2016 at 12:53 PM, Andrew Palumbo < >>>> > ap....@outlook.com> >>>> > > > > > wrote: >>>> > > > > > >>>> > > > > > > Thanks again Raviteja, >>>> > > > > > > Tests pass in my Linux env as well. >>>> > > > > > > >>>> > > > > > > FYI, if the windows script has not yet been officially >>>> deprecated >>>> > > it >>>> > > > > > > should be soon. >>>> > > > > > > >>>> > > > > > > As Suneel said, someone will merge it over the weekend. In >>>> the >>>> > > > > meantime >>>> > > > > > > it would good to ensure that some of the examples are >>>> working in >>>> > > the >>>> > > > > > > $MAHOUT_HOME/examples/bin dir. Could you try running >>>> > > > > > > classify-wikipedia.sh option (2), cluster-reuters.sh >>>> option (1) >>>> > pr >>>> > > > (2) >>>> > > > > > and >>>> > > > > > > classify-20newsgroups.sh option 1 in (pseudo)cluster mode if >>>> > > > possible? >>>> > > > > > > >>>> > > > > > > This would to ensure that seq2sparse is working correctly >>>> which >>>> > > > relies >>>> > > > > > > heavily on lucene. >>>> > > > > > > >>>> > > > > > > Thanks again for the great contribution. >>>> > > > > > > >>>> > > > > > > Andy >>>> > > > > > > >>>> > > > > > > >>>> > > > > > > -------- Original message -------- >>>> > > > > > > From: Raviteja Lokineni <raviteja.lokin...@gmail.com> >>>> > > > > > > Date: 08/05/2016 12:42 PM (GMT-05:00) >>>> > > > > > > To: mahout <dev@mahout.apache.org> >>>> > > > > > > Subject: Re: MAHOUT-1876 - Lucene compatibility >>>> > > > > > > >>>> > > > > > > Just a FYI, all the tests are successful on windows too ;) >>>> > > > > > > >>>> > > > > > > On Fri, Aug 5, 2016 at 12:18 PM, Andrew Palumbo < >>>> > > ap....@outlook.com> >>>> > > > > > > wrote: >>>> > > > > > > >>>> > > > > > > > +1 >>>> > > > > > > > >>>> > > > > > > > ________________________________ >>>> > > > > > > > From: Raviteja Lokineni <raviteja.lokin...@gmail.com> >>>> > > > > > > > Sent: Friday, August 5, 2016 12:14:24 PM >>>> > > > > > > > To: mahout >>>> > > > > > > > Subject: Re: MAHOUT-1876 - Lucene compatibility >>>> > > > > > > > >>>> > > > > > > > Yay! for the heads up on merging. >>>> > > > > > > > >>>> > > > > > > > FYI, I take back my word on failure on windows though. I >>>> had to >>>> > > > > include >>>> > > > > > > the >>>> > > > > > > > hadoop.dll file on PATH. Tests are running (I am running >>>> it >>>> > just >>>> > > to >>>> > > > > > > >>>> > > > > > >>>> > > > > > >>>> > > > > > >>>> > > > > > -- >>>> > > > > > *Raviteja Lokineni* | Business Intelligence Developer >>>> > > > > > TD Ameritrade >>>> > > > > > >>>> > > > > > E: raviteja.lokin...@gmail.com >>>> > > > > > >>>> > > > > > [image: View Raviteja Lokineni's profile on LinkedIn] >>>> > > > > > <http://in.linkedin.com/in/ravitejalokineni> >>>> > > > > > >>>> > > > > >>>> > > > >>>> > > > >>>> > > > >>>> > > > -- >>>> > > > *Raviteja Lokineni* | Business Intelligence Developer >>>> > > > TD Ameritrade >>>> > > > >>>> > > > E: raviteja.lokin...@gmail.com >>>> > > > >>>> > > > [image: View Raviteja Lokineni's profile on LinkedIn] >>>> > > > <http://in.linkedin.com/in/ravitejalokineni> >>>> > > > >>>> > > >>>> > >>>> >>> >> >> >> -- >> *Raviteja Lokineni* | Business Intelligence Developer >> TD Ameritrade >> >> E: raviteja.lokin...@gmail.com >> >> [image: View Raviteja Lokineni's profile on LinkedIn] >> <http://in.linkedin.com/in/ravitejalokineni> >> >> > > > -- > *Raviteja Lokineni* | Business Intelligence Developer > TD Ameritrade > > E: raviteja.lokin...@gmail.com > > [image: View Raviteja Lokineni's profile on LinkedIn] > <http://in.linkedin.com/in/ravitejalokineni> > > -- *Raviteja Lokineni* | Business Intelligence Developer TD Ameritrade E: raviteja.lokin...@gmail.com [image: View Raviteja Lokineni's profile on LinkedIn] <http://in.linkedin.com/in/ravitejalokineni>
