*correction: wasn't working with Java 7 On Sat, Aug 6, 2016 at 10:17 PM, Raviteja Lokineni < raviteja.lokin...@gmail.com> wrote:
> Hi Andy, > > I ran the following tests as you have specified: > > - classify-wikipedia.sh > - Option 2 > - cluster-reuters.sh > - Option 1,2 > - classify-20newsgroups.sh > - Option 1 > > All these examples *ran successfully* on a cloudera quickstart vm 5.7. I > had to change the cluster JVM to 1.8 to make it work otherwise lucene was > failing with incompatible class major/minor version error (because lucene > 6.1.0 was built for JVM 1.8). > > On seeing that this patch wasn't working with Java 8, I was like why, why, > why? > > Thanks, > Raviteja > > On Sat, Aug 6, 2016 at 5:15 PM, Raviteja Lokineni < > raviteja.lokin...@gmail.com> wrote: > >> I will let you know by tomorrow. Will run them now. >> >> On Aug 6, 2016 5:13 PM, "Andrew Palumbo" <ap....@outlook.com> wrote: >> >>> We will likely move to Java 8 at some point of course, but I personally >>> would not be inclined to enforce it right now as most of our current new >>> work is Scala-based, and this (the lucene dep.) is only used in legacy >>> components. Admittedly though, one useful legacy component. Were you >>> able to get the examples to run in pseudo-cluster mode with lucene 6? >>> >>> >>> Thanks, >>> >>> >>> Andy >>> >>> >>> ________________________________ >>> From: Andrew Palumbo <ap....@outlook.com> >>> Sent: Saturday, August 6, 2016 5:03:45 PM >>> To: dev@mahout.apache.org >>> Subject: Re: MAHOUT-1876 - Lucene compatibility >>> >>> Thank you Raviteja, this is something that we will have to discuss. >>> >>> ________________________________ >>> From: Raviteja Lokineni <raviteja.lokin...@gmail.com> >>> Sent: Friday, August 5, 2016 11:41:09 PM >>> To: mahout >>> Subject: Re: MAHOUT-1876 - Lucene compatibility >>> >>> Guys, found an issue lucene 6.x is compatible only with Java 8. What's >>> the >>> plan for mahout compatibility? Do you guys want to call in a vote for >>> Java >>> compatibility? >>> >>> On Aug 5, 2016 4:58 PM, "Andrew Palumbo" <ap....@outlook.com> wrote: >>> >>> > Hi Raviteja, >>> > >>> > >>> > Since this upgrade affects the entire Mahout MapReduce text processing >>> > pipeline it is important to make sure that it is working in the end to >>> end >>> > examples. >>> > >>> > >>> > Could you please set up a Hadoop 2.4.1 pseudo cluster and run through >>> the >>> > previously mentioned examples? >>> > >>> > >>> > The instructions are here (this is from 2.7.1 but should be the same >>> for >>> > 2.4.1) : >>> > >>> > >>> > <https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/h >>> adoop-common/ >>> > SingleCluster.html>https://hadoop.apache.org/docs/r2.7.2/ >>> > hadoop-project-dist/hadoop-common/SingleCluster.html# >>> > Pseudo-Distributed_Operation >>> > >>> > >>> > Thanks very much, >>> > >>> > >>> > Andy >>> > >>> > >>> > >>> > ________________________________ >>> > From: Andrew Palumbo <ap....@outlook.com> >>> > Sent: Friday, August 5, 2016 2:38 PM >>> > To: dev@mahout.apache.org >>> > Subject: Re: MAHOUT-1876 - Lucene compatibility >>> > >>> > Ahh- yes I think we started removing MAHOUT_LOCAL capability I see the >>> > check for MAHOUT_LOCAL was removed in this commit: >>> > >>> > >>> > https://github.com/apache/mahout/commit/daad3a4ce618cbd05be468c4ce6e45 >>> > 1618f3a028 >>> > [https://avatars3.githubusercontent.com/u/692523?v=3&s=200]<https:// >>> > github.com/apache/mahout/commit/daad3a4ce618cbd05be468c4ce6e >>> 451618f3a028> >>> > >>> > MAHOUT-1665: Update hadoop commands in example scripts (akm) closes a… >>> · >>> > apache/mahout@daad3a4<https://github.com/apache/mahout/commit/ >>> > daad3a4ce618cbd05be468c4ce6e451618f3a028> >>> > github.com >>> > …pache/mahout#98 >>> > >>> > >>> > >>> > >>> > So it would make sense that you are seeing that Error in local mode. >>> > >>> > ________________________________ >>> > From: Raviteja Lokineni <raviteja.lokin...@gmail.com> >>> > Sent: Friday, August 5, 2016 2:28:08 PM >>> > To: mahout >>> > Subject: Re: MAHOUT-1876 - Lucene compatibility >>> > >>> > Nope in a Linux environment. >>> > >>> > On Aug 5, 2016 2:21 PM, "Suneel Marthi" <smar...@apache.org> wrote: >>> > >>> > > r u running this on windows prompt or in Cygwin. >>> > > >>> > > Suggest use Cygwin. >>> > > >>> > > On Fri, Aug 5, 2016 at 2:15 PM, Raviteja Lokineni < >>> > > raviteja.lokin...@gmail.com> wrote: >>> > > >>> > > > This is what I get. >>> > > > >>> > > > $ ./classify-20newsgroups.sh >>> > > > /home/lok268/projects/mahout/examples/bin/set-dfs-commands.sh: >>> line >>> > > > 36: /bin/hadoop: No such file or directory >>> > > > /home/lok268/projects/mahout/examples/bin/set-dfs-commands.sh: >>> line >>> > > > 38: [: too many arguments >>> > > > /home/lok268/projects/mahout/examples/bin/set-dfs-commands.sh: >>> line >>> > > > 43: [: -eq: unary operator expected >>> > > > Can't determine Hadoop version. >>> > > > >>> > > > >>> > > > On Fri, Aug 5, 2016 at 2:08 PM, Suneel Marthi <smar...@apache.org> >>> > > wrote: >>> > > > >>> > > > > u don't need a hadoop cluster for that, >>> > > > > >>> > > > > set MAHOUT_LOCAL=true >>> > > > > and u shuld be able to run locally >>> > > > > >>> > > > > On Fri, Aug 5, 2016 at 1:57 PM, Raviteja Lokineni < >>> > > > > raviteja.lokin...@gmail.com> wrote: >>> > > > > >>> > > > > > Hi Andrew, >>> > > > > > >>> > > > > > Looks like the examples don't seem to work unless on a hadoop >>> > > cluster. >>> > > > > If I >>> > > > > > get some time I will download a cloudera quickstart vm and >>> test it >>> > > out. >>> > > > > > >>> > > > > > Thanks, >>> > > > > > Raviteja >>> > > > > > >>> > > > > > On Fri, Aug 5, 2016 at 12:53 PM, Andrew Palumbo < >>> > ap....@outlook.com> >>> > > > > > wrote: >>> > > > > > >>> > > > > > > Thanks again Raviteja, >>> > > > > > > Tests pass in my Linux env as well. >>> > > > > > > >>> > > > > > > FYI, if the windows script has not yet been officially >>> deprecated >>> > > it >>> > > > > > > should be soon. >>> > > > > > > >>> > > > > > > As Suneel said, someone will merge it over the weekend. In >>> the >>> > > > > meantime >>> > > > > > > it would good to ensure that some of the examples are >>> working in >>> > > the >>> > > > > > > $MAHOUT_HOME/examples/bin dir. Could you try running >>> > > > > > > classify-wikipedia.sh option (2), cluster-reuters.sh option >>> (1) >>> > pr >>> > > > (2) >>> > > > > > and >>> > > > > > > classify-20newsgroups.sh option 1 in (pseudo)cluster mode if >>> > > > possible? >>> > > > > > > >>> > > > > > > This would to ensure that seq2sparse is working correctly >>> which >>> > > > relies >>> > > > > > > heavily on lucene. >>> > > > > > > >>> > > > > > > Thanks again for the great contribution. >>> > > > > > > >>> > > > > > > Andy >>> > > > > > > >>> > > > > > > >>> > > > > > > -------- Original message -------- >>> > > > > > > From: Raviteja Lokineni <raviteja.lokin...@gmail.com> >>> > > > > > > Date: 08/05/2016 12:42 PM (GMT-05:00) >>> > > > > > > To: mahout <dev@mahout.apache.org> >>> > > > > > > Subject: Re: MAHOUT-1876 - Lucene compatibility >>> > > > > > > >>> > > > > > > Just a FYI, all the tests are successful on windows too ;) >>> > > > > > > >>> > > > > > > On Fri, Aug 5, 2016 at 12:18 PM, Andrew Palumbo < >>> > > ap....@outlook.com> >>> > > > > > > wrote: >>> > > > > > > >>> > > > > > > > +1 >>> > > > > > > > >>> > > > > > > > ________________________________ >>> > > > > > > > From: Raviteja Lokineni <raviteja.lokin...@gmail.com> >>> > > > > > > > Sent: Friday, August 5, 2016 12:14:24 PM >>> > > > > > > > To: mahout >>> > > > > > > > Subject: Re: MAHOUT-1876 - Lucene compatibility >>> > > > > > > > >>> > > > > > > > Yay! for the heads up on merging. >>> > > > > > > > >>> > > > > > > > FYI, I take back my word on failure on windows though. I >>> had to >>> > > > > include >>> > > > > > > the >>> > > > > > > > hadoop.dll file on PATH. Tests are running (I am running it >>> > just >>> > > to >>> > > > > > > >>> > > > > > >>> > > > > > >>> > > > > > >>> > > > > > -- >>> > > > > > *Raviteja Lokineni* | Business Intelligence Developer >>> > > > > > TD Ameritrade >>> > > > > > >>> > > > > > E: raviteja.lokin...@gmail.com >>> > > > > > >>> > > > > > [image: View Raviteja Lokineni's profile on LinkedIn] >>> > > > > > <http://in.linkedin.com/in/ravitejalokineni> >>> > > > > > >>> > > > > >>> > > > >>> > > > >>> > > > >>> > > > -- >>> > > > *Raviteja Lokineni* | Business Intelligence Developer >>> > > > TD Ameritrade >>> > > > >>> > > > E: raviteja.lokin...@gmail.com >>> > > > >>> > > > [image: View Raviteja Lokineni's profile on LinkedIn] >>> > > > <http://in.linkedin.com/in/ravitejalokineni> >>> > > > >>> > > >>> > >>> >> > > > -- > *Raviteja Lokineni* | Business Intelligence Developer > TD Ameritrade > > E: raviteja.lokin...@gmail.com > > [image: View Raviteja Lokineni's profile on LinkedIn] > <http://in.linkedin.com/in/ravitejalokineni> > > -- *Raviteja Lokineni* | Business Intelligence Developer TD Ameritrade E: raviteja.lokin...@gmail.com [image: View Raviteja Lokineni's profile on LinkedIn] <http://in.linkedin.com/in/ravitejalokineni>
