Just a FYI, all the tests are successful on windows too ;) On Fri, Aug 5, 2016 at 12:18 PM, Andrew Palumbo <ap....@outlook.com> wrote:
> +1 > > ________________________________ > From: Raviteja Lokineni <raviteja.lokin...@gmail.com> > Sent: Friday, August 5, 2016 12:14:24 PM > To: mahout > Subject: Re: MAHOUT-1876 - Lucene compatibility > > Yay! for the heads up on merging. > > FYI, I take back my word on failure on windows though. I had to include the > hadoop.dll file on PATH. Tests are running (I am running it just to satisfy > myself ;) ). > > On Fri, Aug 5, 2016 at 12:11 PM, Suneel Marthi <suneel.mar...@gmail.com> > wrote: > > > Most of us r Linux or MacOS users, so we never hit any of the build > issues > > u have encountered. > > I must say that we never test for Windows :), but its expected that u > would > > run into issues on Windows. > > > > The patch looks good and we shuld be able to merge this over the weekend, > > thanks again for the contrib. > > > > > > > > On Fri, Aug 5, 2016 at 12:01 PM, Raviteja Lokineni < > > raviteja.lokin...@gmail.com> wrote: > > > > > I have built winutils.exe but nevertheless the test fails to run on > > windows > > > due to linker errors. > > > > > > So, for now linux builds are successful on my end. Let me know what is > > your > > > plan with the pull request to upgrade. > > > > > > On Fri, Aug 5, 2016 at 11:12 AM, Raviteja Lokineni < > > > raviteja.lokin...@gmail.com> wrote: > > > > > > > The reason the tests failed previously on my windows box is because > it > > > > wasn't able to locate winutils.exe in HADOOP_HOME. > > > > > > > > Now, to test that I am building winutils.exe from source using the > > > > following resource: http://zutai.blogspot.com/2014/06/build- > > > > install-and-run-hadoop-24-240-on.html > > > > > > > > On Thu, Aug 4, 2016 at 9:53 PM, Raviteja Lokineni < > > > > raviteja.lokin...@gmail.com> wrote: > > > > > > > >> I'm up for the challenge but looks like nothing failed this time > too. > > > >> > > > >> Repo: https://github.com/bond-/mahout > > > >> Branch: mahout-1876 > > > >> > > > >> See the full output in the gist: https://gist.github.com/ > > > >> bond-/33c349136b6914db4892233b8876b9d7 > > > >> > > > >> If someone wants to re-verify running the tests, please go ahead and > > let > > > >> me know. > > > >> > > > >> On Thu, Aug 4, 2016 at 6:16 PM, Andrew Palumbo <ap....@outlook.com> > > > >> wrote: > > > >> > > > >>> Maven >= 3.3.3, > > > >>> java >= 1.7, > > > >>> MAHOUT_HOME=/path/to/install, > > > >>> > > > >>> also MAHOUT_LOCAL=true > > > >>> > > > >>> If ur not running against a hadoop cluster. (Looks like ur not) > > > >>> > > > >>> Upgrading this dep is not a trivial task. > > > >>> > > > >>> > > > >>> -------- Original message -------- > > > >>> From: Raviteja Lokineni <raviteja.lokin...@gmail.com> > > > >>> Date: 08/04/2016 6:09 PM (GMT-05:00) > > > >>> To: mahout <dev@mahout.apache.org> > > > >>> Subject: Re: MAHOUT-1876 - Lucene compatibility > > > >>> > > > >>> Are there any pre-requisites to run tests on local machines? > > > >>> > > > >>> Most of the MR module tests failed with: > > > >>> > > > >>> java.lang.NullPointerException: null > > > >>> at __randomizedtesting.SeedInfo.seed([C8C254E3D9D80CC9:845EFDB0 > > > >>> 7D41873E]:0) > > > >>> at java.lang.ProcessBuilder.start(ProcessBuilder.java:1012) > > > >>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:445) > > > >>> at org.apache.hadoop.util.Shell.run(Shell.java:418) > > > >>> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Sh > > > >>> ell.java:650) > > > >>> at org.apache.hadoop.util.Shell.execCommand(Shell.java:739) > > > >>> at org.apache.hadoop.util.Shell.execCommand(Shell.java:722) > > > >>> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission( > > > >>> RawLocalFileSystem.java:633) > > > >>> at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs( > > > >>> RawLocalFileSystem.java:421) > > > >>> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSyste > > > >>> m.java:281) > > > >>> at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir( > > > >>> JobSubmissionFiles.java:125) > > > >>> at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal( > > > >>> JobSubmitter.java:348) > > > >>> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285) > > > >>> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282) > > > >>> at java.security.AccessController.doPrivileged(Native Method) > > > >>> at javax.security.auth.Subject.doAs(Subject.java:422) > > > >>> at org.apache.hadoop.security.UserGroupInformation.doAs( > > > >>> UserGroupInformation.java:1556) > > > >>> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282) > > > >>> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job. > java:1303) > > > >>> at org.apache.mahout.cf.taste.hadoop.preparation.PreparePrefere > > > >>> nceMatrixJob. > > > >>> run(PreparePreferenceMatrixJob.java:77) > > > >>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > > > >>> at org.apache.mahout.cf.taste.hadoop.item.RecommenderJob. > > > >>> run(RecommenderJob.java:168) > > > >>> at org.apache.mahout.cf.taste.hadoop.item.RecommenderJobTest. > > > >>> testCompleteJobWithFiltering(RecommenderJobTest.java:881) > > > >>> > > > >>> On Thu, Aug 4, 2016 at 6:03 PM, Andrew Palumbo <ap....@outlook.com > > > > > >>> wrote: > > > >>> > > > >>> > Actually, Travis CI is not set up to test all mahout modules. > > Could > > > >>> you > > > >>> > please test again on your local machine, and report any errors. > > > >>> > > > > >>> > Andy > > > >>> > > > > >>> > > > > >>> > -------- Original message -------- > > > >>> > From: Raviteja Lokineni <raviteja.lokin...@gmail.com> > > > >>> > Date: 08/04/2016 4:01 PM (GMT-05:00) > > > >>> > To: mahout <dev@mahout.apache.org> > > > >>> > Subject: Re: MAHOUT-1876 - Lucene compatibility > > > >>> > > > > >>> > All the tests are successful even with my code change and upgrade > > to > > > >>> > Lucene. (The test failure that I reported above was on my local > > > >>> machine, I > > > >>> > guess you can ignore that.) > > > >>> > > > > >>> > So is this pull request good to merge? > > > >>> > > > > >>> > https://github.com/apache/mahout/pull/247 > > > >>> > > > > >>> > https://travis-ci.org/apache/mahout/builds/149840408 > > > >>> > > > > >>> > On Thu, Aug 4, 2016 at 2:05 PM, Raviteja Lokineni < > > > >>> > raviteja.lokin...@gmail.com> wrote: > > > >>> > > > > >>> > > Most of the MR tests are failing with: > > > >>> > > > > > >>> > > java.lang.NullPointerException: null > > > >>> > > at __randomizedtesting.SeedInfo.seed([C8C254E3D9D80CC9: > > > >>> > > 845EFDB07D41873E]:0) > > > >>> > > at java.lang.ProcessBuilder.start(ProcessBuilder.java:1012) > > > >>> > > at org.apache.hadoop.util.Shell.runCommand(Shell.java:445) > > > >>> > > at org.apache.hadoop.util.Shell.run(Shell.java:418) > > > >>> > > at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute( > > > >>> > > Shell.java:650) > > > >>> > > at org.apache.hadoop.util.Shell.execCommand(Shell.java:739) > > > >>> > > at org.apache.hadoop.util.Shell.execCommand(Shell.java:722) > > > >>> > > at org.apache.hadoop.fs.RawLocalFileSystem.setPermission( > > > >>> > > RawLocalFileSystem.java:633) > > > >>> > > at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs( > > > >>> > > RawLocalFileSystem.java:421) > > > >>> > > at org.apache.hadoop.fs.FilterFileSystem.mkdirs( > > > >>> > FilterFileSystem.java:281) > > > >>> > > at org.apache.hadoop.mapreduce.JobSubmissionFiles. > getStagingDir( > > > >>> > > JobSubmissionFiles.java:125) > > > >>> > > at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal( > > > >>> > > JobSubmitter.java:348) > > > >>> > > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285) > > > >>> > > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282) > > > >>> > > at java.security.AccessController.doPrivileged(Native Method) > > > >>> > > at javax.security.auth.Subject.doAs(Subject.java:422) > > > >>> > > at org.apache.hadoop.security.UserGroupInformation.doAs( > > > >>> > > UserGroupInformation.java:1556) > > > >>> > > at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282) > > > >>> > > at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job. > > > java:1303) > > > >>> > > at org.apache.mahout.cf.taste.hadoop.preparation. > > > >>> > > PreparePreferenceMatrixJob.run(PreparePreferenceMatrixJob. > > java:77) > > > >>> > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > > > >>> > > at org.apache.mahout.cf.taste.hadoop.item.RecommenderJob. > > > >>> > > run(RecommenderJob.java:168) > > > >>> > > at org.apache.mahout.cf.taste.hadoop.item.RecommenderJobTest. > > > >>> > > testCompleteJobWithFiltering(RecommenderJobTest.java:881) > > > >>> > > > > > >>> > > Do I need to have hadoop installed on my local machine? > > > >>> > > > > > >>> > > On Thu, Aug 4, 2016 at 1:53 PM, Raviteja Lokineni < > > > >>> > > raviteja.lokin...@gmail.com> wrote: > > > >>> > > > > > >>> > >> Hi Devs, > > > >>> > >> > > > >>> > >> Issue link: https://issues.apache.org/jira/browse/MAHOUT-1876 > > > >>> > >> > > > >>> > >> Problem statement: Mahout should be compatible with the latest > > > >>> lucene > > > >>> > >> version. I was trying to solve a text clustering problem and I > > > >>> stumbled > > > >>> > >> upon the error that the lucene version in use is not > supported. > > > >>> That's > > > >>> > when > > > >>> > >> I raised this issue and one of the guys suggested me to try > > fixing > > > >>> it > > > >>> > and I > > > >>> > >> started doing so. > > > >>> > >> > > > >>> > >> Proposed solution: > > > >>> > >> > > > >>> > >> 1. Change lucene version in the POM file and fix all the > > > >>> compilation > > > >>> > >> failures > > > >>> > >> 2. Fix any failing tests due to this change > > > >>> > >> > > > >>> > >> Current progress: > > > >>> > >> > > > >>> > >> 1. Fixed all the compilation issues > > > >>> > >> 2. Comparing the test failures of before-fix v/s after-fix > > > >>> > >> > > > >>> > >> I might need some help in test failures, I happen to see that > > the > > > >>> same > > > >>> > >> tests are failing in both before-fix and after-fix. > > > >>> > >> > > > >>> > >> Let me know if any comments. > > > >>> > >> > > > >>> > >> Thanks, > > > >>> > >> -- > > > >>> > >> *Raviteja Lokineni* | Business Intelligence Developer > > > >>> > >> TD Ameritrade > > > >>> > >> > > > >>> > >> E: raviteja.lokin...@gmail.com > > > >>> > >> > > > >>> > >> [image: View Raviteja Lokineni's profile on LinkedIn] > > > >>> > >> <http://in.linkedin.com/in/ravitejalokineni> > > > >>> > >> > > > >>> > >> > > > >>> > > > > > >>> > > > > > >>> > > -- > > > >>> > > *Raviteja Lokineni* | Business Intelligence Developer > > > >>> > > TD Ameritrade > > > >>> > > > > > >>> > > E: raviteja.lokin...@gmail.com > > > >>> > > > > > >>> > > [image: View Raviteja Lokineni's profile on LinkedIn] > > > >>> > > <http://in.linkedin.com/in/ravitejalokineni> > > > >>> > > > > > >>> > > > > > >>> > > > > >>> > > > > >>> > -- > > > >>> > *Raviteja Lokineni* | Business Intelligence Developer > > > >>> > TD Ameritrade > > > >>> > > > > >>> > E: raviteja.lokin...@gmail.com > > > >>> > > > > >>> > [image: View Raviteja Lokineni's profile on LinkedIn] > > > >>> > <http://in.linkedin.com/in/ravitejalokineni> > > > >>> > > > > >>> > > > >>> > > > >>> > > > >>> -- > > > >>> *Raviteja Lokineni* | Business Intelligence Developer > > > >>> TD Ameritrade > > > >>> > > > >>> E: raviteja.lokin...@gmail.com > > > >>> > > > >>> [image: View Raviteja Lokineni's profile on LinkedIn] > > > >>> <http://in.linkedin.com/in/ravitejalokineni> > > > >>> > > > >> > > > >> > > > >> > > > >> -- > > > >> *Raviteja Lokineni* | Business Intelligence Developer > > > >> TD Ameritrade > > > >> > > > >> E: raviteja.lokin...@gmail.com > > > >> > > > >> [image: View Raviteja Lokineni's profile on LinkedIn] > > > >> <http://in.linkedin.com/in/ravitejalokineni> > > > >> > > > >> > > > > > > > > > > > > -- > > > > *Raviteja Lokineni* | Business Intelligence Developer > > > > TD Ameritrade > > > > > > > > E: raviteja.lokin...@gmail.com > > > > > > > > [image: View Raviteja Lokineni's profile on LinkedIn] > > > > <http://in.linkedin.com/in/ravitejalokineni> > > > > > > > > > > > > > > > > > -- > > > *Raviteja Lokineni* | Business Intelligence Developer > > > TD Ameritrade > > > > > > E: raviteja.lokin...@gmail.com > > > > > > [image: View Raviteja Lokineni's profile on LinkedIn] > > > <http://in.linkedin.com/in/ravitejalokineni> > > > > > > > > > -- > *Raviteja Lokineni* | Business Intelligence Developer > TD Ameritrade > > E: raviteja.lokin...@gmail.com > > [image: View Raviteja Lokineni's profile on LinkedIn] > <http://in.linkedin.com/in/ravitejalokineni> > -- *Raviteja Lokineni* | Business Intelligence Developer TD Ameritrade E: raviteja.lokin...@gmail.com [image: View Raviteja Lokineni's profile on LinkedIn] <http://in.linkedin.com/in/ravitejalokineni>