Yay! for the heads up on merging. FYI, I take back my word on failure on windows though. I had to include the hadoop.dll file on PATH. Tests are running (I am running it just to satisfy myself ;) ).
On Fri, Aug 5, 2016 at 12:11 PM, Suneel Marthi <suneel.mar...@gmail.com> wrote: > Most of us r Linux or MacOS users, so we never hit any of the build issues > u have encountered. > I must say that we never test for Windows :), but its expected that u would > run into issues on Windows. > > The patch looks good and we shuld be able to merge this over the weekend, > thanks again for the contrib. > > > > On Fri, Aug 5, 2016 at 12:01 PM, Raviteja Lokineni < > raviteja.lokin...@gmail.com> wrote: > > > I have built winutils.exe but nevertheless the test fails to run on > windows > > due to linker errors. > > > > So, for now linux builds are successful on my end. Let me know what is > your > > plan with the pull request to upgrade. > > > > On Fri, Aug 5, 2016 at 11:12 AM, Raviteja Lokineni < > > raviteja.lokin...@gmail.com> wrote: > > > > > The reason the tests failed previously on my windows box is because it > > > wasn't able to locate winutils.exe in HADOOP_HOME. > > > > > > Now, to test that I am building winutils.exe from source using the > > > following resource: http://zutai.blogspot.com/2014/06/build- > > > install-and-run-hadoop-24-240-on.html > > > > > > On Thu, Aug 4, 2016 at 9:53 PM, Raviteja Lokineni < > > > raviteja.lokin...@gmail.com> wrote: > > > > > >> I'm up for the challenge but looks like nothing failed this time too. > > >> > > >> Repo: https://github.com/bond-/mahout > > >> Branch: mahout-1876 > > >> > > >> See the full output in the gist: https://gist.github.com/ > > >> bond-/33c349136b6914db4892233b8876b9d7 > > >> > > >> If someone wants to re-verify running the tests, please go ahead and > let > > >> me know. > > >> > > >> On Thu, Aug 4, 2016 at 6:16 PM, Andrew Palumbo <ap....@outlook.com> > > >> wrote: > > >> > > >>> Maven >= 3.3.3, > > >>> java >= 1.7, > > >>> MAHOUT_HOME=/path/to/install, > > >>> > > >>> also MAHOUT_LOCAL=true > > >>> > > >>> If ur not running against a hadoop cluster. (Looks like ur not) > > >>> > > >>> Upgrading this dep is not a trivial task. > > >>> > > >>> > > >>> -------- Original message -------- > > >>> From: Raviteja Lokineni <raviteja.lokin...@gmail.com> > > >>> Date: 08/04/2016 6:09 PM (GMT-05:00) > > >>> To: mahout <dev@mahout.apache.org> > > >>> Subject: Re: MAHOUT-1876 - Lucene compatibility > > >>> > > >>> Are there any pre-requisites to run tests on local machines? > > >>> > > >>> Most of the MR module tests failed with: > > >>> > > >>> java.lang.NullPointerException: null > > >>> at __randomizedtesting.SeedInfo.seed([C8C254E3D9D80CC9:845EFDB0 > > >>> 7D41873E]:0) > > >>> at java.lang.ProcessBuilder.start(ProcessBuilder.java:1012) > > >>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:445) > > >>> at org.apache.hadoop.util.Shell.run(Shell.java:418) > > >>> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Sh > > >>> ell.java:650) > > >>> at org.apache.hadoop.util.Shell.execCommand(Shell.java:739) > > >>> at org.apache.hadoop.util.Shell.execCommand(Shell.java:722) > > >>> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission( > > >>> RawLocalFileSystem.java:633) > > >>> at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs( > > >>> RawLocalFileSystem.java:421) > > >>> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSyste > > >>> m.java:281) > > >>> at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir( > > >>> JobSubmissionFiles.java:125) > > >>> at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal( > > >>> JobSubmitter.java:348) > > >>> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285) > > >>> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282) > > >>> at java.security.AccessController.doPrivileged(Native Method) > > >>> at javax.security.auth.Subject.doAs(Subject.java:422) > > >>> at org.apache.hadoop.security.UserGroupInformation.doAs( > > >>> UserGroupInformation.java:1556) > > >>> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282) > > >>> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1303) > > >>> at org.apache.mahout.cf.taste.hadoop.preparation.PreparePrefere > > >>> nceMatrixJob. > > >>> run(PreparePreferenceMatrixJob.java:77) > > >>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > > >>> at org.apache.mahout.cf.taste.hadoop.item.RecommenderJob. > > >>> run(RecommenderJob.java:168) > > >>> at org.apache.mahout.cf.taste.hadoop.item.RecommenderJobTest. > > >>> testCompleteJobWithFiltering(RecommenderJobTest.java:881) > > >>> > > >>> On Thu, Aug 4, 2016 at 6:03 PM, Andrew Palumbo <ap....@outlook.com> > > >>> wrote: > > >>> > > >>> > Actually, Travis CI is not set up to test all mahout modules. > Could > > >>> you > > >>> > please test again on your local machine, and report any errors. > > >>> > > > >>> > Andy > > >>> > > > >>> > > > >>> > -------- Original message -------- > > >>> > From: Raviteja Lokineni <raviteja.lokin...@gmail.com> > > >>> > Date: 08/04/2016 4:01 PM (GMT-05:00) > > >>> > To: mahout <dev@mahout.apache.org> > > >>> > Subject: Re: MAHOUT-1876 - Lucene compatibility > > >>> > > > >>> > All the tests are successful even with my code change and upgrade > to > > >>> > Lucene. (The test failure that I reported above was on my local > > >>> machine, I > > >>> > guess you can ignore that.) > > >>> > > > >>> > So is this pull request good to merge? > > >>> > > > >>> > https://github.com/apache/mahout/pull/247 > > >>> > > > >>> > https://travis-ci.org/apache/mahout/builds/149840408 > > >>> > > > >>> > On Thu, Aug 4, 2016 at 2:05 PM, Raviteja Lokineni < > > >>> > raviteja.lokin...@gmail.com> wrote: > > >>> > > > >>> > > Most of the MR tests are failing with: > > >>> > > > > >>> > > java.lang.NullPointerException: null > > >>> > > at __randomizedtesting.SeedInfo.seed([C8C254E3D9D80CC9: > > >>> > > 845EFDB07D41873E]:0) > > >>> > > at java.lang.ProcessBuilder.start(ProcessBuilder.java:1012) > > >>> > > at org.apache.hadoop.util.Shell.runCommand(Shell.java:445) > > >>> > > at org.apache.hadoop.util.Shell.run(Shell.java:418) > > >>> > > at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute( > > >>> > > Shell.java:650) > > >>> > > at org.apache.hadoop.util.Shell.execCommand(Shell.java:739) > > >>> > > at org.apache.hadoop.util.Shell.execCommand(Shell.java:722) > > >>> > > at org.apache.hadoop.fs.RawLocalFileSystem.setPermission( > > >>> > > RawLocalFileSystem.java:633) > > >>> > > at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs( > > >>> > > RawLocalFileSystem.java:421) > > >>> > > at org.apache.hadoop.fs.FilterFileSystem.mkdirs( > > >>> > FilterFileSystem.java:281) > > >>> > > at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir( > > >>> > > JobSubmissionFiles.java:125) > > >>> > > at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal( > > >>> > > JobSubmitter.java:348) > > >>> > > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285) > > >>> > > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282) > > >>> > > at java.security.AccessController.doPrivileged(Native Method) > > >>> > > at javax.security.auth.Subject.doAs(Subject.java:422) > > >>> > > at org.apache.hadoop.security.UserGroupInformation.doAs( > > >>> > > UserGroupInformation.java:1556) > > >>> > > at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282) > > >>> > > at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job. > > java:1303) > > >>> > > at org.apache.mahout.cf.taste.hadoop.preparation. > > >>> > > PreparePreferenceMatrixJob.run(PreparePreferenceMatrixJob. > java:77) > > >>> > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > > >>> > > at org.apache.mahout.cf.taste.hadoop.item.RecommenderJob. > > >>> > > run(RecommenderJob.java:168) > > >>> > > at org.apache.mahout.cf.taste.hadoop.item.RecommenderJobTest. > > >>> > > testCompleteJobWithFiltering(RecommenderJobTest.java:881) > > >>> > > > > >>> > > Do I need to have hadoop installed on my local machine? > > >>> > > > > >>> > > On Thu, Aug 4, 2016 at 1:53 PM, Raviteja Lokineni < > > >>> > > raviteja.lokin...@gmail.com> wrote: > > >>> > > > > >>> > >> Hi Devs, > > >>> > >> > > >>> > >> Issue link: https://issues.apache.org/jira/browse/MAHOUT-1876 > > >>> > >> > > >>> > >> Problem statement: Mahout should be compatible with the latest > > >>> lucene > > >>> > >> version. I was trying to solve a text clustering problem and I > > >>> stumbled > > >>> > >> upon the error that the lucene version in use is not supported. > > >>> That's > > >>> > when > > >>> > >> I raised this issue and one of the guys suggested me to try > fixing > > >>> it > > >>> > and I > > >>> > >> started doing so. > > >>> > >> > > >>> > >> Proposed solution: > > >>> > >> > > >>> > >> 1. Change lucene version in the POM file and fix all the > > >>> compilation > > >>> > >> failures > > >>> > >> 2. Fix any failing tests due to this change > > >>> > >> > > >>> > >> Current progress: > > >>> > >> > > >>> > >> 1. Fixed all the compilation issues > > >>> > >> 2. Comparing the test failures of before-fix v/s after-fix > > >>> > >> > > >>> > >> I might need some help in test failures, I happen to see that > the > > >>> same > > >>> > >> tests are failing in both before-fix and after-fix. > > >>> > >> > > >>> > >> Let me know if any comments. > > >>> > >> > > >>> > >> Thanks, > > >>> > >> -- > > >>> > >> *Raviteja Lokineni* | Business Intelligence Developer > > >>> > >> TD Ameritrade > > >>> > >> > > >>> > >> E: raviteja.lokin...@gmail.com > > >>> > >> > > >>> > >> [image: View Raviteja Lokineni's profile on LinkedIn] > > >>> > >> <http://in.linkedin.com/in/ravitejalokineni> > > >>> > >> > > >>> > >> > > >>> > > > > >>> > > > > >>> > > -- > > >>> > > *Raviteja Lokineni* | Business Intelligence Developer > > >>> > > TD Ameritrade > > >>> > > > > >>> > > E: raviteja.lokin...@gmail.com > > >>> > > > > >>> > > [image: View Raviteja Lokineni's profile on LinkedIn] > > >>> > > <http://in.linkedin.com/in/ravitejalokineni> > > >>> > > > > >>> > > > > >>> > > > >>> > > > >>> > -- > > >>> > *Raviteja Lokineni* | Business Intelligence Developer > > >>> > TD Ameritrade > > >>> > > > >>> > E: raviteja.lokin...@gmail.com > > >>> > > > >>> > [image: View Raviteja Lokineni's profile on LinkedIn] > > >>> > <http://in.linkedin.com/in/ravitejalokineni> > > >>> > > > >>> > > >>> > > >>> > > >>> -- > > >>> *Raviteja Lokineni* | Business Intelligence Developer > > >>> TD Ameritrade > > >>> > > >>> E: raviteja.lokin...@gmail.com > > >>> > > >>> [image: View Raviteja Lokineni's profile on LinkedIn] > > >>> <http://in.linkedin.com/in/ravitejalokineni> > > >>> > > >> > > >> > > >> > > >> -- > > >> *Raviteja Lokineni* | Business Intelligence Developer > > >> TD Ameritrade > > >> > > >> E: raviteja.lokin...@gmail.com > > >> > > >> [image: View Raviteja Lokineni's profile on LinkedIn] > > >> <http://in.linkedin.com/in/ravitejalokineni> > > >> > > >> > > > > > > > > > -- > > > *Raviteja Lokineni* | Business Intelligence Developer > > > TD Ameritrade > > > > > > E: raviteja.lokin...@gmail.com > > > > > > [image: View Raviteja Lokineni's profile on LinkedIn] > > > <http://in.linkedin.com/in/ravitejalokineni> > > > > > > > > > > > > -- > > *Raviteja Lokineni* | Business Intelligence Developer > > TD Ameritrade > > > > E: raviteja.lokin...@gmail.com > > > > [image: View Raviteja Lokineni's profile on LinkedIn] > > <http://in.linkedin.com/in/ravitejalokineni> > > > -- *Raviteja Lokineni* | Business Intelligence Developer TD Ameritrade E: raviteja.lokin...@gmail.com [image: View Raviteja Lokineni's profile on LinkedIn] <http://in.linkedin.com/in/ravitejalokineni>
