Re: Install mahout 0.8 with hadoop 2.0
Honest feedback is always welcome on this mailing list. Don't ever worry about flames for that. Don't forget that mr v1 is an option with hadoop 2. Confusing as that may be. Iterative algos are, as you say, very important. My current inclination is to lean toward a downpour style of implementation. That fits well with yarn but it also actually fits reasonably with mr v1. Sent from my iPhone On Aug 13, 2013, at 20:13, Carlos Mundi cmu...@gmail.com wrote: Anyway, I apologize if anyone takes offense. None is meant, so please flame me off-list if you must. But since I self-identify as a member of the small demand set Ted Dunning describes, I figure I can chime in. As always, YMMV.
Re: Install mahout 0.8 with hadoop 2.0
Ted, The mr v1 option is absolutely viable. It's just question of how long a project wants to rely on a legacy tail, and that's everyone's individual lookout naturally. Downpour style will help, as you say and surely know better than me; I should think about that for my situation. Thanks! On Aug 14, 2013 12:37 PM, Ted Dunning ted.dunn...@gmail.com wrote: Honest feedback is always welcome on this mailing list. Don't ever worry about flames for that. Don't forget that mr v1 is an option with hadoop 2. Confusing as that may be. Iterative algos are, as you say, very important. My current inclination is to lean toward a downpour style of implementation. That fits well with yarn but it also actually fits reasonably with mr v1. Sent from my iPhone On Aug 13, 2013, at 20:13, Carlos Mundi cmu...@gmail.com wrote: Anyway, I apologize if anyone takes offense. None is meant, so please flame me off-list if you must. But since I self-identify as a member of the small demand set Ted Dunning describes, I figure I can chime in. As always, YMMV.
Install mahout 0.8 with hadoop 2.0
Hi all, Somebody compile and install mahout with hadoop 2.0? If yes, that what changes you make in mahout, that it have 100% passed unit tests and successful work with hadoop 2.0? Thanks
Re: Install mahout 0.8 with hadoop 2.0
No. There is very small demand for Mahout on Hadoop 2.0 so far and the forward/backward incompatibility of 2.0 has made it difficult to motivate moving to 2.0. The bigtop guys built a maven profile for 0.23 some time ago. I don't know the status of that. I don't think that the differences are huge ... it is just the standard Hadoop forklift-the-world upgrade experience. On Tue, Aug 13, 2013 at 6:49 AM, Sergey Svinarchuk ssvinarc...@hortonworks.com wrote: Hi all, Somebody compile and install mahout with hadoop 2.0? If yes, that what changes you make in mahout, that it have 100% passed unit tests and successful work with hadoop 2.0? Thanks
Re: Install mahout 0.8 with hadoop 2.0
I think it all minimally works on Hadoop 2.0.x, though I haven't tried it recently -- it does require a recompile. This is different from it working on MRv2 versus MRv1. I'm almost certain it does not work on MRv2 and doubt it will. The effort is not large, but it's subtle. A few hacks may fail in mysterious ways, and certainly to properly use MRv2 you have to switch to use the newer resource configuration scheme -- in terms of megabytes not reducer slots and all that. At least this was most of the work that i remember when I was rebuilding some of this type of stuff on MRv2 + Hadoop 2.0.x On Tue, Aug 13, 2013 at 5:58 PM, Ted Dunning ted.dunn...@gmail.com wrote: No. There is very small demand for Mahout on Hadoop 2.0 so far and the forward/backward incompatibility of 2.0 has made it difficult to motivate moving to 2.0. The bigtop guys built a maven profile for 0.23 some time ago. I don't know the status of that. I don't think that the differences are huge ... it is just the standard Hadoop forklift-the-world upgrade experience. On Tue, Aug 13, 2013 at 6:49 AM, Sergey Svinarchuk ssvinarc...@hortonworks.com wrote: Hi all, Somebody compile and install mahout with hadoop 2.0? If yes, that what changes you make in mahout, that it have 100% passed unit tests and successful work with hadoop 2.0? Thanks
Re: Install mahout 0.8 with hadoop 2.0
I recently asked the same core question on this list. I certainly won't argue with the statistics of small numbers. But I will hazard a prediction: the impetus for Mahout to support Hadoop 2 will appear about the same time the elephant book gets updated for 2.0, provided Twister or something like it does not get there first. My own view, from the outside, is that the best use of limited Mahout dev resources at the moment is stabilizing the core feature-set and positioning Mahout in the increasingly crowded big analytics space. I don't know how easy that is, as I do not know how tightly Mahout is tied to the Hadoop MR model. Although it would be hard to overstate the importance of iterative algos, it is also true that Hadoop MR excels on some very real killer apps. Making those accessible and robust should take precedence, for now, over the incremental ROI for chasing Hadoop 2. I say this as someone who would very much like to run Mahout against Hadoop 2. Anyway, I apologize if anyone takes offense. None is meant, so please flame me off-list if you must. But since I self-identify as a member of the small demand set Ted Dunning describes, I figure I can chime in. As always, YMMV. Cheers, Carlos On Aug 13, 2013 9:59 AM, Ted Dunning ted.dunn...@gmail.com wrote: No. There is very small demand for Mahout on Hadoop 2.0 so far and the forward/backward incompatibility of 2.0 has made it difficult to motivate moving to 2.0. The bigtop guys built a maven profile for 0.23 some time ago. I don't know the status of that. I don't think that the differences are huge ... it is just the standard Hadoop forklift-the-world upgrade experience. On Tue, Aug 13, 2013 at 6:49 AM, Sergey Svinarchuk ssvinarc...@hortonworks.com wrote: Hi all, Somebody compile and install mahout with hadoop 2.0? If yes, that what changes you make in mahout, that it have 100% passed unit tests and successful work with hadoop 2.0? Thanks
