Makes sense On Wed, Sep 7, 2011 at 11:32 AM, <milind.bhandar...@emc.com> wrote:
> +1 for separate hadoop-tools module. However, if a tool is broken at > release time, and no one comes forward to fix it, it should be removed. > (i.e. Unlike contrib modules, where build and test failures were > tolerated.) > > - milind > > On 9/7/11 11:27 AM, "Mahadev Konar" <maha...@hortonworks.com> wrote: > > >I like the idea of having tools as a seperate module and I dont think > >that it will be a dumping ground unless we choose to make one of it. > > > >+1 for hadoop tools module under trunk. > > > >thanks > >mahadev > > > >On Wed, Sep 7, 2011 at 11:18 AM, Alejandro Abdelnur <t...@cloudera.com> > >wrote: > >> Agreed, we should not have a dumping ground. IMO, what it would go into > >> hadoop-tools (i.e. distcp, streaming and someone could argue for > >>FsShell as > >> well) are effectively hadoop CLI utilities. Having them in a separate > >>module > >> rather in than in the core module (common, hdfs, mapreduce) does not > >>mean > >> that they are secondary things, just modularization. Also it will help > >>to > >> get those tools to use public interfaces of the core module, and when we > >> finally have a clean hadoop-client layer, those tools should only > >>depend on > >> that. > >> > >> Finally, the fact that tools would end up under trunk/hadoop-tools, it > >>does > >> not prevent that the packaging from HDFS and MAPREDUCE to bundle the > >> same/different tools > >> > >> +1 for hadoop-tools/ (not binding) > >> > >> Thanks. > >> > >> > >> On Wed, Sep 7, 2011 at 10:50 AM, Eric Yang <eric...@gmail.com> wrote: > >> > >>> Mapreduce and HDFS are distinct function of Hadoop. They are loosely > >>> coupled. If we have tools aggregator module, it will not have as > >>> clear distinct function as other Hadoop modules. Hence, it is > >>> possible for a tool to be depend on both HDFS and map reduce. If > >>> something broke in tools module, it is unclear which subproject's > >>> responsibility to maintain tools function. Therefore, it is safer to > >>> send tools to incubator or apache extra rather than deposit the > >>> utility tools in tools subcategory. There are many short lived > >>> projects that attempts to associate themselves with Hadoop but not > >>> being maintained. It would be better to spin off those utility > >>> projects than use Hadoop as a dumping ground. > >>> > >>> The previous discussion for removing contrib, most people were in > >>> favor of doing so, and only a few contrib owners were reluctant to > >>> remove contrib. Fewer people has participated in restore > >>> functionality of broken contrib projects. History speaks for itself. > >>> -1 (non-binding) for hadoop-tools. > >>> > >>> regards, > >>> Eric > >>> > >>> On Tue, Sep 6, 2011 at 6:55 PM, Alejandro Abdelnur <t...@cloudera.com> > >>> wrote: > >>> > Eric, > >>> > > >>> > Personally I'm fine either way. > >>> > > >>> > Still, I fail to see why a generic/categorized tools increase/reduce > >>>the > >>> > risk of dead code and how they make more-difficult/easier the > >>> > package&deployment. > >>> > > >>> > Would you please explain this? > >>> > > >>> > Thanks. > >>> > > >>> > Alejandro > >>> > > >>> > On Tue, Sep 6, 2011 at 6:38 PM, Eric Yang <eric...@gmail.com> wrote: > >>> > > >>> >> Option #2 proposed by Amareshwari, seems like a better proposal. We > >>> don't > >>> >> want to repeat history for contrib again with hadoop-tools. Having > >>>a > >>> >> generic module like hadoop-tools increases the risk of accumulate > >>>dead > >>> code. > >>> >> It would be better to categorize the hdfs or mapreduce specific > >>>tools > >>> in > >>> >> their respected subcategories. It is also easier to manage from > >>> >> package/deployment prospective. > >>> >> > >>> >> regards, > >>> >> Eric > >>> >> > >>> >> On Sep 6, 2011, at 4:32 PM, Eli Collins wrote: > >>> >> > >>> >> > On Tue, Sep 6, 2011 at 10:11 AM, Allen Wittenauer <a...@apache.org> > >>> wrote: > >>> >> >> > >>> >> >> On Sep 6, 2011, at 9:30 AM, Vinod Kumar Vavilapalli wrote: > >>> >> >>> We still need to answer Amareshwari's question (2) she asked > >>>some > >>> time > >>> >> back > >>> >> >>> about the automated code compilation and test execution of the > >>>tools > >>> >> module. > >>> >> >> > >>> >> >> > >>> >> >> > >>> >> >>>>> My #1 question is if tools is basically contrib reborn. If > >>>not, > >>> what > >>> >> >>>> makes > >>> >> >>>>> it different? > >>> >> >> > >>> >> >> > >>> >> >> I'm still waiting for this answer as well. > >>> >> >> > >>> >> >> Until such, I would be pretty much against a tools module. > >>> >> Changing the name of the dumping ground doesn't make it any less > >>>of a > >>> >> dumping ground. > >>> >> > > >>> >> > IMO if the tools module only gets stuff like distcp that's > >>>maintained > >>> >> > then it's not contrib, if it contains all the stuff from the > >>>current > >>> >> > MR contrib then tools is just a re-labeling of contrib. Given that > >>> >> > this proposal only covers moving distcp to tools it doesn't sound > >>>like > >>> >> > contrib to me. > >>> >> > > >>> >> > Thanks, > >>> >> > Eli > >>> >> > >>> >> > >>> > > >>> > >> > > > >
