Replying to the dev list instead of the jira as this requires everybody's attention/thoughts.
I would prefer all at once for mavenization. All the ivy stuff in ant will be throwaway in maven. The directory structure would require good amount of reorganizing (properly modularizing, moving code to standard directory structure src/main/java, src/test/java, src/test/resources). Trying to do it incrementally, moving out code and creating sub-projects is ok, but the effort spent on changing ant scripts for that might be a waste of effort as it is going to be thrown away. It would be better to have the build in broken state for few days and complete mavenization faster instead of trying to keep ant working when we move to maven. Since hcatalog is not that complex, mavenizing should be easy and quick except for the e2e tests. Even that can be quickly done with maven ant plugin before completely moving to maven. Regards, Rohini On Fri, Jun 22, 2012 at 11:14 AM, Travis Crawford <[email protected]>wrote: > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/5496/ > > On June 22nd, 2012, 5:21 p.m., *Rohini Palaniswamy* wrote: > > > http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/build.xml<https://reviews.apache.org/r/5496/diff/2/?file=115880#file115880line329> > (Diff > revision 2) > > 329 > > <antcall target="ivy-publish"/> > > Do we want to do ivy-publish inside the jar target. Would prefer it to be > called separately in only apache hudson build. ivy-publish can probably > depend on jar. All of us run jar in our hosts. Putting ivy-publish here might > lead to issues. > > On June 22nd, 2012, 5:27 p.m., *Travis Crawford* wrote: > > So this publishes to the local ivy cache, which is needed because > hcatalog-pig-adapter depends on hcatalog.jar. Otherwise the subproject > dependency does not work. > > On June 22nd, 2012, 6 p.m., *Rohini Palaniswamy* wrote: > > Good then. Thought that it publishes a snapshot jar to the maven repo. Should > have paid more attention to the "local" keyword there. > > What are your thoughts on this general approach of reorganizing the repo to > match the maven layout, then we can mavenize? Are you okay with this > incremental approach, or do you think it would be better all at once? > > Looking at the current build, I think we also want subprojects for: > > * hcatalog-server-extensions.jar > * hcatalog-core.jar (today just hcatalog.jar; we could keep this just > hcatalog.jar if people like it) > > I don't fully understand the storage-handler stuff, since doesn't that stuff > belong in Hive? For now I was planning to leave as-is. > > Let me know if you want to do this all at once, or incremental. > > > - Travis > > On June 22nd, 2012, 3:45 p.m., Travis Crawford wrote: > Review request for hcatalog. > By Travis Crawford. > > *Updated June 22, 2012, 3:45 p.m.* > Description > > Update HCatalog build to package pig classes as a separate jar. I did not > update ivy yet, but if the general approach looks good I will update. This > will let the core hcatalog.jar depends only on stuff needed by all processing > frameworks; then people that want to use pig can use the pig adapter which > has the pig dependency. > > I believe we'll have more adapters in the future, so I'm trying to make this > reusable. > > For example: > > Traviss-iMac:hcatalog travis$ jar -tvf > hcatalog-pig-adapter/build/hcatalog-pig-adapter-0.5.0-dev.jar > 0 Thu Jun 21 10:34:18 PDT 2012 META-INF/ > 107 Thu Jun 21 10:34:16 PDT 2012 META-INF/MANIFEST.MF > 0 Thu Jun 21 10:34:16 PDT 2012 org/ > 0 Thu Jun 21 10:34:16 PDT 2012 org/apache/ > 0 Thu Jun 21 10:34:16 PDT 2012 org/apache/hcatalog/ > 0 Thu Jun 21 10:34:16 PDT 2012 org/apache/hcatalog/pig/ > 0 Thu Jun 21 10:34:16 PDT 2012 org/apache/hcatalog/pig/drivers/ > 4352 Thu Jun 21 10:34:16 PDT 2012 > org/apache/hcatalog/pig/HCatBaseLoader.class > 1261 Thu Jun 21 10:34:16 PDT 2012 > org/apache/hcatalog/pig/HCatBaseStorer$1.class > 12413 Thu Jun 21 10:34:16 PDT 2012 > org/apache/hcatalog/pig/HCatBaseStorer.class > 632 Thu Jun 21 10:34:16 PDT 2012 org/apache/hcatalog/pig/HCatLoader$1.class > 8518 Thu Jun 21 10:34:16 PDT 2012 org/apache/hcatalog/pig/HCatLoader.class > 6801 Thu Jun 21 10:34:16 PDT 2012 org/apache/hcatalog/pig/HCatStorer.class > 1019 Thu Jun 21 10:34:16 PDT 2012 > org/apache/hcatalog/pig/PigHCatUtil$1.class > 13117 Thu Jun 21 10:34:16 PDT 2012 org/apache/hcatalog/pig/PigHCatUtil.class > 3711 Thu Jun 21 10:34:16 PDT 2012 > org/apache/hcatalog/pig/drivers/LoadFuncBasedInputFormat$LoadFuncBasedRecordReader.class > 2383 Thu Jun 21 10:34:16 PDT 2012 > org/apache/hcatalog/pig/drivers/LoadFuncBasedInputFormat.class > 2189 Thu Jun 21 10:34:16 PDT 2012 > org/apache/hcatalog/pig/drivers/StoreFuncBasedOutputFormat$StoreFuncBasedOutputCommitter.class > 1775 Thu Jun 21 10:34:16 PDT 2012 > org/apache/hcatalog/pig/drivers/StoreFuncBasedOutputFormat$StoreFuncBasedRecordWriter.class > 2647 Thu Jun 21 10:34:16 PDT 2012 > org/apache/hcatalog/pig/drivers/StoreFuncBasedOutputFormat.class > Traviss-iMac:hcatalog travis$ > > Diffs > > - > > http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/build-common-new.xml > (PRE-CREATION) > - http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/build.xml > (1352540) > - > > http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/hcatalog-pig-adapter/build.xml > (PRE-CREATION) > - > > http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/hcatalog-pig-adapter/ivy.xml > (PRE-CREATION) > - http://svn.apache.org/repos/asf/incubator/hcatalog/trunk/ivy.xml > (1352540) > > View Diff <https://reviews.apache.org/r/5496/diff/> >
