On Mon, Jun 18, 2012 at 10:39 AM, Alan Gates <[email protected]> wrote: > > On Jun 15, 2012, at 4:23 PM, Travis Crawford wrote: > >> Hey hcat gurus - >> >> Today I started looking at publishing HCatalog to maven >> (https://issues.apache.org/jira/browse/HCATALOG-132) and found its >> pretty straightforward. >> >> One thing we need to do is file a request with the repo administrators >> giving us access to publish. Its documented here: >> >> http://www.apache.org/dev/publishing-maven-artifacts.html#signing-up >> >> Does the following look good? If so, I'll open the request ticket. >> >> """ >> Project URL: http://incubator.apache.org/hcatalog/ >> >> SVN URL: http://svn.apache.org/repos/asf/incubator/hcatalog/ >> >> Maven Group Ids: org.apache.hcatalog >> >> Managed By This TLP Project: Incubator > > +1, looks good.
Filed https://issues.apache.org/jira/browse/INFRA-4936 > >> """ >> >> I included our whole pom at the end of this message (not sure if >> attachments are stripped). I see two janky things that I'm not sure >> how we want to address: >> >> (a) One of the major goals of HCat is to let many different processing >> frameworks all use the metastore. For example, pig, mapreduce, and I >> need to look at scalding soon. We don't have a "core" jar, then >> per-framework jars (hcatalog-pig.jar for example). This means anyone >> depending on hcat will have transitive dependencies on all sorts of >> processing frameworks they don't actually use. >> >> What does everyone think about renaming the current jar >> "hcatalog-core-0.5.0-dev.jar"? It would always contain the core stuff, >> and input/output formats. Processing-frameworks like pig could be >> extracted into "hcatalog-pig-0.5.0-dev.jar". New frameworks would go >> straight into subproject jars. > > This makes sense. No reason to inflict Pig or other tools on people who > don't want it. Filed https://issues.apache.org/jira/browse/HCATALOG-434 to package pig support as a separate jar. Let's keep hcatalog.jar (not hcatalog-core.jar) as its simpler that way. >> >> (b) We explicitly depend on many jars that really we should get >> transitively from Hive. antlr is an example. I looked into this and >> the Hive pom lists most dependencies as "compile" not "runtime" so we >> don't pull them in. This would really shrink our pom file after >> fixing. > > Let's file a bug against Hive to fix this. Agreed. I started filing the bug, but came across this: http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope """ compile This is the default scope, used if none is specified. Compile dependencies are available in all classpaths of a project. Furthermore, those dependencies are propagated to dependent projects. """ This doesn't jive with what I see in "ant ivy-report". I'd like to double-check what's going on here before asking the Hive folks to make a change, since it appears I don't fully understand the issue yet. --travis > Alan. > >> >> --travis >> >> >> >> >> >> <?xml version="1.0" encoding="UTF-8"?> >> <!-- >> Apache Maven 2 POM generated by Apache Ivy >> http://ant.apache.org/ivy/ >> Apache Ivy version: 2.1.0 20090925235825 >> --> >> <project xmlns="http://maven.apache.org/POM/4.0.0"; >> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; >> xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 >> http://maven.apache.org/maven-v4_0_0.xsd";> >> >> <modelVersion>4.0.0</modelVersion> >> <groupId>org.apache.hcatalog</groupId> >> <artifactId>hcatalog</artifactId> >> <packaging>jar</packaging> >> <version>0.5.0-dev</version> >> <dependencies> >> <dependency> >> <groupId>org.antlr</groupId> >> <artifactId>antlr</artifactId> >> <version>3.0.1</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>org.antlr</groupId> >> <artifactId>antlr-runtime</artifactId> >> <version>3.0.1</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>org.apache.pig</groupId> >> <artifactId>pig</artifactId> >> <version>0.8.0</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>commons-cli</groupId> >> <artifactId>commons-cli</artifactId> >> <version>1.2</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>commons-lang</groupId> >> <artifactId>commons-lang</artifactId> >> <version>2.4</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>commons-logging</groupId> >> <artifactId>commons-logging</artifactId> >> <version>1.0.4</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>commons-logging</groupId> >> <artifactId>commons-logging-api</artifactId> >> <version>1.0.4</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>org.apache.hadoop</groupId> >> <artifactId>hadoop-tools</artifactId> >> <version>1.0.2</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>org.apache.hadoop</groupId> >> <artifactId>hadoop-core</artifactId> >> <version>1.0.2</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>org.apache.hadoop</groupId> >> <artifactId>hadoop-test</artifactId> >> <version>1.0.2</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>javax.jms</groupId> >> <artifactId>jms</artifactId> >> <version>1.1</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>org.apache.activemq</groupId> >> <artifactId>activemq-core</artifactId> >> <version>5.5.0</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>org.apache.activemq</groupId> >> <artifactId>kahadb</artifactId> >> <version>5.5.0</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>javax.management.j2ee</groupId> >> <artifactId>management-api</artifactId> >> <version>1.1-rev-1</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>com.google.code.p.arat</groupId> >> <artifactId>rat-lib</artifactId> >> <version>0.5.1</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>org.vafer</groupId> >> <artifactId>jdeb</artifactId> >> <version>0.8</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>org.codehaus.jackson</groupId> >> <artifactId>jackson-mapper-asl</artifactId> >> <version>1.7.3</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>org.codehaus.jackson</groupId> >> <artifactId>jackson-core-asl</artifactId> >> <version>1.7.3</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>org.apache.hive</groupId> >> <artifactId>hive-metastore</artifactId> >> <version>0.10.0-SNAPSHOT</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>org.apache.hive</groupId> >> <artifactId>hive-common</artifactId> >> <version>0.10.0-SNAPSHOT</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>org.apache.hive</groupId> >> <artifactId>hive-exec</artifactId> >> <version>0.10.0-SNAPSHOT</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>org.apache.hive</groupId> >> <artifactId>hive-cli</artifactId> >> <version>0.10.0-SNAPSHOT</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>org.apache.hive</groupId> >> <artifactId>hive-hbase-handler</artifactId> >> <version>0.10.0-SNAPSHOT</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>org.apache.maven</groupId> >> <artifactId>maven-ant-tasks</artifactId> >> <version>2.1.3</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>org.apache.thrift</groupId> >> <artifactId>libfb303</artifactId> >> <version>0.7.0</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>junit</groupId> >> <artifactId>junit</artifactId> >> <version>4.10</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>com.google.guava</groupId> >> <artifactId>guava</artifactId> >> <version>11.0</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>org.slf4j</groupId> >> <artifactId>slf4j-api</artifactId> >> <version>1.6.1</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>org.slf4j</groupId> >> <artifactId>slf4j-log4j12</artifactId> >> <version>1.6.1</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>log4j</groupId> >> <artifactId>log4j</artifactId> >> <version>1.2.16</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>javax.jdo</groupId> >> <artifactId>jdo2-api</artifactId> >> <version>2.3-ec</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>org.datanucleus</groupId> >> <artifactId>datanucleus-core</artifactId> >> <version>2.0.3</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>org.datanucleus</groupId> >> <artifactId>datanucleus-connectionpool</artifactId> >> <version>2.0.3</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>org.datanucleus</groupId> >> <artifactId>datanucleus-enhancer</artifactId> >> <version>2.0.3</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>org.datanucleus</groupId> >> <artifactId>datanucleus-rdbms</artifactId> >> <version>2.0.3</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>commons-dbcp</groupId> >> <artifactId>commons-dbcp</artifactId> >> <version>1.4</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>commons-pool</groupId> >> <artifactId>commons-pool</artifactId> >> <version>1.5.4</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>org.apache.derby</groupId> >> <artifactId>derby</artifactId> >> <version>10.4.2.0</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>commons-configuration</groupId> >> <artifactId>commons-configuration</artifactId> >> <version>1.6</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>commons-httpclient</groupId> >> <artifactId>commons-httpclient</artifactId> >> <version>3.0.1</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>org.apache.hive</groupId> >> <artifactId>hive-builtins</artifactId> >> <version>0.10.0-SNAPSHOT</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>org.mortbay.jetty</groupId> >> <artifactId>jetty</artifactId> >> <version>6.1.26</version> >> <optional>true</optional> >> </dependency> >> <dependency> >> <groupId>org.mortbay.jetty</groupId> >> <artifactId>jetty-util</artifactId> >> <version>6.1.26</version> >> <optional>true</optional> >> </dependency> >> </dependencies> >> </project> >
