[
https://issues.apache.org/jira/browse/HCATALOG-520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13483232#comment-13483232
]
Travis Crawford commented on HCATALOG-520:
------------------------------------------
Hey [~toffer] – can you clarify what you mean by building with 0.23? The only
place I see 0.23 used is in the shims layer. When resolving dependencies we
download always download deps for both 0.20 and 0.23. The build flag enables
which set of jars is used. AFAICT this works for both. Its possible I'm being
dense here but I don't understand the issue you're seeing.
Re: e2e tests – I put a bunch of work into getting them to run on developer
machines. If you can take a look at HCATALOG-535 we can get that in, then I can
run them before submitting future changes, including this one.
> Simplify HCatalog dependencies
> ------------------------------
>
> Key: HCATALOG-520
> URL: https://issues.apache.org/jira/browse/HCATALOG-520
> Project: HCatalog
> Issue Type: Improvement
> Reporter: Travis Crawford
> Assignee: Travis Crawford
> Attachments: HCATALOG-520_simfy_deps.1.patch
>
>
> Looking through the hcatalog-core dependencies I believe we have an
> opportunity to trim them down. A major goal of HCatalog is to be a dependency
> of other processing tools, and we can make that more attractive by invading
> their classpath as little as possible.
> I believe the following look good (minus hive-exec which is a fat jar, but
> that's a separate issue):
> {code}
> <dependency org="org.apache.hadoop" name="hadoop-tools"
> rev="${hadoop20.version}" conf="default->*"/>
> <dependency org="org.apache.hive" name="hive-builtins"
> rev="${hive.version}"/>
> <dependency org="org.apache.hive" name="hive-metastore"
> rev="${hive.version}"/>
> <dependency org="org.apache.hive" name="hive-common"
> rev="${hive.version}"/>
> <dependency org="org.apache.hive" name="hive-exec" rev="${hive.version}"/>
> <dependency org="org.apache.hive" name="hive-cli" rev="${hive.version}"/>
> <dependency org="org.apache.hive" name="hive-hbase-handler"
> rev="${hive.version}">
> <exclude org="org.apache.maven.plugins"/>
> <exclude org="org.jruby"/>
> </dependency>
> {code}
> The following are where I believe we can make improvements:
> {code}
> <dependency org="org.apache.pig" name="pig" rev="${pig.version}"
> conf="default->*"/>
> {code}
> Pig is still depended on in hcatalog-core tests, but has not yet been moved
> to the test target. A major goal of switching to subprojects was to stop
> forcing processing frameworks as dependencies on people using HCat. This
> should move to the test target (since some core tests use pig for
> convenience).
> {code}
> <dependency org="javax.management.j2ee" name="management-api"
> rev="${javax-mgmt.version}"/>
> {code}
> Does anyone know why management-api is needed? I'm not familiar with this and
> don't see any usages from a quick grep. Its something JMS-related, and maybe
> was needed by hcatalog-server-extensions at some point? If tests pass without
> this I think we should remove it.
> {code}
> <dependency org="org.codehaus.jackson" name="jackson-mapper-asl"
> rev="${jackson.version}"/>
> <dependency org="org.codehaus.jackson" name="jackson-core-asl"
> rev="${jackson.version}"/>
> {code}
> HCatalog build requests jackson 1.7.3, and hive-exec depends on 1.8.8. Any
> objection to using the versions provided by Hive?
> {code}
> <dependency org="org.apache.thrift" name="libfb303" rev="${fb303.version}"/>
> {code}
> I don't believe this is required because hive-metastore depends on libfb303.
> {code}
> <dependency org="commons-dbcp" name="commons-dbcp"
> rev="${commons-dbcp.version}">
> <exclude module="commons-pool"/>
> <exclude org="org.apache.geronimo.specs" module="geronimo-jta_1.1_spec"/>
> </dependency>
> {code}
> hive-metastore depends on commons-dbcp and I don't believe we need to
> explicitly depend on this.
> {code}
> <dependency org="com.google.guava" name="guava" rev="${guava.version}"/>
> {code}
> hive-exec depends on guava 11.0.2 too so I don't believe we need to depend on
> this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
