[
https://issues.apache.org/jira/browse/CRUNCH-16?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Wills updated CRUNCH-16:
-----------------------------
Attachment: CRUNCH-16-avro-preparation.patch
Re: protobufs, cool, thanks.
The attached patch will pass all unit/integration tests w/Avro 1.5.4, 1.6.2,
and 1.7.0, w/both Hadoop 1 and 2. The one thing I'm missing at this point is
code to specifically disallow AvroTypes that combine Specific and Reflect-based
schemas in the same MR job, since that will still fail in 1.5.4 and 1.6.2 but
not in 1.7.0. We should have Gabriel and Christian take a look at this one
before we integrate it.
> Fix dependency versions
> -----------------------
>
> Key: CRUNCH-16
> URL: https://issues.apache.org/jira/browse/CRUNCH-16
> Project: Crunch
> Issue Type: Task
> Reporter: Matthias Friedrich
> Assignee: Matthias Friedrich
> Priority: Critical
> Labels: maven
> Attachments: 0001-CRUNCH-16-Fix-dependency-versions.patch,
> CRUNCH-16-avro-preparation.patch
>
>
> Right now, the compile time classpath doesn't exactly match the runtime
> classpath. For example, Hadoop CDH3u4 comes with slf4j-api-1.4.3 while Crunch
> references slf4j-api-1.6.3. This could result in NoSuchMethodError and other
> nasty problems.
> We have to make sure that version numbers match (check "mvn dependency:tree")
> and that we don't ship artifacts that are already present on the runtime
> classpath. If our dependencies introduce incompatible transitive
> dependencies, we might have to downgrade them.
> Note: Maven's dependency resolution is order-dependent, see
> http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
