[
https://issues.apache.org/jira/browse/CRUNCH-16?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13438976#comment-13438976
]
Gabriel Reid commented on CRUNCH-16:
------------------------------------
I've realized that this is directly related to something else I've been working
on with Crunch where I'm combining reflect and specific data, so I took a
second look at doing automatic initialization of the
CAN_COMBINE_SPECIFIC_AND_REFLECT_SCHEMAS.
As far as I know/recall, the relevant issue that was fixed in Avro 1.7.0 was
AVRO-1046, which was about the ReflectDatumReader properly handling specific
data; this was important because the ReflectDatumReader is used as the fallback
anytime there is reflect data included.
The attached patch fixes a small bug in the existing patches (the
ReflectDatumReader wouldn't be used if the
CAN_COMBINE_SPECIFIC_AND_REFLECT_SCHEMAS bit was set), and it also includes
code to dynamically initialize the CAN_COMBINE_SPECIFIC_AND_REFLECT_SCHEMAS
(which is now final). Can you take a look and let me know what you think? I've
run the integration tests on Avro 1.5.4, 1.6.2, and 1.7.0 and it works on all
of them.
> Fix dependency versions
> -----------------------
>
> Key: CRUNCH-16
> URL: https://issues.apache.org/jira/browse/CRUNCH-16
> Project: Crunch
> Issue Type: Task
> Reporter: Matthias Friedrich
> Assignee: Matthias Friedrich
> Priority: Critical
> Labels: maven
> Attachments: 0001-CRUNCH-16-Fix-dependency-versions.patch,
> CRUNCH-16-avro-preparation.patch,
> CRUNCH-16-avro-reflect-specific-dynamic-init.patch.txt,
> CRUNCH-16-avro-reflect-specific-static-init.patch
>
>
> Right now, the compile time classpath doesn't exactly match the runtime
> classpath. For example, Hadoop CDH3u4 comes with slf4j-api-1.4.3 while Crunch
> references slf4j-api-1.6.3. This could result in NoSuchMethodError and other
> nasty problems.
> We have to make sure that version numbers match (check "mvn dependency:tree")
> and that we don't ship artifacts that are already present on the runtime
> classpath. If our dependencies introduce incompatible transitive
> dependencies, we might have to downgrade them.
> Note: Maven's dependency resolution is order-dependent, see
> http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
