Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/7004#issuecomment-126088128
Re: the avro dependency, this is a net new dependency for core. Previously
this came in via the hive module (the metastore dependency to be specific). I
suppose it relies on the Hive profile therefore, but not the YARN profile.
In any event the right thing to do is include the dependency if it's being
used, of course. I suppose this is evidence that the Spark assembly -- the Hive
flavors -- have had this dep and have been fine.
Avro doesn't bring anything in that we didn't already have, except Avro:
```
[INFO] +- org.apache.avro:avro-mapred:jar:hadoop2:1.7.7:compile
[INFO] | +- org.apache.avro:avro-ipc:jar:1.7.7:compile
[INFO] | | +- (org.apache.avro:avro:jar:1.7.7:compile - version managed
from 1.7.5; omitted for duplicate)
[INFO] | | +- (org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile -
version managed from 1.9.2; omitted for duplicate)
[INFO] | | +- (org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
- version managed from 1.9.2; omitted for duplicate)
[INFO] | | \- (org.slf4j:slf4j-api:jar:1.7.10:compile - version managed
from 1.6.4; scope managed from runtime; omitted for duplicate)
[INFO] | +- org.apache.avro:avro-ipc:jar:tests:1.7.7:compile
[INFO] | | +- (org.apache.avro:avro:jar:1.7.7:compile - version managed
from 1.7.5; omitted for duplicate)
[INFO] | | +- (org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile -
version managed from 1.9.2; omitted for duplicate)
[INFO] | | +- (org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
- version managed from 1.9.2; omitted for duplicate)
[INFO] | | \- (org.slf4j:slf4j-api:jar:1.7.10:compile - version managed
from 1.6.4; scope managed from runtime; omitted for duplicate)
[INFO] | +- (org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile -
version managed from 1.9.2; omitted for duplicate)
[INFO] | +- (org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile -
version managed from 1.9.2; omitted for duplicate)
[INFO] | \- (org.slf4j:slf4j-api:jar:1.7.10:compile - version managed from
1.6.4; scope managed from runtime; omitted for duplicate)
```
So, I think the net change here is only that Avro has been added to core.
Unless there's an objection to adding Avro at all, I think this is OK from a
build standpoint.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
