This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new cc49244 [SPARK-31258][BUILD] Pin the avro version in SBT
cc49244 is described below
commit cc492447a6411bae51f91f408207272e8aeca39b
Author: Kent Yao <yaooq...@hotmail.com>
AuthorDate: Thu Mar 26 10:48:11 2020 +0900
[SPARK-31258][BUILD] Pin the avro version in SBT
### What changes were proposed in this pull request?
add arvo dep in SparkBuild
### Why are the changes needed?
fix sbt unidoc like
https://github.com/apache/spark/pull/28017#issuecomment-603828597
```scala
[warn] Multiple main classes detected. Run 'show discoveredMainClasses' to
see the list
[warn] Multiple main classes detected. Run 'show discoveredMainClasses' to
see the list
[info] Main Scala API documentation to
/home/jenkins/workspace/SparkPullRequestBuilder6/target/scala-2.12/unidoc...
[info] Main Java API documentation to
/home/jenkins/workspace/SparkPullRequestBuilder6/target/javaunidoc...
[error]
/home/jenkins/workspace/SparkPullRequestBuilder6/core/src/main/scala/org/apache/spark/serializer/GenericAvroSerializer.scala:123:
value createDatumWriter is not a member of org.apache.avro.generic.GenericData
[error] writerCache.getOrElseUpdate(schema,
GenericData.get.createDatumWriter(schema))
[error] ^
[info] No documentation generated with unsuccessful compiler run
[error] one error found
```
### Does this PR introduce any user-facing change?
no
### How was this patch tested?
pass jenkins
and verify manually with `sbt dependencyTree`
```scala
kentyaohulk ~/spark dep build/sbt dependencyTree | grep avro |
grep -v Resolving
[info] +-org.apache.avro:avro-mapred:1.8.2
[info] | +-org.apache.avro:avro-ipc:1.8.2
[info] | | +-org.apache.avro:avro:1.8.2
[info] +-org.apache.avro:avro:1.8.2
[info] | | +-org.apache.avro:avro:1.8.2
[info] org.apache.spark:spark-avro_2.12:3.1.0-SNAPSHOT [S]
[info] | | | +-org.apache.avro:avro-mapred:1.8.2
[info] | | | | +-org.apache.avro:avro-ipc:1.8.2
[info] | | | | | +-org.apache.avro:avro:1.8.2
[info] | | | +-org.apache.avro:avro:1.8.2
[info] | | | | | +-org.apache.avro:avro:1.8.2
```
Closes #28020 from yaooqinn/dep.
Authored-by: Kent Yao <yaooq...@hotmail.com>
Signed-off-by: HyukjinKwon <gurwls...@apache.org>
(cherry picked from commit 336621e277794ab2e6e917391a928ec662498fcf)
Signed-off-by: HyukjinKwon <gurwls...@apache.org>
---
project/SparkBuild.scala | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/project/SparkBuild.scala b/project/SparkBuild.scala
index 3889013..7bd92b3 100644
--- a/project/SparkBuild.scala
+++ b/project/SparkBuild.scala
@@ -624,7 +624,8 @@ object DependencyOverrides {
dependencyOverrides += "com.google.guava" % "guava" % "14.0.1",
dependencyOverrides += "commons-io" % "commons-io" % "2.4",
dependencyOverrides += "xerces" % "xercesImpl" % "2.12.0",
- dependencyOverrides += "jline" % "jline" % "2.14.6")
+ dependencyOverrides += "jline" % "jline" % "2.14.6",
+ dependencyOverrides += "org.apache.avro" % "avro" % "1.8.2")
}
/**
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
