Jungtaek Lim created SPARK-27205:
------------------------------------
Summary: spark-shell with packages option fails to load transitive
dependencies even ivy successfully pulls jars
Key: SPARK-27205
URL: https://issues.apache.org/jira/browse/SPARK-27205
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 3.0.0
Reporter: Jungtaek Lim
I found this bug while testing my patch regarding Spark SQL Kafka module - I
tend to open spark-shell and link kafka module via `–packages`.
When we run
{code:java}
./bin/spark-shell --packages
org.apache.spark:spark-sql-kafka-0-10_2.11:2.4.0{code}
we should be able to import "org.apache.kafka" in spark-shell, but it doesn't
work for current master branch.
There's not enough evidence as well as I have no idea what's happening here
even with `–verbose` option, so I had to spend couple of hours dealing with git
bisect.
Turned out the commit introducing the bug was
[81dd21fda99da48ed76adb739a07d1dabf1ffb51|https://github.com/apache/spark/commit/81dd21fda99da48ed76adb739a07d1dabf1ffb51].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
