[
https://issues.apache.org/jira/browse/SPARK-13877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15195843#comment-15195843
]
Hari Shreedharan commented on SPARK-13877:
------------------------------------------
I think we could easily make this a sub-project and not necessarily release it
with Spark itself. Moving all of these sub-projects out makes sense from a
development point of view, but the biggest benefit (since the
MQTT/Twitter/Flume modules see relatively few changes anyway) we get is if we
can move the Kafka module out so it is independent of the Spark version and can
be built against a matrix of spark and kafka (spark 1.6 x kafka 0.8, spark 2 x
kafka 0.8, spark 2 x kafka 0.9 etc) versions without having to further
complicate the Spark build.
> Consider removing Kafka modules from Spark / Spark Streaming
> ------------------------------------------------------------
>
> Key: SPARK-13877
> URL: https://issues.apache.org/jira/browse/SPARK-13877
> Project: Spark
> Issue Type: Sub-task
> Components: Spark Core, Streaming
> Affects Versions: 1.6.1
> Reporter: Hari Shreedharan
>
> Based on the discussion the PR for SPARK-13843
> ([here|https://github.com/apache/spark/pull/11672#issuecomment-196553283]),
> we should consider moving the Kafka modules out of Spark as well.
> Providing newer functionality (like security) has become painful while
> maintaining compatibility with older versions of Kafka. Moving this out
> allows more flexibility, allowing users to mix and match Kafka and Spark
> versions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
