[
https://issues.apache.org/jira/browse/SPARK-13877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15195764#comment-15195764
]
Sean Owen commented on SPARK-13877:
-----------------------------------
(Copying my comment from the other JIRA as further discussion really concerns
this change.)
[Moving out the flume, MQTT modules, etc] is a little different in that it
started as part of the ASF project. It has been removed from the project (OK),
it's been forked and maintained by others outside the project (OK), and nobody
has now less access to it (i.e. I assume any committer would be added as a
project member if they wanted to). We have to make sure it's not presented from
official docs as still part of Spark, and can't release it together in a way
that suggests it's official (I assume we won't).
We also have to be careful this doesn't add up to appearing to take a part of a
community project "private". Those modules are so ancillary that I can't
imagine it's controversial. It's going to be more of an issue for the Kafka
integration (the topic here).
It's worth asking: do these concerns (which are real and painful) mean a
separate project is the answer, or just several modules?
> Consider removing Kafka modules from Spark / Spark Streaming
> ------------------------------------------------------------
>
> Key: SPARK-13877
> URL: https://issues.apache.org/jira/browse/SPARK-13877
> Project: Spark
> Issue Type: Sub-task
> Components: Spark Core, Streaming
> Affects Versions: 1.6.1
> Reporter: Hari Shreedharan
>
> Based on the discussion the PR for SPARK-13843
> ([here|https://github.com/apache/spark/pull/11672#issuecomment-196553283]),
> we should consider moving the Kafka modules out of Spark as well.
> Providing newer functionality (like security) has become painful while
> maintaining compatibility with older versions of Kafka. Moving this out
> allows more flexibility, allowing users to mix and match Kafka and Spark
> versions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
