[
https://issues.apache.org/jira/browse/SPARK-5267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14293043#comment-14293043
]
Tathagata Das commented on SPARK-5267:
--------------------------------------
Hey this is a great initiative! However, we are trying to limit the number of
external dependencies in the Spark umbrella project for better manageability.
So while we want to add more functionality and integration with other systems,
we are very cautious about adding these dependencies into the Spark project. A
better place for such contributions is the http://spark-packages.org/ where
people contribute such functionality and maintain them on their own. I strongly
encourage you to add your camel integration to the spark-packages.
> Add a streaming module to ingest Apache Camel Messages from a configured
> endpoints
> ----------------------------------------------------------------------------------
>
> Key: SPARK-5267
> URL: https://issues.apache.org/jira/browse/SPARK-5267
> Project: Spark
> Issue Type: Improvement
> Components: Streaming
> Affects Versions: 1.2.0
> Reporter: Steve Brewin
> Labels: features
> Original Estimate: 120h
> Remaining Estimate: 120h
>
> The number of input stream protocols supported by Spark Streaming is quite
> limited, which constrains the number of systems with which it can be
> integrated.
> This proposal solves the problem by adding an optional module that integrates
> Apache Camel, which supports many additional input protocols. Our tried and
> tested implementation of this proposal is "spark-streaming-camel".
> An Apache Camel service is run on a separate Thread, consuming each
> http://camel.apache.org/maven/current/camel-core/apidocs/org/apache/camel/Message.html
> and storing it into Spark's memory. The provider of the Message is specified
> by any consuming component URI documented at
> http://camel.apache.org/components.html, making all of these protocols
> available to Spark Streaming.
> Thoughts?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
