I am afraid not. The whole point of Spark Streaming is to make it easy to
do complicated processing on streaming data while interoperating with core
Spark, MLlib, SQL without the operational overheads of maintain 4 different
systems. As a slight cost of achieving that unification, there maybe some
overheads compared to specialized systems that are designed to one specific
thing.
If you have to do something simple that could have been done using Flume,
then the resources needed by the Spark Streaming program shouldn't be too
high. Can you provide more details?
TD
On Thu, Apr 2, 2015 at 11:51 AM, Harut Martirosyan
harut.martiros...@gmail.com wrote:
Hi guys.
Is there a more lightweight way of stream processing with Spark? What we
want is a simpler way, preferably with no scheduling, which just streams
the data to destinations multiple.
We extensively use Spark Core, SQL, Streaming, GraphX, so it's our main
tool and don't want to add new things to the stack like Storm or Flume, but
from other side, it really takes much more resources on same streaming than
our previous setup with Flume, especially if we have multiple destinations
(triggers multiple actions/scheduling)
--
RGRDZ Harut
