Creating Custom Receiver for Spark Streaming

[Something Something]

Is it safe to assume that Spark will always create a single instance of
Custom Receiver? Or would it create multiple instances on each node in a
cluster? Wondering if I need to worry about receiving the same message on
different nodes etc.

Please help. Thanks.

Re: Creating Custom Receiver for Spark Streaming

[Shixiong Zhu]

Each ReceiverInputDStream will create one Receiver. If you only use
one ReceiverInputDStream, there will be only one Receiver in the cluster.
But if you create multiple ReceiverInputDStreams, there will be multiple
Receivers.

Best Regards,
Shixiong Zhu

2015-10-12 23:47 GMT+08:00 Something Something :

> Is it safe to assume that Spark will always create a single instance of
> Custom Receiver? Or would it create multiple instances on each node in a
> cluster? Wondering if I need to worry about receiving the same message on
> different nodes etc.
>
> Please help. Thanks.
>