A typical production setup is to use Kafka in the middle.
Andrew
On Mon, Sep 18, 2017, 3:02 AM Margus Roo wrote:
> Hi
>
> I need to take flow with Spark streaming from Nifi port. As we know
> Spark supports spark.streaming.receiver.maxRate and
> spark.streaming.receiver.backpressure
>
> Seems that org.apache.nifi.spark.NiFiReceiver does support it at all.
>
> https://github.com/apache/nifi/tree/master/nifi-external/nifi-spark-receiver/src/main/java/org/apache/nifi/spark
> - code is quite old too.
>
> My question is - what is the recommended aproach today getting stream
> from nifi with Spark? Is
>
> https://github.com/apache/nifi/tree/master/nifi-external/nifi-spark-receiver/src/main/java/org/apache/nifi/spark
> the best we have? If it is then what is the best aproach to integrate
> spark maxRate or backpressure in it?
>
> --
> Margus (margusja) Roo
> http://margus.roo.ee
> skype: margusja
> https://www.facebook.com/allan.tuuring
> +372 51 48 780
>
>
