A typical production setup is to use Kafka in the middle. Andrew
On Mon, Sep 18, 2017, 3:02 AM Margus Roo <mar...@roo.ee> wrote: > Hi > > I need to take flow with Spark streaming from Nifi port. As we know > Spark supports spark.streaming.receiver.maxRate and > spark.streaming.receiver.backpressure > > Seems that org.apache.nifi.spark.NiFiReceiver does support it at all. > > https://github.com/apache/nifi/tree/master/nifi-external/nifi-spark-receiver/src/main/java/org/apache/nifi/spark > - code is quite old too. > > My question is - what is the recommended aproach today getting stream > from nifi with Spark? Is > > https://github.com/apache/nifi/tree/master/nifi-external/nifi-spark-receiver/src/main/java/org/apache/nifi/spark > the best we have? If it is then what is the best aproach to integrate > spark maxRate or backpressure in it? > > -- > Margus (margusja) Roo > http://margus.roo.ee > skype: margusja > https://www.facebook.com/allan.tuuring > +372 51 48 780 > >
