Aditya, MapR Stream is Kafka compatible so you could in theory just push from NiFi into the MapR cluster and get Spark to consume from there. However, if you want to bypass MapR streams then you can use Output ports as documented here:
https://blogs.apache.org/nifi/entry/stream_processing_nifi_and_spark and https://community.hortonworks.com/articles/12708/nifi-feeding-data-to-spark-streaming.html If I recall correctly, when using direct Spark to NiFi communications, the Spark job will behave like a NiFi site 2 site peer (be mindful of access controls) and consume data from there. If what you want is to upload data from the Spark job into NiFi then you once again have a few options one of them (easiest one?) is configuring NiFi to read straight out of MapR-FS and send data into HDFS. Cheers On Tue, Mar 7, 2017 at 9:32 PM, Aditya Gaurav (agaurav) <[email protected]> wrote: > Hi, > > We are building a system to consume an AVRO file for ingest into HDFS. We > are planning to use MAPR-STREAM for the same and found Nifi interesting for > handling input requests. > > I wanted to check if we can put a file to a Nifi port which can be read > via a Spark job? > > Pls let me know. > > > Thanks > Aditya >
