Wendell created NIFI-4006: ----------------------------- Summary: As a spark developer, I want to add nifi-spark processors to easier integrate with Spark cluster Key: NIFI-4006 URL: https://issues.apache.org/jira/browse/NIFI-4006 Project: Apache NiFi Issue Type: New Feature Components: Extensions Reporter: Wendell
Some customers already had lots of spark job developed for ETL or other batch processes. When implemented NiFi for them, they did require an easier way to submit spark job from NiFi flow, and query spark job status. * Initial nifi-spark processors ** Submit asynchronous Spark v1.6.x job to Yarn cluster ** Fetch Spark v1.6.x job status from Yarn cluster ** Submit asynchronous Spark v2.1.x job to Yarn cluster ** Fetch Spark v2.1.x job status from Yarn cluster * Security Requirements ** Support Kerberized Yarn cluster -- This message was sent by Atlassian JIRA (v6.3.15#6346)