Re: ExecuteScript Processor - Control Flow

2016-08-29 Thread James Wing
Koustav, subprocess.call will indeed wait for the shell script to exit, but I believe you need to clarify if the "run_sqoop_job.sh" script has logic to wait for the completion of the Sqoop job. Thanks, James On Mon, Aug 29, 2016 at 12:01 PM, koustav choudhuri wrote: >

Re: NiFi reference to process group

2016-08-29 Thread Joe Witt
Gunjan We have a feature proposal for this. Def a good idea that will help simplify flow development. It makes for process groups as functions. No clear time table for when it might happen but check out the feature proposal and comment on it and join the discussion. Thanks Joe On Aug 29,

NiFi reference to process group

2016-08-29 Thread Gunjan Dave
Hi Team, does NiFi currently have ability to reference a process group instead of making an actual connection. This would simplify the visual aspects of complex flows.

Re: Request for enhancement

2016-08-29 Thread Gunjan Dave
Hi Joe, I dont seem to have access to nifi jira to create one. Can it be given? Or if not, could someone help raising it. On Tue, Aug 30, 2016, 7:49 AM Joe Percivall wrote: > - Moving users list to BCC > > Hello Gunjan, > > This seems like a good potential idea. The

Re: Request for enhancement

2016-08-29 Thread Joe Percivall
- Moving users list to BCC Hello Gunjan, This seems like a good potential idea. The proper place to submit the suggestion is through the Apache NiFi Jira[1]. It can more easily be discussed and worked on there. [1] https://issues.apache.org/jira/browse/NIFI Suggestions/ideas from users are

Re: Request for enhancement

2016-08-29 Thread Gunjan Dave
Seems like below didnot get delivered. On Mon, Aug 29, 2016, 12:30 PM Gunjan Dave wrote: > Hi Team, > I would like to propose if the following enhacement if seen feasible can > be incorporated in the provenance graph. > > Current graph only shows the type, rather i

Re: ExecuteScript Processor - Control Flow

2016-08-29 Thread koustav choudhuri
[image: Inline image 2] This is the code that I am using to call the shell script . On Mon, Aug 29, 2016 at 4:15 AM, Nathamuni, Ramanujam wrote: > I do have similar question – as I have the Execute script using python to > run code and it produces the output file

RE: Kill-and-Fill Pattern?

2016-08-29 Thread Peter Wicks (pwicks)
Toivo, I started down this path, but then came up with a broader solution (which I have not tested): 1. Do a normal JSONToSQL 2. Use MergeContent to group all of the FlowFiles from the same batch into a single new FlowFile using FlowFile Stream Merge Format. 3. Update

RE: ExecuteScript Processor - Control Flow

2016-08-29 Thread Nathamuni, Ramanujam
I do have similar question – as I have the Execute script using python to run code and it produces the output file (/tmp/test.xml) but not sure how to use that file to next processor without using additional flow file using GetFile processor to get a file produced by python execute script.

Re: Configuration Management of Flows - Proposed Book of Work

2016-08-29 Thread Gunjan Dave
Thanks Joe, i'll try to digest the information. In mean time, could you kindly help if there is any specific jira's or links in terms of what has improved in NiFi 1.0 for using templates with version control and diff tool, that alone at the moment should solve lot of problems, i guess. In the

Re: Configuration Management of Flows - Proposed Book of Work

2016-08-29 Thread Joe Witt
Gunjan We've long since supported the concept of flow templates. These are powerful because they allow you to save, share, import already designed flows. They have to date had their full potential limited by a few things: 1) The exported template was non-deterministic in terms of the XML it

Configuration Management of Flows - Proposed Book of Work

2016-08-29 Thread Gunjan Dave
Hello NiFi Team, I understand that config mgmt of flows is part of the proposed road map. Is there any tangible action which has started on this front? Any tentative release plan even if very preliminary? Any plans to integrate with Git internally within the framework itself to version manage?