Re: Processor with cron scheduling in middle of flow

2023-02-22 Thread John McGinn via users
Mark, Thanks for the comments regarding the bin packing settings. To confirm, I get data from multiple sources and merge them together. Basically, I start with a list of key values that I then need, enhance at a 1-1 level to a different data set, then go to another data source to get 1-N reco

Re: Processor with cron scheduling in middle of flow

2023-02-22 Thread Joe Witt
e flow when not to run. > > > -Ursprüngliche Nachricht- > > Von: Mark Payne > > Gesendet: Dienstag, 21. Februar 2023 21:37 > > An: users@nifi.apache.org; John McGinn > > Betreff: Re: Processor with cron scheduling in middle of flow > > > Key-Wor

Re: Processor with cron scheduling in middle of flow

2023-02-22 Thread Mark Payne
nifi.apache.org<mailto:users@nifi.apache.org>; John McGinn mailto:amruginn-n...@yahoo.com>> Betreff: Re: Processor with cron scheduling in middle of flow Key-Work IT-Sicherheit: Es handelt sich um eine externe E-Mail. Bitte nur auf Links oder Anhänge klicken, sofern die Echtheit der Nachricht k

Re: Processor with cron scheduling in middle of flow

2023-02-22 Thread Jens M. Kofoed
alled, but it is so more simpler being able to tell the processor in the middle of the flow when not to run. -Ursprüngliche Nachricht- Von: Mark Payne Gesendet: Dienstag, 21. Februar 2023 21:37 An: users@nifi.apache.org; John McGinn Betreff: Re: Processor with cron scheduling in midd

Re: Processor with cron scheduling in middle of flow

2023-02-22 Thread Mark Payne
ing able to > tell the processor in the middle of the flow when not to run. > > -Ursprüngliche Nachricht- > Von: Mark Payne > Gesendet: Dienstag, 21. Februar 2023 21:37 > An: users@nifi.apache.org; John McGinn > Betreff: Re: Processor with cron scheduling in middle of f

AW: Processor with cron scheduling in middle of flow

2023-02-22 Thread Dobbernack, Harald (Key-Work)
ocessor in the middle of the flow when not to run. -Ursprüngliche Nachricht- Von: Mark Payne Gesendet: Dienstag, 21. Februar 2023 21:37 An: users@nifi.apache.org; John McGinn Betreff: Re: Processor with cron scheduling in middle of flow Key-Work IT-Sicherheit: Es handelt sich um eine

Re: Processor with cron scheduling in middle of flow

2023-02-21 Thread Mark Payne
John, You should not be using CRON driven for any processors in the middle of a flow. In fact, we really should probably just disable that all together. In fact, it’s exceedingly rare that you’d want anything other than Timer-Driven with a Run Schedule of 0 sec. MergeContent will not create any

Re: Processor with cron scheduling in middle of flow

2023-02-21 Thread Joe Witt
John MergeContent cannot reliably work well with cron scheduling. That component is designed to get threads consistently so it can perform its bin packing function and time and size based kick out functions. If it ever worked with cron scheduling that was mostly on accident I'd say. Thanks Joe

Re: Processor with cron scheduling in middle of flow

2023-02-21 Thread John McGinn via users
I've created an XML template as a GitHub Gist of the possible MergeContent cron scheduling bug. The template was created in 1.12.0, and uploaded to 1.18.0. Worked in 1.12, didn't work in 1.18.0. I downloaded 1.20.0 locally, and it doesn't work there either. I have the initial GenerateFlowFile a

Processor with cron scheduling in middle of flow

2023-02-17 Thread John McGinn via users
Hello, NiFi 1.19.0 - I need some help in trying to make my idea work, or figure out the better way to do this. I've got a flow that retrieves data from a few data sources, enhances individual flow files, converts attributes to CSV and then merges that all back into a single file. It takes roug