Kevin, The release cadence is a very loose schedule. Usually “point releases” (minor version upgrades) come approximately 10 - 14 weeks after the previous (i.e. 1.5.0 -> 1.6.0). In some cases, a bug fix release (1.5.0 -> 1.5.1) will be released on a shorter time frame if deemed necessary.
As 1.5.0 was released in mid-January, I would expect (note, am not committing to) a release vote towards mid-to-late April. In general, a "critical mass” of new features is reached and a committer/member of the PMC begins a release discussion thread on the mailing list, and the community weighs in with their thoughts on releasing a new version. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Mar 2, 2018, at 2:54 PM, Kevin Verhoeven <kevin.verhoe...@ds-iq.com> wrote: > > I am also interested in modifying concurrency based on dev/prod environment. > > Do you know when 1.6 will be released? Is there a published roadmap for the > releases? > > Thanks! > > Kevin > > From: Ed B [mailto:bdes...@gmail.com <mailto:bdes...@gmail.com>] > Sent: Friday, March 2, 2018 10:23 AM > To: users@nifi.apache.org <mailto:users@nifi.apache.org> > Subject: Re: setting processor concurrency based on the > development/production environment > > Boris, > as the WA - I believe that after deploying the flow on another env, you can > run post-deployment scripts. > You could use PUT rest API to update processors by ID. > <image001.png> > > > > On Fri, Mar 2, 2018 at 12:04 PM Andrew Grande <apere...@gmail.com > <mailto:apere...@gmail.com>> wrote: > There are 2 efforts, with somewhat different focus. You are already aware of > the community-driven nipyapi, but there's also an official module Ientioned > before, will be included with the 1.6 release. > > Andrew > > > On Fri, Mar 2, 2018, 8:59 AM Boris Tyukin <bo...@boristyukin.com > <mailto:bo...@boristyukin.com>> wrote: > Hi Andrew, > > thanks for the idea. I've been playing with nipyapi recently so might give > this a try. > > Thanks > > On Thu, Mar 1, 2018 at 7:32 PM, Andrew Grande <apere...@gmail.com > <mailto:apere...@gmail.com>> wrote: > Boris, > > Here's an idea youncould explore _today_. > > Assume your dev and prod flows live in different bucket/registry instance. > Given that you are trying out NiFi 1.6, you should be able to extract the > versioned flow from DEV and process it to change the concurrency level for > PROD before committing it to the prod registry instance. Any script which > understands json would do. nifi-toolkit-cli will take care of extracting and > moving flow versions. > > It's not ideal (yes, would like concurrency to be a customizable flow var), > and it assumes an explicit process to promote between environments, but > technically it is possible already. The user experience can be improved in > the future. > > Andrew > > > On Thu, Mar 1, 2018, 1:52 PM Kevin Doran <kdo...@apache.org > <mailto:kdo...@apache.org>> wrote: > I think you could put it under either project. Ultimately, if we go with that > approach, most (all?) of the logic/enhancement would be in the NiFi code base > during save version / import flow / change version operations, so probably > best to create it there. > > Glad you are finding NiFi useful. > > Cheers, > Kevin > > From: Boris Tyukin <bo...@boristyukin.com <mailto:bo...@boristyukin.com>> > Reply-To: <users@nifi.apache.org <mailto:users@nifi.apache.org>> > Date: Thursday, March 1, 2018 at 13:44 > To: <users@nifi.apache.org <mailto:users@nifi.apache.org>> > Subject: Re: setting processor concurrency based on the > development/production environment > > thanks Bryan and Kevin. I will be happy to open a jira - would it be a NiFi > jira or NiFi registry? <> > > I like the approach that Bryan suggested. > > I guess for now I will just color code the processors that need to be changed > in production. > > P.S. I really, really like where NiFi is going...I've looked at StreamSets > and Cask, but for my purposes, I was looking for a tool when I can process > various tables without creating a flow per table. I was able to create a very > simple flow in NiFi, that will handle 25 tables. My next project is to handle > 600 tables in near real-time. I just could see how I would do that with > StreamSets or Cask, when you have to create a pipeline per table. I was only > being able to do something similar with Apache Airflow, but airflow cannot do > things in near real-time. The concept of FlowFiles with attributes is a > genius idea, and I am blown away with all the possibilities to extend the > functionality of NiFi with custom processors and Groovy scripts. Awesome job, > guys. > > On Thu, Mar 1, 2018 at 1:29 PM, Kevin Doran <kdo...@apache.org > <mailto:kdo...@apache.org>> wrote: > Hi Boris, > > Good point regarding concurrent tasks; thanks for sharing! > > This is a great candidate for something that one should be able to create > environment-specific values for, as Bryan suggests. I agree we should create > a NiFi JIRA to track this enhancement. > > Thanks, > Kevin > > On 3/1/18, 11:44, "Bryan Bende" <bbe...@gmail.com <mailto:bbe...@gmail.com>> > wrote: > > Hello, > > Glad you are having success with NiFi + NiFi Registry! > > You brought up an interesting point about the concurrent tasks... > > I think we may want to consider making the concurrent tasks work > similar to variables, in that we capture the concurrent tasks that the > flow was developed with and would use it initially, but then if you > have modified this value in the target environment it would not > trigger a local change and would be retained across upgrades so that > you don't have to reset it. > > For now you could probably always leave the versioned flow with the > lower value of 2, then once you are in prod you bump it to 4 until the > next upgrade is available, you then revert the local changes, do the > upgrade, and put it back to 4, but its not ideal because it shows a > local change the entire time. > > I don't think there is much you can do differently right now, but I > think this is a valid case to create a JIRA for. > > Thanks, > > Bryan > > On Thu, Mar 1, 2018 at 11:29 AM, Boris Tyukin <bo...@boristyukin.com > <mailto:bo...@boristyukin.com>> wrote: > > Hello NiFi community, > > > > started using NiFi recently and fell in love with it! We run 1.6 NiFi > alone > > with new NiFi registry and I am trying to figure out how to promote NiFi > > flow, created in VM environment to our cluster. > > > > One of the things is "Concurrent Tasks" processor parameter. I bump it > to 2 > > or 4 for some processors in my flow, when I develop / test it in VM. > > > > Then we deploy this flow to a beefy cluster node (with 48 cores) and > want to > > change concurrency to let's say 8 or 10 or 12 for some processors. > > > > Then I work on a new version/make some changes in my VM, and need to be > more > > shy with concurrency so set it back to 2 or 4. > > > > Then the story repeats... > > > > Is there a better way than to manually set this parameter? I do not > believe > > I can use a variable there and have to type the actual number of tasks. > > > > > > Thanks > > Boris >
signature.asc
Description: Message signed with OpenPGP using GPGMail
