Re: [discuss] NiFi 1.20 and NiFi 2.0
Hello Here is a good picture of what the 1.20 RC looks like. I've tagged several JIRAs today to ensure we get them in. A theme is really around stabilizing nifi/nifi-registry integration as we're seeing substantial uptick in usage and thus various community reported findings. We'll get that quite smooth with these included. https://issues.apache.org/jira/projects/NIFI/versions/12352581 Thanks On Mon, Jan 23, 2023 at 8:50 AM Joe Witt wrote: > Team, > > I'm going through the outstanding JIRAs/PRs and flagging which look like > they should be 'must have' for 1.20 and then will work the RC as soon as > those land. > > Hopefully have the RC up within a day or two but we'll see how these land > as some have review comments pending action. > > Thanks > > On Thu, Jan 12, 2023 at 2:53 AM Isha Lamboo < > isha.lam...@virtualsciences.nl> wrote: > >> Hi all, >> >> I would like to contribute to the migration tooling (mostly testing I >> suppose) when that comes up. >> >> My team's largest client has a completely template-based pipeline with >> external scripts replacing variable values before deploying to target >> clusters, so we've already started looking at this when the goals for 2.0 >> were discussed and approved. The migration to flowdefinitions and >> parameters is quite complex and we've hit several blockers when we tried to >> implement a direct translation. >> >> I expect that any time I spend helping to improve the tooling will pay >> off handsomely for our clients. >> >> Regards, >> >> Isha >> >> -Oorspronkelijk bericht- >> Van: Adam Taft >> Verzonden: woensdag 11 januari 2023 23:42 >> Aan: dev@nifi.apache.org >> Onderwerp: Re: [discuss] NiFi 1.20 and NiFi 2.0 >> >> This is really insightful and spot on ... >> >> Kevin wrote: >> > Good migration tooling will take a while to develop and test, and the >> > core contributors to that effort may not have sufficient variety of >> > flows to evaluate when the migration tools are "done" for the majority >> > of the community to have success upgrading to 2.x. A milestone release >> > would >> allow >> > us get more feedback on migration over a longer period than the vote >> window >> > of an RC candidate. >> >> It's exactly this case, that an early 2.0 release might not have had time >> to fully work its way through existing production deployments, that's >> concerning. The pace and voting of an "RC" is much too short to get any >> quality feedback from users in the field. >> >> I think it's really smart to consider the "Milestone" release approach >> here. We release 2.0.0-M1, 2.0.0-M2, ... waiting an adequate amount of time >> for feedback. We can put these milestones on a calendar, as needed, so that >> feedback is required some 'x' number of weeks/months after each milestone. >> >> And to this end, I'd personally rather see us keep the 'main' branch >> current with the 1.x line _until_ we're ready and are satisfied with the >> end goals of the 2.0 release objectives. When the milestone releases have >> been completed and there's a comfort level with the 2.x line, it's at the >> point we'd isolate the 1.x line into its own branch and switch main over to >> the 2.x line. >> >> This is an attractive way of: >> a) continuing business-as-usual with the 1.x line >> b) making headway on the 2.x release milestones >> c) giving adequate time for feedback against the 2.0 milestones coming >> from the field >> >> I don't mind the known-unknowns. But it's really the unknown-unknowns >> that are going to drive a delay in the 2.0 release. I think it's smart to >> be able to get some of the unknowns ironed out before we finalize the 2.0 >> release ceremony. The milestone approach really helps with that. >> >> /Adam >> >> On Wed, Jan 11, 2023 at 11:11 AM Kevin Doran wrote: >> >> > Sorry, Joe, I was not clear, and to be honest the two thoughts are >> > somewhat unrelated in my mind too :) >> > >> > I agree that good migration tooling is key. Otherwise, we risk users >> > staying on 1.x or creating a schism of 1.x and 2.x users. >> > >> > Good migration tooling will take a while to develop and test, and the >> > core contributors to that effort may not have sufficient variety of >> > flows to evaluate when the migration tools are "done" for the majority >> > of the community to have success upgrading to 2.x. A milestone release >> > would allow us get more feedback on migration over a longer period >> > than the vote window of an RC candidate. >> > >> > Perhaps we could continue to release from the 1.x line (including >> > minor releases with some features) until we are ready to drop the >> "milestone" >> > qualifier from 2.0.0, and only then put 1.x into maintenance-only >> status. >> > It would be the same proposal to move main to target 2.0.0-M1, but >> > relax restrictions for what can land on the 1.x branch and be open to >> > a 1.21, 1.22, etc. if 2.0.0 work takes longer than anticipated. For >> > example, maybe we would be open to landing new/backported p
Re: Fw: ICLA Registration
Harry
Yep agreed on the terms being confusing and yep you're all set to file
JIRAs, create PRs, and so on. Very able to contribute readily at this
point. The ICLA on hand is a fine step regardless so thanks for that too.
Joe
On Mon, Jan 23, 2023 at 10:47 AM Harry Clarke
wrote:
> Apologies for splitting the thread, but I don't see how to reply
> retrospectively.
>
> (However, I have added myself to the mailing list going forward)
>
> # Confusion
> There has definitely been confusion on my end between contributors and
> committers, what is required of each of them, and what they have access to.
>
> I didn't request anything from Craig beyond "Please could you document my
> ICLA?" so mentioning of account requests through Whimsy was offered freely.
> There's also slight confusion, I believe, in the wording on the roles (
> https://apache.org/foundation/how-it-works.html#roles), where CLAs are
> listed as a step for committers ("and has a signed Contributor License
> Agreement (CLA) on file").
>
> # Request
> All I want is the ability to create pull requests and discuss issues in
> Jira, which it sounds like I'm set up to do, so thank you for your help!
>
>
> From: Joe Witt
> Sent: 23 January 2023 16:52
> To: harry-cla...@outlook.com
> Subject: Fwd: Fw: ICLA Registration
>
> Please see this thread. Please respond on the dev list to clarify your
> request since that thread is there.
>
> Thanks for your interest in being a contributor.
>
> Thanks
>
> -- Forwarded message -
> From: Joe Witt mailto:joe.w...@gmail.com>>
> Date: Mon, Jan 23, 2023 at 9:51 AM
> Subject: Re: Fw: ICLA Registration
> To: mailto:dev@nifi.apache.org>>
>
>
> Harry
>
>
>
> First please subscribe to this dev mailing list so that we do not have to
> moderate your posts and so that you will see the replies.
>
> You requested a JIRA account and that has been setup.
>
> This thread appears to be you submitting your ICLA to Apache and it
> appears to be received. It appears based on Craig's response (ASF
> Secretary) that your ICLA is received and that he thinks you are now
> looking for an Apache account/user/email perhaps. If that is the case can
> you please state that and your request overall - I dont see whatever came
> before Craig's response to you. I am not aware of us setting up any such
> account for someone prior to them becoming a committer so that would not be
> something we'd pursue at this point. We're of course happy you're planning
> to contribute and indeed that might lead to being offered committership
> down the road. This document [1] might have some helpful context for you
> as well.
>
>
> [1]
> https://cwiki.apache.org/confluence/display/NIFI/Progression+from+user+to+Project+Management+Committee
>
> Thanks
> Joe
>
> On Mon, Jan 23, 2023 at 9:45 AM Harry Clarke > wrote:
> Hey all,
>
> Is this something you'd be able to help me with?
>
> Thanks,
> Harry C
>
> From: Craig L Russell mailto:c...@apache.org>>
> Sent: 21 January 2023 20:34
> To: Harry Clarke mailto:harry-cla...@outlook.com
> >>
> Cc: secret...@apache.org <
> secret...@apache.org>; Clarke, Harry <
> harry.cla...@gs.com>; gs-o...@gs.com gs-o...@gs.com> mailto:gs-o...@gs.com>>
> Subject: Re: ICLA Registration
>
> Dear Harry Clarke,
>
> Thanks for the ICLA, however we already have a copy.
> The (P)PMC can use the existing ICLA to request your account by using
> Whimsy:
>
> https://whimsy.apache.org/officers/acreq.cgi (PMC chair and ASF members
> only)
>
> Please take up any further queries with the (P)PMC.
>
> Warm Regards,
>
> --
> Craig L Russell
> Assistant Secretary, Apache Software Foundation
>
Re: Fw: ICLA Registration
Apologies for splitting the thread, but I don't see how to reply
retrospectively.
(However, I have added myself to the mailing list going forward)
# Confusion
There has definitely been confusion on my end between contributors and
committers, what is required of each of them, and what they have access to.
I didn't request anything from Craig beyond "Please could you document my
ICLA?" so mentioning of account requests through Whimsy was offered freely.
There's also slight confusion, I believe, in the wording on the roles
(https://apache.org/foundation/how-it-works.html#roles), where CLAs are listed
as a step for committers ("and has a signed Contributor License Agreement (CLA)
on file").
# Request
All I want is the ability to create pull requests and discuss issues in Jira,
which it sounds like I'm set up to do, so thank you for your help!
From: Joe Witt
Sent: 23 January 2023 16:52
To: harry-cla...@outlook.com
Subject: Fwd: Fw: ICLA Registration
Please see this thread. Please respond on the dev list to clarify your request
since that thread is there.
Thanks for your interest in being a contributor.
Thanks
-- Forwarded message -
From: Joe Witt mailto:joe.w...@gmail.com>>
Date: Mon, Jan 23, 2023 at 9:51 AM
Subject: Re: Fw: ICLA Registration
To: mailto:dev@nifi.apache.org>>
Harry
First please subscribe to this dev mailing list so that we do not have to
moderate your posts and so that you will see the replies.
You requested a JIRA account and that has been setup.
This thread appears to be you submitting your ICLA to Apache and it appears to
be received. It appears based on Craig's response (ASF Secretary) that your
ICLA is received and that he thinks you are now looking for an Apache
account/user/email perhaps. If that is the case can you please state that and
your request overall - I dont see whatever came before Craig's response to you.
I am not aware of us setting up any such account for someone prior to them
becoming a committer so that would not be something we'd pursue at this point.
We're of course happy you're planning to contribute and indeed that might lead
to being offered committership down the road. This document [1] might have
some helpful context for you as well.
[1]
https://cwiki.apache.org/confluence/display/NIFI/Progression+from+user+to+Project+Management+Committee
Thanks
Joe
On Mon, Jan 23, 2023 at 9:45 AM Harry Clarke
mailto:harry-cla...@outlook.com>> wrote:
Hey all,
Is this something you'd be able to help me with?
Thanks,
Harry C
From: Craig L Russell mailto:c...@apache.org>>
Sent: 21 January 2023 20:34
To: Harry Clarke mailto:harry-cla...@outlook.com>>
Cc: secret...@apache.org
mailto:secret...@apache.org>>; Clarke, Harry
mailto:harry.cla...@gs.com>>;
gs-o...@gs.com mailto:gs-o...@gs.com>>
Subject: Re: ICLA Registration
Dear Harry Clarke,
Thanks for the ICLA, however we already have a copy.
The (P)PMC can use the existing ICLA to request your account by using Whimsy:
https://whimsy.apache.org/officers/acreq.cgi (PMC chair and ASF members only)
Please take up any further queries with the (P)PMC.
Warm Regards,
--
Craig L Russell
Assistant Secretary, Apache Software Foundation
Re: Fw: ICLA Registration
Harry First please subscribe to this dev mailing list so that we do not have to moderate your posts and so that you will see the replies. You requested a JIRA account and that has been setup. This thread appears to be you submitting your ICLA to Apache and it appears to be received. It appears based on Craig's response (ASF Secretary) that your ICLA is received and that he thinks you are now looking for an Apache account/user/email perhaps. If that is the case can you please state that and your request overall - I dont see whatever came before Craig's response to you. I am not aware of us setting up any such account for someone prior to them becoming a committer so that would not be something we'd pursue at this point. We're of course happy you're planning to contribute and indeed that might lead to being offered committership down the road. This document [1] might have some helpful context for you as well. [1] https://cwiki.apache.org/confluence/display/NIFI/Progression+from+user+to+Project+Management+Committee Thanks Joe On Mon, Jan 23, 2023 at 9:45 AM Harry Clarke wrote: > Hey all, > > Is this something you'd be able to help me with? > > Thanks, > Harry C > > From: Craig L Russell > Sent: 21 January 2023 20:34 > To: Harry Clarke > Cc: secret...@apache.org ; Clarke, Harry < > harry.cla...@gs.com>; gs-o...@gs.com > Subject: Re: ICLA Registration > > Dear Harry Clarke, > > Thanks for the ICLA, however we already have a copy. > The (P)PMC can use the existing ICLA to request your account by using > Whimsy: > > https://whimsy.apache.org/officers/acreq.cgi (PMC chair and ASF members > only) > > Please take up any further queries with the (P)PMC. > > Warm Regards, > > -- > Craig L Russell > Assistant Secretary, Apache Software Foundation >
Fw: ICLA Registration
Hey all, Is this something you'd be able to help me with? Thanks, Harry C From: Craig L Russell Sent: 21 January 2023 20:34 To: Harry Clarke Cc: secret...@apache.org ; Clarke, Harry ; gs-o...@gs.com Subject: Re: ICLA Registration Dear Harry Clarke, Thanks for the ICLA, however we already have a copy. The (P)PMC can use the existing ICLA to request your account by using Whimsy: https://whimsy.apache.org/officers/acreq.cgi (PMC chair and ASF members only) Please take up any further queries with the (P)PMC. Warm Regards, -- Craig L Russell Assistant Secretary, Apache Software Foundation
Re: [discuss] NiFi 1.20 and NiFi 2.0
Team, I'm going through the outstanding JIRAs/PRs and flagging which look like they should be 'must have' for 1.20 and then will work the RC as soon as those land. Hopefully have the RC up within a day or two but we'll see how these land as some have review comments pending action. Thanks On Thu, Jan 12, 2023 at 2:53 AM Isha Lamboo wrote: > Hi all, > > I would like to contribute to the migration tooling (mostly testing I > suppose) when that comes up. > > My team's largest client has a completely template-based pipeline with > external scripts replacing variable values before deploying to target > clusters, so we've already started looking at this when the goals for 2.0 > were discussed and approved. The migration to flowdefinitions and > parameters is quite complex and we've hit several blockers when we tried to > implement a direct translation. > > I expect that any time I spend helping to improve the tooling will pay off > handsomely for our clients. > > Regards, > > Isha > > -Oorspronkelijk bericht- > Van: Adam Taft > Verzonden: woensdag 11 januari 2023 23:42 > Aan: dev@nifi.apache.org > Onderwerp: Re: [discuss] NiFi 1.20 and NiFi 2.0 > > This is really insightful and spot on ... > > Kevin wrote: > > Good migration tooling will take a while to develop and test, and the > > core contributors to that effort may not have sufficient variety of > > flows to evaluate when the migration tools are "done" for the majority > > of the community to have success upgrading to 2.x. A milestone release > > would > allow > > us get more feedback on migration over a longer period than the vote > window > > of an RC candidate. > > It's exactly this case, that an early 2.0 release might not have had time > to fully work its way through existing production deployments, that's > concerning. The pace and voting of an "RC" is much too short to get any > quality feedback from users in the field. > > I think it's really smart to consider the "Milestone" release approach > here. We release 2.0.0-M1, 2.0.0-M2, ... waiting an adequate amount of time > for feedback. We can put these milestones on a calendar, as needed, so that > feedback is required some 'x' number of weeks/months after each milestone. > > And to this end, I'd personally rather see us keep the 'main' branch > current with the 1.x line _until_ we're ready and are satisfied with the > end goals of the 2.0 release objectives. When the milestone releases have > been completed and there's a comfort level with the 2.x line, it's at the > point we'd isolate the 1.x line into its own branch and switch main over to > the 2.x line. > > This is an attractive way of: > a) continuing business-as-usual with the 1.x line > b) making headway on the 2.x release milestones > c) giving adequate time for feedback against the 2.0 milestones coming > from the field > > I don't mind the known-unknowns. But it's really the unknown-unknowns that > are going to drive a delay in the 2.0 release. I think it's smart to be > able to get some of the unknowns ironed out before we finalize the 2.0 > release ceremony. The milestone approach really helps with that. > > /Adam > > On Wed, Jan 11, 2023 at 11:11 AM Kevin Doran wrote: > > > Sorry, Joe, I was not clear, and to be honest the two thoughts are > > somewhat unrelated in my mind too :) > > > > I agree that good migration tooling is key. Otherwise, we risk users > > staying on 1.x or creating a schism of 1.x and 2.x users. > > > > Good migration tooling will take a while to develop and test, and the > > core contributors to that effort may not have sufficient variety of > > flows to evaluate when the migration tools are "done" for the majority > > of the community to have success upgrading to 2.x. A milestone release > > would allow us get more feedback on migration over a longer period > > than the vote window of an RC candidate. > > > > Perhaps we could continue to release from the 1.x line (including > > minor releases with some features) until we are ready to drop the > "milestone" > > qualifier from 2.0.0, and only then put 1.x into maintenance-only status. > > It would be the same proposal to move main to target 2.0.0-M1, but > > relax restrictions for what can land on the 1.x branch and be open to > > a 1.21, 1.22, etc. if 2.0.0 work takes longer than anticipated. For > > example, maybe we would be open to landing new/backported processors > > on the 1.x branch, but not core framework features or API changes. > > > > This might not be necessary, but I think it is fair that saying "no > > new features on 1.x" and also "no new features in 2.0.0" puts the > > project in a rough place if 2.0.0 takes longer than a few months, so > > if we go that route, we need to commit to a quick release of 2.0.0 > > that most users can move to easily. > > > > Thanks, > > Kevin > > > > On Jan 11, 2023 at 12:32:46, Joe Witt wrote: > > > > > Kevin, > > > > > > Yeah we can do whatever we want as far as 'releases' of 2.0 that are > > prior > >
Re: AccessApi CreateDownloadToken deprecated
Thanks David, that makes good sense and I just couldn't find the relevant Jira for myself earlier. On Mon, Jan 23, 2023 at 2:24 PM David Handermann < exceptionfact...@apache.org> wrote: > Hi Dan, > > NiFi 1.15.0 included the removal of One-Time Password Authentication as > described in NIFI-8931 [1]. > > Depending on the authentication strategy configured, the Access Token REST > API can be used to create a Bearer Token that can be used for a longer > duration. Configuring and authorizing X.509 Client Certificates is another > option for programmatic access, but there is no direct replacement for > One-Time Password Authentication. > > Regards, > David Handermann > > [1] https://issues.apache.org/jira/browse/NIFI-8931 > > On Mon, Jan 23, 2023 at 3:37 AM Daniel Chaffelson > wrote: > > > Hi folks, > > A user has created an Issue ticket[1] on NiPyAPI after upgrading NiFi > from > > 1.12 to 1.16. > > It seems that sometime in this window the AccessApi has had the > > CreateDownloadToken function deprecated. This function appears to have > > issued a one-time auth token to download flowfile content, and I cannot > see > > a replacement for this particular functionality. > > I can see that a user with the correct authz against Flowfile content > would > > be able to use the FlowfileQueuesApi to DownloadFlowFileContent once > their > > client had a bearer token, but this would be subtly different from a > > one-time token indicated in this older function. > > > > Can anyone advise on this deprecation and if the functionality moved > > elsewhere? > > > > [1] https://github.com/Chaffelson/nipyapi/issues/324 > > >
Re: AccessApi CreateDownloadToken deprecated
Hi Dan, NiFi 1.15.0 included the removal of One-Time Password Authentication as described in NIFI-8931 [1]. Depending on the authentication strategy configured, the Access Token REST API can be used to create a Bearer Token that can be used for a longer duration. Configuring and authorizing X.509 Client Certificates is another option for programmatic access, but there is no direct replacement for One-Time Password Authentication. Regards, David Handermann [1] https://issues.apache.org/jira/browse/NIFI-8931 On Mon, Jan 23, 2023 at 3:37 AM Daniel Chaffelson wrote: > Hi folks, > A user has created an Issue ticket[1] on NiPyAPI after upgrading NiFi from > 1.12 to 1.16. > It seems that sometime in this window the AccessApi has had the > CreateDownloadToken function deprecated. This function appears to have > issued a one-time auth token to download flowfile content, and I cannot see > a replacement for this particular functionality. > I can see that a user with the correct authz against Flowfile content would > be able to use the FlowfileQueuesApi to DownloadFlowFileContent once their > client had a bearer token, but this would be subtly different from a > one-time token indicated in this older function. > > Can anyone advise on this deprecation and if the functionality moved > elsewhere? > > [1] https://github.com/Chaffelson/nipyapi/issues/324 >
AccessApi CreateDownloadToken deprecated
Hi folks, A user has created an Issue ticket[1] on NiPyAPI after upgrading NiFi from 1.12 to 1.16. It seems that sometime in this window the AccessApi has had the CreateDownloadToken function deprecated. This function appears to have issued a one-time auth token to download flowfile content, and I cannot see a replacement for this particular functionality. I can see that a user with the correct authz against Flowfile content would be able to use the FlowfileQueuesApi to DownloadFlowFileContent once their client had a bearer token, but this would be subtly different from a one-time token indicated in this older function. Can anyone advise on this deprecation and if the functionality moved elsewhere? [1] https://github.com/Chaffelson/nipyapi/issues/324
