Josef, You're welcome, and thanks for the understanding reply.
I updated the Migration Guidance based on your suggestion, highlighting the issue when upgrading from 1.15.3 and earlier to 1.17.0 or 1.18.0. I am also tracking the SFTP issue described in NIFI-9989. This issue is resolved in the upstream SSHJ main branch, but the resolution for NiFi is waiting on a new release of the library. This will definitely be incorporated in upcoming releases, as timing allows. Thanks again for your reply and community participation! Regards, David Handermann [1] https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance On Fri, Oct 14, 2022 at 1:38 AM <josef.zahn...@swisscom.com> wrote: > Thank you guys for fixing it and for providing the workaround. We searched > in the release notes for any hints regarding the migration from 1.15.3 to > 1.18.0 and found nothing, may be a short notice would be great that a > upgrade from <.1.16.0 to 1.17.0/1.18.0 could brake some of the sensitive > properties. Not everybody can install every release, so it would help > others as well 😉… > > > > We don’t know yet how we will proceed. We are dealing as well with the > SFTP SSH_MSG_UNIMPLEMENTED message [1], but we wanted to test the newest > nifi version as there are a lot of bugfixes and new features. Due to the > complexity of our configuration it’s very likely that we run into an > upgrade issue which then needs to be fixed first. This is not a complaint, > it’s just a fact due to the complexity and because of the numbers of > changes on each and every nifi release in our case more or less on every > new release where we jump on, we saw some sort of a new introduced issue. > > > > We just love nifi and the community, it’s really an amazing tool! > > > > Cheers Josef > > > > > > [1] https://issues.apache.org/jira/browse/NIFI-9989 > > > > *From: *David Handermann <exceptionfact...@apache.org> > *Reply to: *"users@nifi.apache.org" <users@nifi.apache.org> > *Date: *Thursday, 13 October 2022 at 23:47 > *To: *"users@nifi.apache.org" <users@nifi.apache.org> > *Subject: *Re: NiFi 1.18.0 Sensitive Property broken after Upgrade > > > > Thanks for reporting this issue Josef, and thanks Mark for outlining the > background and workaround steps. > > > > I submitted the the following pull request to address the problem: > > https://github.com/apache/nifi/pull/6524 > > Regards, > > David Handermann > > > > On Thu, Oct 13, 2022 at 8:04 AM Mark Payne <marka...@hotmail.com> wrote: > > Hey Josef, > > > > I’m sorry about the trouble. It looks like this issue was reported here > [1]. We are looking into a fix for it. > > > > Fortunately, if you don’t want to wait for the fix there is a workaround > available. > > > > The work around is to follow these steps: > > 1. Instead of jumping straight to 1.18, update first to 1.16.4 > > 2. Start NiFi and wait for it to start up. Ensure that all looks healthy. > > 3. Shutdown NiFi > > 4. Upgrade to 1.18, ensuring that you copy over the conf/flow.json.gz file > from 1.16.4 > > > > So essentially, you’d need to upgrade from 1.15 to 1.16, and then to 1.18. > > > > The reason this works is that prior to 1.16, we stored the flow in > conf/flow.xml.gz. But in 1.16 we updated that to flow.json.gz - and also > kept around flow.xml.gz in order to make this change seemless. > > But it looks like when Sensitive Dynamic Properties was added, there was a > bug that caused us to not properly load things from flow.xml.gz, only from > flow.json.gz. > > So, if you upgrade first to 1.16.4, you’ll end up with a flow.json.gz that > you can then copy over to your 1.18 instance. > > > > I know this is not ideal, and I apologize for that. But if you’re looking > to upgrade right away this will be quicker than waiting for a resolution of > NIFI-10567. > > > > Thanks! > > -Mark > > > > [1] https://issues.apache.org/jira/browse/NIFI-10567 > > > > > > On Oct 13, 2022, at 8:28 AM, josef.zahn...@swisscom.com wrote: > > > > I just found this blog > https://exceptionfactory.com/posts/2022/08/02/implementing-apache-nifi-support-for-sensitive-dynamic-properties/ > about > the jira ticket > https://issues.apache.org/jira/browse/NIFI-9957?jql=text%20~%20%22sensitive%20dynamical%22 > . > What we found out it is that the controller DBCPConnectionPool is fine with > the password as well as the invokeHTTP. So for the ones where sensitive > dynamic properties has been enabled the migration of the password was > successful, but not for the others… > > > > Cheers Josef > > > > > > *From: *"Zahner Josef, GSB-LR-TRW-LI" <josef.zahn...@swisscom.com> > *Date: *Thursday, 13 October 2022 at 13:59 > *To: *"users@nifi.apache.org" <users@nifi.apache.org> > *Subject: *NiFi 1.18.0 Sensitive Property broken after Upgrade > > > > Hi guys > > > > We just upgraded from NiFi 1.15.3 to 1.18.0. We have somehow a migration > issue, it seems that all our sensitive properties are broken with 1.18.0. > Check my screenshot below, It’s related to controller services as well as > to processors. All sensitive properties shows us an error. If we replace > the password it’s fine, so it seems that the password got corrupt due to > the upgrade. Any hints? It leads to a ton of work, we have hundreds of > processors with a hardcoded password… I’ve seen that we can use external > password providers, do we have to migrate somehow? > > > > > > > > <image001.png> > > > > > > > > <image002.png> > > > > > > Cheers Josef > > > >
