Hi Peter, Can you review the configuration for your local state provider from state-management.xml for 1.26.0 and 2.0.0-M3 configurations?
From comparing the code in 1.26.0 vs 2.x, My working theory is that you have partitions set to the default of 16 for both old and new Nifi configurations. NiFI 1.26.0 uses <property name="Partitions">16</property> to decide how many partition subdirectories to create under your state base path (./state/local) directory. Partition directories for 1.26.0 are in your ./state/local/ and begin with prefix partition- But NiFi 2.x ignores the number specified by <property name="Partitions">16</property> and never creates any partition subdirectories as far I can see in the code. Therefore NiFI 2.x does NOT look for any partition subdirectories. This partition directory change in WriteAheadLocalStateProvider.java was a side effect of this change. MinimalLockingWriteAheadLog.java was the deprecated class that was removed. MinimalLockingWriteAheadLog had the logic for interacting with partition directories. https://issues.apache.org/jira/browse/NIFI-11833 Not sure if this is easy or not because I do not have local state partition directories handy to inspect, but as a workaround can you manually move/combine all the state information from the partition directories' files and merge them into a single file that you place directly in ./state/local Please let know what you find out, Peter, Jim Steinebrey > On Jul 9, 2024, at 4:13 PM, Sharp, Peter <[email protected]> wrote: > > Hello, > > I am reaching out with an issue I am facing when attempting to upgrade Nifi > from 1.26.0 to 2.0.0-M3. > > After migrating from 1.26.0 to 2.0.0-M3 all of the ListSFTP processors lose > their stored state. > > We are running Nifi as a standalone instance and are using local state > management. > > What I did and what has worked for me in the past with minor version upgrades > (1.23.2 to 1.26.0) is copying over the existing /state/local directory to the > new version. > > I validated that the state-mangement.xml file is referencing the local state > provider with the correct path to the state directory. I also validated the > state properties in the nifi.properties file are consistent with the old > version. > > When checking the app logs between the old running version (1.26.0) and the > new running version (2.0.0-M3) I am seeing some differences between where its > trying to load state. (see below screenshots of logs) > > 1.26.0 nifi-app-log - > <image002.png> > > 2.0.0-M3 nifi-app-log - > <image003.png> > > It appears that its unable to pick the snapshot file successfully even though > it exists in the /state/local directory. > > Any help on this would be greatly appreciated! > > Thanks ~ > > Peter Sharp > Software Engineer > Data & Analytics | ICE Mortgage Technology > M: +1 760-390-4895 > [email protected] <mailto:[email protected]> > www.ice.com <http://www.ice.com/> > <image001.png> > > The information contained in this message is proprietary and/or confidential. > If you are not the intended recipient, please: (i) delete the message and all > copies; (ii) do not disclose, distribute or use the message in any manner; > and (iii) notify the sender immediately. In addition, please be aware that > any message addressed to our domain is subject to archiving and review by > persons other than the intended recipient. Thank you.
