shawn - i believe it is related to our default settings and have phoned a friend to jump in here when able. but default retention and default sharding i *think* can result in this. You can generate a thread dump before and after the locked state to see what it is stuck/sitting on. That will help here
Thanks On Mon, Aug 10, 2020 at 10:24 AM Shawn Weeks <swe...@weeksconsulting.us> wrote: > Out of the box even the initial admin user has to be granted permission I > think, mine worked fine for several months since 1.11.1 was released and > just started having an issues a couple of weeks ago. I’ve increasing the > retention time a bit to see if that improves the situation a bit. > > > > Thanks > > Shawn Weeks > > > > *From: *Wyll Ingersoll <wyllys.ingers...@keepertech.com> > *Reply-To: *"users@nifi.apache.org" <users@nifi.apache.org> > *Date: *Monday, August 10, 2020 at 12:22 PM > *To: *"users@nifi.apache.org" <users@nifi.apache.org> > *Subject: *Re: Data Provenance Stops Working > > > > I run 1.11.4 in a cluster on AWS also and have a similar issue with the > provenance data, I can't ever view it. It's probably somehow misconfigured > but I haven't been able to figure it out. > ------------------------------ > > *From:* Andy LoPresto <alopre...@apache.org> > *Sent:* Monday, August 10, 2020 1:11 PM > *To:* users@nifi.apache.org <users@nifi.apache.org> > *Subject:* Re: Data Provenance Stops Working > > > > Shawn, > > > > I don’t know if this is specifically related, but there were a number of > critical issues discovered in the 1.11.x release line that have been fixed > in 1.11.4. I would not recommend running any prior version on that release > line. > > > > 1.12.0 should be coming imminently, so if you are going to upgrade anyway, > you may want to wait a week or so and get the newest bits with hundreds of > new features, but for stability alone, I would strongly recommend 1.11.4. > > > > https://cwiki.apache.org/confluence/display/NIFI/Release+Notes > > > > > > Andy LoPresto > alopre...@apache.org > *alopresto.apa...@gmail.com <alopresto.apa...@gmail.com>* > He/Him > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > > > On Aug 10, 2020, at 10:00 AM, Shawn Weeks <swe...@weeksconsulting.us> > wrote: > > > > I’m running a three node NiFi Cluster on AWS EC2s using integrated > Zookeeper and SSL Enabled. Version is 1.11.1, OS is RedHat 7.7, Java is > 1.8.0_242. For some reason after a period of time Data Provenance goes > blank, old records are no longer queryable and new data provenance doesn’t > appear to get written. No Lucene or other exceptions are logged and > restarting the node causes data provenance to go back to being written > however old data provenance does not re-appear. No exceptions appear when > querying data provenance. All tests have been run as the initial admin user > and roles and permissions appear to be correct. I’ve also checked selinux > audit logs to see if something is being blocked but nothing appears. > > > > WriteAheadProvenanceRepository, max storage is set to 24 hours, 1GB, 30 > seconds for rollover, 100mb rollover size, 2 query threads, 2 index > threads, compress on rollover, and don’t sync always. > > > > Thanks > > Shawn Weeks > > >
