"The problem is that our PVCs aren't backed up, nor are they resilient
to an availability zone failure.
So I'm trying to find a method for backing up the flow files, but I've
not found anything for Kubernetes."
Yes, that's the problem; recommend configuring persistent storage for
the Kubernetes containers running NiFi.
On 10/23/2023 7:41 AM, David Dean via users wrote:
Hi everyone.
I'm running a single instance of NiFi on a Kubernetes cluster using
the the Helm chart from Cetic (I can't share the link here, it's on
GitHub).
This Helm chart uses PVCs for flow file storage, which works fine for
persisting flows between restarts of NiFi.
The problem is that our PVCs aren't backed up, nor are they resilient
to an availability zone failure.
So I'm trying to find a method for backing up the flow files, but I've
not found anything for Kubernetes.
Ideally I would like to write the flow files to a PostgreSQL database,
which is already present in the Kubernetes cluster and is backed up.
But from what I can read (and I couldn't easily be wrong!) I don't
think NiFi can persist flow files in a PostgreSQL database?
The other option would be to write the flow files to object storage,
like an S3 or Azure blob storage bucket, but I don't think NiFi can do
that either?
My question for anyone else running NiFi on Kubernetes - how do you
back up the flow files so they can be resolved in the event that the
PVCs are permanently lost?
Thanks.
