Ok, I guess the better question to as is
If NiFi is running in an ephemeral environment or mode such as kubernetes and docker with NO external volumes how can NiFi be backed up+recovered in event of a tragedy? Why not a simple S3 endpoint integration? Erik Anderson Bloomberg On Wed, Jan 16, 2019, at 10:25 AM, Bryan Bende wrote: > Hello, > > ZooKeeper is not used at all in stand alone mode. > > In a cluster it used for leader election, as well as to store state for > processors that maintain state, such as source processors that keep track of > a time stamp or id. > > -Bryan > > On Wed, Jan 16, 2019 at 7:49 AM Erik Anderson <eand...@pobox.com> wrote: >> Our NiFi is running in docker container. I am using some cool ideas from >> >> >> https://github.com/apache/nifi/blob/master/nifi-docker/dockerhub/sh/start.sh#L39 >> >> Simply set environment variables, stop and restart the container and the >> systems are completely reconfigured. >> >> At the moment, I am using an external volume for the nifi/conf directory. >> Inside this directory is so much contents. Provenance, flow files, canvas, >> NiFi state? >> >> I know if I kill the container and restart it, because of the external >> Docker volume, the system comes backup in its original state. The NiFi >> runtime is ephemeral but due to the external volume, it all restores. >> >> My Question is: >> If I start using Zookeeper, where even the nifi/conf directory becomes >> ephemeral, what is Zookeeper doing for me in both the single instance and >> clustered instances of NiFi. >> >> 1) Single Node NiFi >> - Does Zookeeper store the provenance events >> - Does Zookeeper store the state? >> - Does Zookeeper store the flows and canvas themselves? >> - Does Zookeeper also store the nifi.properties files and everything under >> nifi/conf? >> 2) Cluster mode - >> https://github.com/apache/nifi/blob/master/nifi-docker/dockerhub/sh/start.sh#L34 >> - What is Zookeeper doing in the cluster mode that its not doing in the >> Single Node? Just leader election processes? >> >> >> Thanks, >> Erik Anderson >> Bloomberg > -- > Sent from Gmail Mobile
