Hi Mike, Registry's docker image is a little off from some of the other NiFi project configurations but the following approach should be fine and is fairly conservative in how it is carried out.
1. Exec into the container - *docker exec -it <registry container name/id> /bin/bash* 2. Inside container: stop Registry, and have your interactive session terminated as PID 1 was terminated - *./bin/nifi-registry.sh stop* 3. On your host, copy the h2 file to your host - *docker cp <registry container name/id> **/opt/nifi-registry/nifi-registry-0.1.0/database/nifi-registry.mv.db <path on host>* 4. Copy the database file in question to your volume for your new instance 5. Start the new container to verify import/migration 6. Cleanup the old container and files as appropriate 1 & 2 are questionable in terms of their necessity but with the potential for the container to get sigkilled, lend to a safer approach. The Java process doesn't seem to handle the sigterm from my quick testing of docker stop. You have the ability to restart where you left off on the original container via *docker start **<registry container name/id>*. I did a quick run through of these steps with vanilla versions of 0.1.0 and 0.2.0 and it seemed to work as anticipated. Let us know how it goes. On Thu, Jun 28, 2018 at 1:52 PM Kevin Doran <kdo...@apache.org> wrote: > Oh right. My mistake. Not sure if there is a way to stop the nifi registry > process without stopping the container. Other's more knowledgeable of > Docker might know... In that case I'd say it's worth a shot as that DB file > should be valid so long as nothing is being written to it. > > The adoption of the git persistence provider stuff is probably still small > given how new it is, but anecdotally, I've been using it for about a week > now, syncing to a github repo as a remote, and I've not run into any issues > with it nor have I heard of anyone else having a problem with it. It seems > pretty solid to me. One of the nice things about the git option is that the > format saved to git in the snapshots is compatible with a tool like NiFi > CLI, so (while some custom scripting still required), you should be in a > pretty good position to do disaster recovery to recreate a NiFi Registry > instance with your bucket/flow/flow-snapshot data just from the contents of > your git repository should you need to for whatever reason (again, would > need to script the right NiFi CLI commands to crawl the git directory tree, > but it's pretty straightforward as it's just a matter of parsing json and > yml files). One really nice feature is that deletes (a flow or bucket for > instance) happen as a new commit so you can even recover objects that were > deleted from NiFi Registry by accident, for instance. > > If your team/users are using NiFi Registry and have feedback, positive or > otherwise, let us know as it would be great to factor that in to the > existing backlog for what to target in future releases. > > Best, > Kevin > > If your team runs into any trouble with this, do let us know as I'm not > sure the deployment footprint > > On 6/28/18, 13:31, "Mike Thomsen" <mikerthom...@gmail.com> wrote: > > Kevin, > > The problem with our situation is that it's all inside Docker so I > don't > think I can just stop the registry and proceed. That's why I was > asking if > it sounded feasible to just say "hands off for 5 minutes while we copy > the > data." > > Also, what is the maturity level of the GitFlow option at this point? > That > was particularly attractive as a storage solution because it should > make it > much easier to export everything from our dev team to our end users. > > Thanks, > > Mike > > On Thu, Jun 28, 2018 at 1:28 PM Kevin Doran <kdo...@apache.org> wrote: > > > Mike, > > > > I don't know the extent of H2's capabilities, but for NiFi Registry > > changing the location of those files will require restarting the > server to > > pick up the new database location from nifi-registry.properties, > correct? > > So if I follow correctly what the objective is, my recommendation > would be > > the following steps: > > > > 1. Stop NiFi Registry > > 2. Move/copy the H2 database files to the Docker volume location > (should > > be safe to do when Registry is stopped) > > 3. Change nifi-registry.properties to point the database config > props to > > the new location > > 4. Start NiFi Registry > > > > If the goal is actually just to copy a snapshot of the H2 database > to the > > docker volume and not re-point NiFi Registry, which you want to keep > > running, I suppose you could do that. It should be fine to copy it if > > nothing is being written, but of course no guarantees and YMMV, so > at the > > least I'd try to verify that it is a validate db file after copying > it, > > using something like [1]. > > > > [1] > > > https://stackoverflow.com/questions/32727588/how-to-check-h2-database-health-and-corruption > > > > Hope this helps, > > Kevin > > > > On 6/28/18, 12:34, "Mike Thomsen" <mikerthom...@gmail.com> wrote: > > > > Kevin/Bryan, > > > > We have a bunch of users that are using Docker and might not > have set > > things up to point to a docker volume for the H2 database > location. Do > > either of you know if H2 will allow you to copy the files over > to the > > Docker volume while everything's running? > > > > Thanks, > > > > Mike > > > > On Wed, Jun 20, 2018 at 11:58 AM Bryan Bende <bbe...@gmail.com> > wrote: > > > > > Ah good point Mark... yes the old db properties are only need > this > > > first time so that it can auto-migrate the old DB to the new > one, > > > after that you don't need the old properties anymore. > > > > > > Thanks Kevin! > > > > > > On Wed, Jun 20, 2018 at 11:57 AM, Kevin Doran < > kdo...@apache.org> > > wrote: > > > > Thanks Mark and Bryan. I will add a NiFi Registry 0.1 -> 0.2 > > migration > > > guide to include these steps as part of updating the site with > news > > of the > > > new release. > > > > > > > > Thanks, > > > > Kevin > > > > > > > > ________________________________ > > > > From: Mark Bean <mark.o.b...@gmail.com> > > > > Sent: Wednesday, June 20, 2018 8:53:39 AM > > > > To: dev@nifi.apache.org > > > > Subject: Re: Upgrading NiFi Registry > > > > > > > > Thanks Bryan. There is actually another step not explicitly > > mentioned. At > > > > least for 0.1.0 -> 0.2.0, I needed to modify the > > nifi-registry.properties > > > > file as well. The 0.2.0 version has new properties/values > not in > > the > > > 0.1.0. > > > > And, I had to set the following for the database (using > values from > > > 0.1.0). > > > > This was only required on the first startup; these > properties can > > be > > > empty > > > > values on subsequent startups. > > > > nifi.registry.db.directory= > > > > nifi.registry.db.url.append= > > > > > > > > In the future, if using a database location that is external > to the > > > > installation directory, is nifi.registry.db.url the only > property > > that > > > > needs to be modified? > > > > > > > > > > > > On Wed, Jun 20, 2018 at 11:18 AM Bryan Bende < > bbe...@gmail.com> > > wrote: > > > > > > > >> Mark, > > > >> > > > >> The database directory and flow storage directory are where > all > > the > > > >> data are. By default these are created in the root of NiFi > > Registry, > > > >> so depending how you want to set it up you could move those > > > >> directories to the new install, or you could set them up to > be > > > >> external locations so you don't have to move them every > time, or > > you > > > >> could upgrade the lib directory of your current install and > leave > > > >> everything in place. > > > >> > > > >> The policies are the same as NiFi... stored in users.xml and > > > >> authorizations.xml the conf directory, depending how you > > configured > > > >> everything. So just copying those two files over to the new > > install. > > > >> > > > >> -Bryan > > > >> > > > >> > > > >> On Wed, Jun 20, 2018 at 11:13 AM, Mark Bean < > > mark.o.b...@gmail.com> > > > wrote: > > > >> > How does one upgrade the NiFi Registry? > > > >> > > > > >> > After unpacking the .tar.gz file, how does one get all > the flows > > > >> registered > > > >> > in a previous version of NiFi Registry into the newly > installed > > > version? > > > >> > And, how does one ensure all the policies transfer as > well? > > > >> > > > > >> > Thanks, > > > >> > Mark > > > >> > > > > > > > > > > > > > > >
