>> My question is: Is there a way to force the snapshot creation / sync from the leader?
3.5.6 will automatically create a clean snapshot as part of server start up process. So a snapshot should be available after initial upgrade and there is no need to force a snapshot creation. On Mon, Dec 9, 2019 at 2:11 AM Thomas Cooper <[email protected]> wrote: > Hi, > > I am working on upgrading a Kubernetes based 3.4.14 cluster to 3.5.6. > > I are hitting the issue where the new snapshot checks prevent the newly > upgraded 3.5.6 cluster from starting. This usually happens in our test > set-up as the 3.4.14 cluster is not very old and has not hit the snapCount > limit yet. Typically only one node (presumable the leader) has a snapshot > file but the other do not. > > I can disable the snapshot checks via the config value for the upgrade to > 3.5.6, however we obviously need to re-enable the checks ASAP. The issue is > making sure all of the ZK nodes have snapshots so that the checks can be > re-enabled. > > My question is: Is there a way to force the snapshot creation / sync from > the leader? > > As part of our upgrade process we need to restart the servers soon after > the initial upgrade of the ZK container images to enable config changes. If > the snapshot creation / sync hasn't happened by this point then obviously > the checks will fail. If we can force a snapshot creation, after the > initial upgrade, we can re-enable the checks as part of the upgrade > process. If not, we will need to create some method to check if the > snapshots have been created and periodically inspect the ZK server file > system, re-enabling the checks once they all have snapshots. > > Any pointers would be appreciated. > > Thanks in advance, > > Tom Cooper >
