[jira] [Commented] (ZOOKEEPER-2936) Duplicate Keys in log4j.properties config files
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16379725#comment-16379725
]
ASF GitHub Bot commented on ZOOKEEPER-2936:
---
Github user asutosh936 commented on the issue:
https://github.com/apache/zookeeper/pull/473
Can someone please review and merge this PR.
> Duplicate Keys in log4j.properties config files
> ---
>
> Key: ZOOKEEPER-2936
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2936
> Project: ZooKeeper
> Issue Type: Bug
> Components: contrib, other
>Affects Versions: 3.4.8
> Environment: Apache ZooKeeper source tarball
>Reporter: Hari Sekhon
>Priority: Trivial
>
> Recent versions of ZooKeeper have introduced the following duplicate keys in
> to the contrib log4j.properties files.
> In this file:
> {code}
> ./zookeeper-3.4.8/contrib/rest/conf/log4j.properties
> {code}
> and this file:
> {code}
> ./zookeeper-3.4.8/src/contrib/rest/conf/log4j.properties
> {code}
> the following duplicate keys are found:
> {code}
> log4j.appender.ROLLINGFILE.layout=org.apache.log4j.PatternLayout
> log4j.appender.ROLLINGFILE.layout.ConversionPattern=%d{ISO8601} - %-5p
> [%t:%C{1}@%L] - %m%n
> {code}
> This was discovered because I've written file validators for most major
> formats which recurse all my github repos and this was failing my integration
> tests when pulling ZooKeeper source code. I actually added --exclude and
> --ignore-duplicate-keys switches to {code}validate_ini.py{code} to work
> around this and fix my builds for now but just remembered to raise this to
> you guys.
> The validator tools if you're interested can be found at:
> https://github.com/harisekhon/pytools
> Cheers
> Hari
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] zookeeper issue #473: ZOOKEEPER-2936 - Removed duplicate code
Github user asutosh936 commented on the issue: https://github.com/apache/zookeeper/pull/473 Can someone please review and merge this PR. ---
Success: ZOOKEEPER- PreCommit Build #1520
Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1520/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 78.14 MB...] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +0 tests included. The patch appears to be a documentation patch that doesn't require tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 core tests. The patch passed core unit tests. [exec] [exec] +1 contrib tests. The patch passed contrib unit tests. [exec] [exec] Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1520//testReport/ [exec] Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1520//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html [exec] Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1520//console [exec] [exec] This message is automatically generated. [exec] [exec] [exec] == [exec] == [exec] Adding comment to Jira. [exec] == [exec] == [exec] [exec] [exec] Error: No value specified for option "issue" [exec] [exec] [exec] == [exec] == [exec] Finished build. [exec] == [exec] == [exec] [exec] [exec] Unable to log in to server: https://issues.apache.org/jira/rpc/soap/jirasoapservice-v2 with user: hadoopqa. [exec] Cause: ; nested exception is: [exec] javax.net.ssl.SSLException: Received fatal alert: protocol_version [exec] mv: '/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess' and '/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess' are the same file BUILD SUCCESSFUL Total time: 18 minutes 28 seconds Archiving artifacts Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Recording test results Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 [description-setter] Could not determine description. Putting comment on the pull request Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Email was triggered for: Success Sending email for trigger: Success Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7 ### ## FAILED TESTS (if any) ## All tests passed
ZooKeeper_branch34 - Build # 2258 - Failure
See https://builds.apache.org/job/ZooKeeper_branch34/2258/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 124.92 KB...] [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 27.296 sec [junit] Running org.apache.zookeeper.test.RepeatStartupTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.101 sec [junit] Running org.apache.zookeeper.test.RestoreCommittedLogTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.253 sec [junit] Running org.apache.zookeeper.test.SaslAuthDesignatedClientTest [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.719 sec [junit] Running org.apache.zookeeper.test.SaslAuthDesignatedServerTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.59 sec [junit] Running org.apache.zookeeper.test.SaslAuthFailDesignatedClientTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.313 sec [junit] Running org.apache.zookeeper.test.SaslAuthFailNotifyTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.508 sec [junit] Running org.apache.zookeeper.test.SaslAuthFailTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.606 sec [junit] Running org.apache.zookeeper.test.SaslAuthMissingClientConfigTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.523 sec [junit] Running org.apache.zookeeper.test.SaslClientTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.078 sec [junit] Running org.apache.zookeeper.test.SessionInvalidationTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.621 sec [junit] Running org.apache.zookeeper.test.SessionTest [junit] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 33.074 sec [junit] Running org.apache.zookeeper.test.StandaloneTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.858 sec [junit] Running org.apache.zookeeper.test.StatTest [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.901 sec [junit] Running org.apache.zookeeper.test.StaticHostProviderTest [junit] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.422 sec [junit] Running org.apache.zookeeper.test.SyncCallTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.589 sec [junit] Running org.apache.zookeeper.test.TruncateTest [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.459 sec [junit] Running org.apache.zookeeper.test.UpgradeTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.334 sec [junit] Running org.apache.zookeeper.test.WatchedEventTest [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.093 sec [junit] Running org.apache.zookeeper.test.WatcherFuncTest [junit] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.488 sec [junit] Running org.apache.zookeeper.test.WatcherTest [junit] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.684 sec [junit] Running org.apache.zookeeper.test.ZkDatabaseCorruptionTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.509 sec [junit] Running org.apache.zookeeper.test.ZooKeeperQuotaTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.62 sec fail.build.on.test.failure: BUILD FAILED /home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34/build.xml:1474: The following error occurred while executing this line: /home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34/build.xml:1382: The following error occurred while executing this line: /home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34/build.xml:1385: Tests failed! Total time: 46 minutes 1 second Build step 'Execute shell' marked build as failure Archiving artifacts Recording test results Email was triggered for: Failure - Any Sending email for trigger: Failure - Any ### ## FAILED TESTS (if any) ## 1 tests failed. FAILED: org.apache.zookeeper.server.quorum.CnxManagerTest.testSocketTimeout Error Message: Waited more than necessary Stack Trace: junit.framework.AssertionFailedError: Waited more than necessary at org.apache.zookeeper.server.quorum.CnxManagerTest.testSocketTimeout(CnxManagerTest.java:383) at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:55)
[GitHub] zookeeper pull request #475: Fixed PurgeTxnLog params order in the zkCleanup...
GitHub user hamsterready opened a pull request: https://github.com/apache/zookeeper/pull/475 Fixed PurgeTxnLog params order in the zkCleanup.sh You can merge this pull request into a Git repository by running: $ git pull https://github.com/hamsterready/zookeeper hamsterready-patch-1 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zookeeper/pull/475.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #475 commit 08a20923cfb19cf5e45cdb5579ab0b858eb954f2 Author: Maciej Lopacinski Date: 2018-02-28T00:42:57Z Fixed PurgeTxnLog params order in the zkCleanup.sh ---
ZooKeeper-trunk - Build # 3744 - Failure
See https://builds.apache.org/job/ZooKeeper-trunk/3744/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 11.24 KB...] at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) Caused by: hudson.plugins.git.GitException: Command "git clean -fdx" returned status code 1: stdout: stderr: warning: failed to remove build/test/tmp/test7299294491915492585.junit.dir/data/version-2/snapshot.0 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1996) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1964) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1960) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1597) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1609) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.clean(CliGitAPIImpl.java:787) at hudson.plugins.git.GitAPI.clean(GitAPI.java:311) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:922) at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:896) at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:853) at hudson.remoting.UserRequest.perform(UserRequest.java:207) at hudson.remoting.UserRequest.perform(UserRequest.java:53) at hudson.remoting.Request$2.run(Request.java:358) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to H12 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1693) at hudson.remoting.UserResponse.retrieve(UserRequest.java:310) at hudson.remoting.Channel.call(Channel.java:908) at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:281) at com.sun.proxy.$Proxy110.clean(Unknown Source) at org.jenkinsci.plugins.gitclient.RemoteGitImpl.clean(RemoteGitImpl.java:450) at hudson.plugins.git.extensions.impl.CleanBeforeCheckout.decorateFetchCommand(CleanBeforeCheckout.java:30) at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:858) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1129) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1160) at hudson.scm.SCM.checkout(SCM.java:495) at hudson.model.AbstractProject.checkout(AbstractProject.java:1202) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499) at hudson.model.Run.execute(Run.java:1724) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) ERROR: Error fetching remote repo 'origin' [FINDBUGS] Skipping publisher since build result is FAILURE [WARNINGS] Skipping publisher since build result is FAILURE Archiving artifacts Recording fingerprints Recording test results ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error? Publishing Javadoc Email was triggered for: Failure - Any Sending email for trigger: Failure - Any ### ## FAILED TESTS (if any) ## No tests ran.
ZooKeeper-trunk-openjdk7 - Build # 1816 - Still Failing
See https://builds.apache.org/job/ZooKeeper-trunk-openjdk7/1816/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 61.94 KB...] [junit] Running org.apache.zookeeper.test.SaslClientTest in thread 2 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.215 sec, Thread: 2, Class: org.apache.zookeeper.test.SaslClientTest [junit] Running org.apache.zookeeper.test.ServerCnxnTest in thread 2 [junit] Running org.apache.zookeeper.test.SaslSuperUserTest in thread 1 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.145 sec, Thread: 1, Class: org.apache.zookeeper.test.SaslSuperUserTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 38.814 sec, Thread: 7, Class: org.apache.zookeeper.test.RestoreCommittedLogTest [junit] Running org.apache.zookeeper.test.SessionInvalidationTest in thread 7 [junit] Running org.apache.zookeeper.test.SessionTest in thread 1 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.638 sec, Thread: 2, Class: org.apache.zookeeper.test.ServerCnxnTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.788 sec, Thread: 7, Class: org.apache.zookeeper.test.SessionInvalidationTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 51.681 sec, Thread: 4, Class: org.apache.zookeeper.test.RecoveryTest [junit] Running org.apache.zookeeper.test.SessionTrackerCheckTest in thread 2 [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.155 sec, Thread: 2, Class: org.apache.zookeeper.test.SessionTrackerCheckTest [junit] Running org.apache.zookeeper.test.SessionUpgradeTest in thread 7 [junit] Running org.apache.zookeeper.test.StandaloneTest in thread 4 [junit] Running org.apache.zookeeper.test.StatTest in thread 2 [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.894 sec, Thread: 2, Class: org.apache.zookeeper.test.StatTest [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.958 sec, Thread: 4, Class: org.apache.zookeeper.test.StandaloneTest [junit] Running org.apache.zookeeper.test.StaticHostProviderTest in thread 2 [junit] Running org.apache.zookeeper.test.StringUtilTest in thread 4 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.135 sec, Thread: 4, Class: org.apache.zookeeper.test.StringUtilTest [junit] Running org.apache.zookeeper.test.SyncCallTest in thread 4 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.046 sec, Thread: 4, Class: org.apache.zookeeper.test.SyncCallTest [junit] Running org.apache.zookeeper.test.TruncateTest in thread 4 [junit] Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.181 sec, Thread: 2, Class: org.apache.zookeeper.test.StaticHostProviderTest [junit] Running org.apache.zookeeper.test.WatchEventWhenAutoResetTest in thread 2 [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.756 sec, Thread: 4, Class: org.apache.zookeeper.test.TruncateTest [junit] Running org.apache.zookeeper.test.WatchedEventTest in thread 4 [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.144 sec, Thread: 4, Class: org.apache.zookeeper.test.WatchedEventTest [junit] Running org.apache.zookeeper.test.WatcherFuncTest in thread 4 [junit] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.58 sec, Thread: 4, Class: org.apache.zookeeper.test.WatcherFuncTest [junit] Running org.apache.zookeeper.test.WatcherTest in thread 4 [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.168 sec, Thread: 7, Class: org.apache.zookeeper.test.SessionUpgradeTest [junit] Running org.apache.zookeeper.test.X509AuthTest in thread 7 [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.201 sec, Thread: 7, Class: org.apache.zookeeper.test.X509AuthTest [junit] Running org.apache.zookeeper.test.ZkDatabaseCorruptionTest in thread 7 [junit] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 34.183 sec, Thread: 1, Class: org.apache.zookeeper.test.SessionTest [junit] Running org.apache.zookeeper.test.ZooKeeperQuotaTest in thread 1 [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.282 sec, Thread: 2, Class: org.apache.zookeeper.test.WatchEventWhenAutoResetTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.335 sec, Thread: 1, Class: org.apache.zookeeper.test.ZooKeeperQuotaTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.818 sec, Thread: 7, Class: org.apache.zookeeper.test.ZkDatabaseCorruptionTest [junit] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3
[jira] [Commented] (ZOOKEEPER-2845) Data inconsistency issue due to retain database in leader election
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16378693#comment-16378693 ] ASF GitHub Bot commented on ZOOKEEPER-2845: --- Github user revans2 commented on the issue: https://github.com/apache/zookeeper/pull/453 Thanks @afine I closed them. > Data inconsistency issue due to retain database in leader election > -- > > Key: ZOOKEEPER-2845 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2845 > Project: ZooKeeper > Issue Type: Bug > Components: quorum >Affects Versions: 3.4.10, 3.5.3, 3.6.0 >Reporter: Fangmin Lv >Assignee: Robert Joseph Evans >Priority: Critical > Fix For: 3.5.4, 3.6.0, 3.4.12 > > > In ZOOKEEPER-2678, the ZKDatabase is retained to reduce the unavailable time > during leader election. In ZooKeeper ensemble, it's possible that the > snapshot is ahead of txn file (due to slow disk on the server, etc), or the > txn file is ahead of snapshot due to no commit message being received yet. > If snapshot is ahead of txn file, since the SyncRequestProcessor queue will > be drained during shutdown, the snapshot and txn file will keep consistent > before leader election happening, so this is not an issue. > But if txn is ahead of snapshot, it's possible that the ensemble will have > data inconsistent issue, here is the simplified scenario to show the issue: > Let's say we have a 3 servers in the ensemble, server A and B are followers, > and C is leader, and all the snapshot and txn are up to T0: > 1. A new request reached to leader C to create Node N, and it's converted to > txn T1 > 2. Txn T1 was synced to disk in C, but just before the proposal reaching out > to the followers, A and B restarted, so the T1 didn't exist in A and B > 3. A and B formed a new quorum after restart, let's say B is the leader > 4. C changed to looking state due to no enough followers, it will sync with > leader B with last Zxid T0, which will have an empty diff sync > 5. Before C take snapshot it restarted, it replayed the txns on disk which > includes T1, now it will have Node N, but A and B doesn't have it. > Also I included the a test case to reproduce this issue consistently. > We have a totally different RetainDB version which will avoid this issue by > doing consensus between snapshot and txn files before leader election, will > submit for review. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ZOOKEEPER-2845) Data inconsistency issue due to retain database in leader election
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16378692#comment-16378692 ] ASF GitHub Bot commented on ZOOKEEPER-2845: --- Github user revans2 closed the pull request at: https://github.com/apache/zookeeper/pull/455 > Data inconsistency issue due to retain database in leader election > -- > > Key: ZOOKEEPER-2845 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2845 > Project: ZooKeeper > Issue Type: Bug > Components: quorum >Affects Versions: 3.4.10, 3.5.3, 3.6.0 >Reporter: Fangmin Lv >Assignee: Robert Joseph Evans >Priority: Critical > Fix For: 3.5.4, 3.6.0, 3.4.12 > > > In ZOOKEEPER-2678, the ZKDatabase is retained to reduce the unavailable time > during leader election. In ZooKeeper ensemble, it's possible that the > snapshot is ahead of txn file (due to slow disk on the server, etc), or the > txn file is ahead of snapshot due to no commit message being received yet. > If snapshot is ahead of txn file, since the SyncRequestProcessor queue will > be drained during shutdown, the snapshot and txn file will keep consistent > before leader election happening, so this is not an issue. > But if txn is ahead of snapshot, it's possible that the ensemble will have > data inconsistent issue, here is the simplified scenario to show the issue: > Let's say we have a 3 servers in the ensemble, server A and B are followers, > and C is leader, and all the snapshot and txn are up to T0: > 1. A new request reached to leader C to create Node N, and it's converted to > txn T1 > 2. Txn T1 was synced to disk in C, but just before the proposal reaching out > to the followers, A and B restarted, so the T1 didn't exist in A and B > 3. A and B formed a new quorum after restart, let's say B is the leader > 4. C changed to looking state due to no enough followers, it will sync with > leader B with last Zxid T0, which will have an empty diff sync > 5. Before C take snapshot it restarted, it replayed the txns on disk which > includes T1, now it will have Node N, but A and B doesn't have it. > Also I included the a test case to reproduce this issue consistently. > We have a totally different RetainDB version which will avoid this issue by > doing consensus between snapshot and txn files before leader election, will > submit for review. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] zookeeper issue #453: ZOOKEEPER-2845: Apply commit log when restarting serve...
Github user revans2 commented on the issue: https://github.com/apache/zookeeper/pull/453 Thanks @afine I closed them. ---
[jira] [Commented] (ZOOKEEPER-2845) Data inconsistency issue due to retain database in leader election
[ https://issues.apache.org/jira/browse/ZOOKEEPER-2845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16378691#comment-16378691 ] ASF GitHub Bot commented on ZOOKEEPER-2845: --- Github user revans2 closed the pull request at: https://github.com/apache/zookeeper/pull/454 > Data inconsistency issue due to retain database in leader election > -- > > Key: ZOOKEEPER-2845 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2845 > Project: ZooKeeper > Issue Type: Bug > Components: quorum >Affects Versions: 3.4.10, 3.5.3, 3.6.0 >Reporter: Fangmin Lv >Assignee: Robert Joseph Evans >Priority: Critical > Fix For: 3.5.4, 3.6.0, 3.4.12 > > > In ZOOKEEPER-2678, the ZKDatabase is retained to reduce the unavailable time > during leader election. In ZooKeeper ensemble, it's possible that the > snapshot is ahead of txn file (due to slow disk on the server, etc), or the > txn file is ahead of snapshot due to no commit message being received yet. > If snapshot is ahead of txn file, since the SyncRequestProcessor queue will > be drained during shutdown, the snapshot and txn file will keep consistent > before leader election happening, so this is not an issue. > But if txn is ahead of snapshot, it's possible that the ensemble will have > data inconsistent issue, here is the simplified scenario to show the issue: > Let's say we have a 3 servers in the ensemble, server A and B are followers, > and C is leader, and all the snapshot and txn are up to T0: > 1. A new request reached to leader C to create Node N, and it's converted to > txn T1 > 2. Txn T1 was synced to disk in C, but just before the proposal reaching out > to the followers, A and B restarted, so the T1 didn't exist in A and B > 3. A and B formed a new quorum after restart, let's say B is the leader > 4. C changed to looking state due to no enough followers, it will sync with > leader B with last Zxid T0, which will have an empty diff sync > 5. Before C take snapshot it restarted, it replayed the txns on disk which > includes T1, now it will have Node N, but A and B doesn't have it. > Also I included the a test case to reproduce this issue consistently. > We have a totally different RetainDB version which will avoid this issue by > doing consensus between snapshot and txn files before leader election, will > submit for review. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] zookeeper pull request #455: ZOOKEEPER-2845: Apply commit log when restartin...
Github user revans2 closed the pull request at: https://github.com/apache/zookeeper/pull/455 ---
[GitHub] zookeeper pull request #454: ZOOKEEPER-2845: Apply commit log when restartin...
Github user revans2 closed the pull request at: https://github.com/apache/zookeeper/pull/454 ---
ZooKeeper-trunk-jdk8 - Build # 1391 - Failure
See https://builds.apache.org/job/ZooKeeper-trunk-jdk8/1391/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 60.30 KB...] [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.101 sec, Thread: 8, Class: org.apache.zookeeper.test.SaslClientTest [junit] Running org.apache.zookeeper.test.SaslSuperUserTest in thread 6 [junit] Running org.apache.zookeeper.test.ServerCnxnTest in thread 8 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.038 sec, Thread: 6, Class: org.apache.zookeeper.test.SaslSuperUserTest [junit] Running org.apache.zookeeper.test.SessionInvalidationTest in thread 6 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.344 sec, Thread: 6, Class: org.apache.zookeeper.test.SessionInvalidationTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.547 sec, Thread: 8, Class: org.apache.zookeeper.test.ServerCnxnTest [junit] Running org.apache.zookeeper.test.SessionTest in thread 6 [junit] Running org.apache.zookeeper.test.SessionTrackerCheckTest in thread 8 [junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.146 sec, Thread: 8, Class: org.apache.zookeeper.test.SessionTrackerCheckTest [junit] Running org.apache.zookeeper.test.SessionUpgradeTest in thread 8 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 96.463 sec, Thread: 3, Class: org.apache.zookeeper.test.RecoveryTest [junit] Running org.apache.zookeeper.test.StandaloneTest in thread 3 [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.787 sec, Thread: 3, Class: org.apache.zookeeper.test.StandaloneTest [junit] Running org.apache.zookeeper.test.StatTest in thread 3 [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.179 sec, Thread: 3, Class: org.apache.zookeeper.test.StatTest [junit] Running org.apache.zookeeper.test.StaticHostProviderTest in thread 3 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 61.382 sec, Thread: 5, Class: org.apache.zookeeper.test.RestoreCommittedLogTest [junit] Running org.apache.zookeeper.test.StringUtilTest in thread 5 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.105 sec, Thread: 5, Class: org.apache.zookeeper.test.StringUtilTest [junit] Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.133 sec, Thread: 3, Class: org.apache.zookeeper.test.StaticHostProviderTest [junit] Running org.apache.zookeeper.test.SyncCallTest in thread 5 [junit] Running org.apache.zookeeper.test.TruncateTest in thread 3 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.974 sec, Thread: 5, Class: org.apache.zookeeper.test.SyncCallTest [junit] Running org.apache.zookeeper.test.WatchEventWhenAutoResetTest in thread 5 [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.233 sec, Thread: 8, Class: org.apache.zookeeper.test.SessionUpgradeTest [junit] Running org.apache.zookeeper.test.WatchedEventTest in thread 8 [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.159 sec, Thread: 8, Class: org.apache.zookeeper.test.WatchedEventTest [junit] Running org.apache.zookeeper.test.WatcherFuncTest in thread 8 [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.805 sec, Thread: 3, Class: org.apache.zookeeper.test.TruncateTest [junit] Running org.apache.zookeeper.test.WatcherTest in thread 3 [junit] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.642 sec, Thread: 8, Class: org.apache.zookeeper.test.WatcherFuncTest [junit] Running org.apache.zookeeper.test.X509AuthTest in thread 8 [junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.139 sec, Thread: 8, Class: org.apache.zookeeper.test.X509AuthTest [junit] Running org.apache.zookeeper.test.ZkDatabaseCorruptionTest in thread 8 [junit] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 36.603 sec, Thread: 6, Class: org.apache.zookeeper.test.SessionTest [junit] Running org.apache.zookeeper.test.ZooKeeperQuotaTest in thread 6 [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.485 sec, Thread: 6, Class: org.apache.zookeeper.test.ZooKeeperQuotaTest [junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.011 sec, Thread: 5, Class: org.apache.zookeeper.test.WatchEventWhenAutoResetTest [junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.194 sec, Thread: 8, Class: org.apache.zookeeper.test.ZkDatabaseCorruptionTest [junit] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 32.086 sec, Thread: 3, Class: org.apache.zookeeper.test.WatcherTest [juni
