[jira] [Commented] (BOOKKEEPER-588) SSL support
[ https://issues.apache.org/jira/browse/BOOKKEEPER-588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110352#comment-16110352 ] Hudson commented on BOOKKEEPER-588: --- FAILURE: Integrated in Jenkins build bookkeeper-master #1837 (See [https://builds.apache.org/job/bookkeeper-master/1837/]) BOOKKEEPER-588: SSL Support for Bookkeeper (sijie: [https://github.com/apache/bookkeeper/commit/8e0bd2c3d81b522e97434d8646915f36422a104b]) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PerChannelBookieClient.java * (add) bookkeeper-server/src/test/resources/keyStoreClientPassword.txt * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/auth/AuthProviderFactoryFactory.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/tls/SecurityHandlerFactory.java * (edit) bookkeeper-server/src/main/proto/BookkeeperProtocol.proto * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/ServerConfiguration.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/PendingAddOp.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/DefaultPerChannelBookieClientPool.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/ClientConfiguration.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/AbstractConfiguration.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieClient.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/BookieInitializationTest.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/replication/AuthAutoRecoveryTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BKException.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/util/LocalBookKeeper.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeperClientStats.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/tls/TLSContextFactory.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/auth/TestAuth.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/AuthHandler.java * (add) bookkeeper-server/src/test/resources/server.jks * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PerChannelBookieClientFactory.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/auth/ClientAuthProvider.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/tls/SecurityException.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookkeeperInternalCallbacks.java * (add) bookkeeper-server/src/test/resources/keyStoreServerPassword.txt * (add) bookkeeper-server/src/test/resources/trustStorePassword.txt * (edit) bookkeeper-server/pom.xml * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/auth/BookieAuthProvider.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieRequestProcessor.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/tls/SecurityProviderFactoryFactory.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieRequestHandler.java * (edit) pom.xml * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieNettyServer.java * (edit) bookkeeper-server/conf/bk_server.conf * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/ConnectionPeer.java * (add) bookkeeper-server/src/test/resources/client.jks * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookkeeperProtocol.java * (add) bookkeeper-server/src/test/java/org/apache/bookkeeper/tls/TestTLS.java * (add) bookkeeper-server/src/test/resources/cacerts * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieServer.java > SSL support > --- > > Key: BOOKKEEPER-588 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-588 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Ivan Kelly >Assignee: Enrico Olivelli > Fix For: 4.5.0 > > Attachments: 0001-MutualTLS-for-Bookkeeper.patch, > 0004-BOOKKEEPER-588-SSL-support-for-bookkeeper.patch > > > SSL support using startTLS -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (BOOKKEEPER-1102) org.apache.bookkeeper.client.BookKeeperDiskSpaceWeightedLedgerPlacementTest.testDiskSpaceWeightedBookieSelectionWithBookiesBeingAdded is unreliable
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108941#comment-16108941 ] Hudson commented on BOOKKEEPER-1102: FAILURE: Integrated in Jenkins build bookkeeper-master #1836 (See [https://builds.apache.org/job/bookkeeper-master/1836/]) BOOKKEEPER-1102: Clarify BookieInfoReader and fix associated test (sijie: [https://github.com/apache/bookkeeper/commit/a5f8580f53464065243a9af038935f5893434166]) * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/BookKeeperDiskSpaceWeightedLedgerPlacementTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/ClientConfiguration.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PerChannelBookieClient.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookieInfoReader.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookKeeperClusterTestCase.java > org.apache.bookkeeper.client.BookKeeperDiskSpaceWeightedLedgerPlacementTest.testDiskSpaceWeightedBookieSelectionWithBookiesBeingAdded > is unreliable > --- > > Key: BOOKKEEPER-1102 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1102 > Project: Bookkeeper > Issue Type: Bug > Components: bookkeeper-client >Affects Versions: 4.5.0 >Reporter: Samuel Just >Assignee: Samuel Just >Priority: Minor > Fix For: 4.5.0 > > > org.apache.bookkeeper.client.BookKeeperDiskSpaceWeightedLedgerPlacementTest.testDiskSpaceWeightedBookieSelectionWithBookiesBeingAdded > can intermittently fail depending on the timing of the client receiving the > info back from the bookies. > Additionally, the synchronization in BookieInfoReader is more complicated > than necessary and not entirely correct. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (BOOKKEEPER-935) Publish sources and javadocs to Maven Central
[ https://issues.apache.org/jira/browse/BOOKKEEPER-935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106131#comment-16106131 ] Hudson commented on BOOKKEEPER-935: --- FAILURE: Integrated in Jenkins build bookkeeper-master #1832 (See [https://builds.apache.org/job/bookkeeper-master/1832/]) BOOKKEEPER-935: Javadoc improvements and publish sources and javadocs to (sijie: [https://github.com/apache/bookkeeper/commit/cf2936548aa7f91e99bb53a8d53cbca567649535]) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/auth/BookieAuthProvider.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieNettyServer.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/auth/TestAuth.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/package-info.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/AuthHandler.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/AsyncCallback.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/MacDigestManager.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/sasl/SASLClientProviderFactory.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/ReadLastConfirmedAndEntryOp.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerMetadata.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookieInfoReader.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/ITopologyAwareEnsemblePlacementPolicy.java * (edit) pom.xml * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/common/annotation/InterfaceStability.java * (delete) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/ClientConnectionPeer.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/EnsemblePlacementPolicy.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/common/annotation/package-info.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/sasl/SASLClientAuthProvider.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/replication/ReplicationWorker.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/SpeculativeRequestExecutor.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/replication/AuthAutoRecoveryTest.java * (edit) bookkeeper-server/pom.xml * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/RegionAwareEnsemblePlacementPolicy.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/ClientConnectionPeer.java * (delete) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/BookieConnectionPeer.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookiesListener.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/common/annotation/InterfaceAudience.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/ReadLastConfirmedAndEntryContext.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/SpeculativeRequestExecutionPolicy.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/sasl/SASLBookieAuthProviderFactory.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PerChannelBookieClient.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LocalBookieEnsemblePlacementPolicy.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/WeightedRandomSelection.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeperClientStats.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/SynchCallbackUtils.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/common/package-info.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/auth/AuthProviderFactoryFactory.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/DefaultSpeculativeRequestExecutionPolicy.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/RackawareEnsemblePlacementPolicy.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeper.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerChecker.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BKException.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieConnectionPeer.java * (edit) bookkeeper-stats/pom.xml * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/sasl/SASLBookieAuthProvider.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/auth/ClientAuthProvider.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/DefaultEnsemblePlacementPolicy.java > Publish sources and javadocs to Maven Central >
[jira] [Commented] (BOOKKEEPER-1103) LedgerMetadataCreateTest bug in ledger id generation causes intermittent hang
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106132#comment-16106132 ] Hudson commented on BOOKKEEPER-1103: FAILURE: Integrated in Jenkins build bookkeeper-master #1832 (See [https://builds.apache.org/job/bookkeeper-master/1832/]) BOOKKEEPER-1103: Fix LedgerMetadataCreateTest random id loop (sijie: [https://github.com/apache/bookkeeper/commit/3908d9ce6a5530ae2ae8526b9988b983423c026c]) * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/LedgerMetadataCreationTest.java > LedgerMetadataCreateTest bug in ledger id generation causes intermittent hang > - > > Key: BOOKKEEPER-1103 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1103 > Project: Bookkeeper > Issue Type: Bug > Components: bookkeeper-server >Affects Versions: 4.5.0 >Reporter: Samuel Just >Assignee: Samuel Just >Priority: Minor > Fix For: 4.5.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (BOOKKEEPER-1104) BookieInitializationTest.testWithDiskFullAndAbilityToCreateNewIndexFile testcase is unreliable
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16104994#comment-16104994 ] Hudson commented on BOOKKEEPER-1104: FAILURE: Integrated in Jenkins build bookkeeper-master #1831 (See [https://builds.apache.org/job/bookkeeper-master/1831/]) BOOKKEEPER-1104: Fix testWithDiskFullAndAbilityToCreateNewIndexFile (jiazhai: [https://github.com/apache/bookkeeper/commit/0ccbc0e9c3e6a82d5e44b8347cc6567e043f253f]) * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/BookieInitializationTest.java > BookieInitializationTest.testWithDiskFullAndAbilityToCreateNewIndexFile > testcase is unreliable > -- > > Key: BOOKKEEPER-1104 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1104 > Project: Bookkeeper > Issue Type: Bug >Reporter: Samuel Just >Assignee: Samuel Just >Priority: Minor > Fix For: 4.5.0 > > > The bug is that the test sets the full threshhold really close the actual > usage (though just below), and if the disk is otherwise in use, the usage can > fall below that threshhold during the test. Instead, just set it to .001 > since that's not what we're testing here anyway. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (BOOKKEEPER-1017) Create documentation for ZooKeeper ACLs
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101681#comment-16101681 ] Hudson commented on BOOKKEEPER-1017: SUCCESS: Integrated in Jenkins build bookkeeper-master #1829 (See [https://builds.apache.org/job/bookkeeper-master/1829/]) BOOKKEEPER-1017: Create documentation for ZooKeeper ACLs (jiazhai: [https://github.com/apache/bookkeeper/commit/313cfa66018836e8d6372b06c29ec551fca91e34]) * (edit) bookkeeper-server/conf/bk_server.conf * (add) doc/bookkeeperSecurity.textile * (add) bookkeeper-server/conf/jaas_example.conf > Create documentation for ZooKeeper ACLs > --- > > Key: BOOKKEEPER-1017 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1017 > Project: Bookkeeper > Issue Type: Sub-task > Components: bookkeeper-client, bookkeeper-server, Documentation >Affects Versions: 4.5.0 >Reporter: Enrico Olivelli >Assignee: Enrico Olivelli >Priority: Blocker > Fix For: 4.5.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (BOOKKEEPER-1044) Entrylogger is not readding rolled logs back to the logChannelsToFlush list when exception happens while trying to flush rolled logs
[
https://issues.apache.org/jira/browse/BOOKKEEPER-1044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16101680#comment-16101680
]
Hudson commented on BOOKKEEPER-1044:
SUCCESS: Integrated in Jenkins build bookkeeper-master #1829 (See
[https://builds.apache.org/job/bookkeeper-master/1829/])
BOOKKEEPER-1044: Entrylogger is not readding rolled logs back to the
(eolivelli:
[https://github.com/apache/bookkeeper/commit/24ac8ead6240a845afccb4e606fbe0da1602])
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/EntryLogger.java
> Entrylogger is not readding rolled logs back to the logChannelsToFlush list
> when exception happens while trying to flush rolled logs
>
>
> Key: BOOKKEEPER-1044
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1044
> Project: Bookkeeper
> Issue Type: Bug
>Reporter: Charan Reddy Guttapalem
>Assignee: Sijie Guo
>Priority: Blocker
> Fix For: 4.5.0
>
>
> SyncThread.checkpoint(Checkpoint checkpoint) (which is called periodically by
> SyncThread's executor for every flushInterval) ultimately calls
> EntryLogger.flushRotatedLogs.
> In EntryLogger.flushRotatedLogs, first we set 'logChannelsToFlush' to null
> and then we try to flush and close individual file. Now, if IOException
> happens while trying to flush/close the logchannel, then exception is thrown
> as it is and it get propagates back upto SyncThread.checkpoint. Here we catch
> that IOException, log it and return without calling the checkpointComplete.
> But by now we lost reference of 'logChannelsToFlush' (rolled logs which are
> yet to be closed), because it is set to null before we try to flush/close
> individually rolledlogs. The next execution of 'checkpoint' (after
> flushinterval) wouldn't be knowing about the rolledlogs it failed to
> flush/close the previous time and it would flush the newly rolledlogs. So the
> failure of flush/close of the previous rolledlogs goes unnoticed completely.
> in EntryLogger.java
> void flushRotatedLogs() throws IOException {
> List channels = null;
> long flushedLogId = INVALID_LID;
> synchronized (this) {
> channels = logChannelsToFlush;
> logChannelsToFlush = null; <- here we set
> 'logChannelsToFlush' to null before it tries to flush/close rolledlogs
> }
> if (null == channels) {
> return;
> }
> for (BufferedLogChannel channel : channels) {
> channel.flush(true);
> call
> // since this channel is only used for writing, after flushing
> the channel,
> // we had to close the underlying file channel. Otherwise, we
> might end up
> // leaking fds which cause the disk spaces could not be reclaimed.
> closeFileChannel(channel);
> if (channel.getLogId() > flushedLogId) {
> flushedLogId = channel.getLogId();
> }
> LOG.info("Synced entry logger {} to disk.", channel.getLogId());
> }
> // move the leastUnflushedLogId ptr
> leastUnflushedLogId = flushedLogId + 1;
> }
> in SyncThread.java
> public void checkpoint(Checkpoint checkpoint) {
> try {
> checkpoint = ledgerStorage.checkpoint(checkpoint);
> } catch (NoWritableLedgerDirException e) {
> LOG.error("No writeable ledger directories", e);
> dirsListener.allDisksFull();
> return;
> } catch (IOException e) {
> LOG.error("Exception flushing ledgers", e); <-that IOExc gets
> propagated to this method and here it is caught and not dealt appropriately
>
> return;
> }
> try {
> checkpointSource.checkpointComplete(checkpoint, true);
> } catch (IOException e) {
> LOG.error("Exception marking checkpoint as complete", e);
> dirsListener.allDisksFull();
> }
> }
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (BOOKKEEPER-1027) Cleanup main README and main website page
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16094791#comment-16094791 ] Hudson commented on BOOKKEEPER-1027: FAILURE: Integrated in Jenkins build bookkeeper-master #1823 (See [https://builds.apache.org/job/bookkeeper-master/1823/]) BOOKKEEPER-1027: Cleanup README (eolivelli: [https://github.com/apache/bookkeeper/commit/2d992e39da0fc121901a72849b424c86fb099f03]) * (add) README.md * (delete) README * (edit) pom.xml * (edit) src/assemble/src.xml * (edit) src/assemble/bin.xml > Cleanup main README and main website page > - > > Key: BOOKKEEPER-1027 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1027 > Project: Bookkeeper > Issue Type: Task > Components: Documentation >Affects Versions: 4.5.0 >Reporter: Enrico Olivelli >Assignee: Sijie Guo >Priority: Trivial > Fix For: 4.5.0 > > > README file is really out-of date, it references Java6 and Hedwig > we should update it as it is one of the first descriptions of the project for > new developers which are evaluating BookKeeper -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (BOOKKEEPER-1029) BookieDecommision Workflow
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16078102#comment-16078102 ] Hudson commented on BOOKKEEPER-1029: FAILURE: Integrated in Jenkins build bookkeeper-master #1808 (See [https://builds.apache.org/job/bookkeeper-master/1808/]) BOOKKEEPER-1028 and BOOKKEEPER-1029 (cguttapalem: [https://github.com/apache/bookkeeper/commit/29eb420f1cca07c1a3a7748ed7fbb8a5bc54f2fa]) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/replication/ReplicationException.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/ZkLedgerUnderreplicationManager.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerMetadata.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerChecker.java * (edit) bookkeeper-server/conf/log4j.shell.properties * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/util/BookKeeperConstants.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeperAdmin.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/replication/Auditor.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/replication/AuditorLedgerCheckerTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/BookieShell.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/LedgerUnderreplicationManager.java * (add) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/BookKeeperAdminTest.java > BookieDecommision Workflow > -- > > Key: BOOKKEEPER-1029 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1029 > Project: Bookkeeper > Issue Type: New Feature >Reporter: Charan Reddy Guttapalem >Assignee: Charan Reddy Guttapalem > > - LostBookieRecoveryDelay config param is stored in ZK > - if LostBookieRecoveryDelay is reset to same value then it force > triggers audit immediately > - Added logic to trigger immediately or schedule pending audittask > depending on the changed value in ZK > - good number of testcases validating focetrigger/reschedluing audittask > - added bookieshell command to get/set LostBookieRecoveryDelay from ZK > - added bookieshell command to triggeraudit by resetting > LostBookieRecoveryDelay > - added decommissionbookie bkshell command, which validates the complete > replication of ledgers stored in the bookie -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (BOOKKEEPER-1033) Handle DirsPartitionDuplication
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16078103#comment-16078103 ] Hudson commented on BOOKKEEPER-1033: FAILURE: Integrated in Jenkins build bookkeeper-master #1808 (See [https://builds.apache.org/job/bookkeeper-master/1808/]) BOOKKEEPER-1033: Handle DirsPartitionDuplication (cguttapalem: [https://github.com/apache/bookkeeper/commit/9aff6d52e01b50f25f5dbe1086ff10012ca25c48]) * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/BookieInitializationTest.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/EntryLogTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerDirsManager.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/CompactionTest.java * (edit) bookkeeper-server/conf/bk_server.conf * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/CreateNewLogTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/BookieShell.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/GetBookieInfoProcessorV3.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerDirsMonitor.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/BookieException.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/TestLedgerDirsManager.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/util/DiskChecker.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/ServerConfiguration.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/IndexPersistenceMgrTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/ReadOnlyEntryLogger.java > Handle DirsPartitionDuplication > --- > > Key: BOOKKEEPER-1033 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1033 > Project: Bookkeeper > Issue Type: New Feature >Reporter: Charan Reddy Guttapalem >Assignee: Charan Reddy Guttapalem > > - Provide configuration for allowDirsPartitionDuplication > - while calculating total disk metrics account Partition Duplication -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (BOOKKEEPER-1028) inc/excl opts listunderreplicate
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16078101#comment-16078101 ] Hudson commented on BOOKKEEPER-1028: FAILURE: Integrated in Jenkins build bookkeeper-master #1808 (See [https://builds.apache.org/job/bookkeeper-master/1808/]) BOOKKEEPER-1028 and BOOKKEEPER-1029 (cguttapalem: [https://github.com/apache/bookkeeper/commit/29eb420f1cca07c1a3a7748ed7fbb8a5bc54f2fa]) * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/replication/AuditorLedgerCheckerTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/replication/Auditor.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeperAdmin.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/util/BookKeeperConstants.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/BookieShell.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerChecker.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerMetadata.java * (edit) bookkeeper-server/conf/log4j.shell.properties * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/ZkLedgerUnderreplicationManager.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/LedgerUnderreplicationManager.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/replication/ReplicationException.java * (add) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/BookKeeperAdminTest.java > inc/excl opts listunderreplicate > > > Key: BOOKKEEPER-1028 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1028 > Project: Bookkeeper > Issue Type: New Feature >Reporter: Charan Reddy Guttapalem >Assignee: Charan Reddy Guttapalem >Priority: Minor > > - Introduce including and excluding BookieId options > for listunderreplicatedLedgers -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (BOOKKEEPER-772) Reorder read sequnce
[ https://issues.apache.org/jira/browse/BOOKKEEPER-772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16073676#comment-16073676 ] Hudson commented on BOOKKEEPER-772: --- SUCCESS: Integrated in Jenkins build bookkeeper-master #1805 (See [https://builds.apache.org/job/bookkeeper-master/1805/]) BOOKKEEPER-772: Reorder Read Sequence (zhaijia: [https://github.com/apache/bookkeeper/commit/ee0dddee6849d1968500af666571df668d34393a]) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeper.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/PendingReadOp.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/TestSyncThread.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeperClientStats.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookKeeperClusterTestCase.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieClient.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/ClientConfiguration.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/FileInfo.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerEntryPage.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerCacheImpl.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/AsyncCallback.java * (add) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestReadLastConfirmedAndEntry.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/ReadLastConfirmedAndEntryOp.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerStorage.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandle.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/LongPollReadEntryProcessorV3.java * (add) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestReadLastConfirmedLongPoll.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/ShortReadException.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/GcLedgersTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PerChannelBookieClient.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerDescriptorImpl.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LastAddConfirmedUpdateNotification.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerCache.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/ServerConfiguration.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/InterleavedLedgerStorage.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieRequestProcessor.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/LedgerManagerTestCase.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/IndexPersistenceMgr.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerDescriptor.java > Reorder read sequnce > - > > Key: BOOKKEEPER-772 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-772 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-client >Reporter: Sijie Guo >Assignee: Sijie Guo > Fix For: 4.5.0 > > > We should reorder the read sequence base on location, bookie availability, > for latency consideration. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (BOOKKEEPER-759) bookkeeper: delay ensemble change if it doesn't break ack quorum requirement
[ https://issues.apache.org/jira/browse/BOOKKEEPER-759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16070148#comment-16070148 ] Hudson commented on BOOKKEEPER-759: --- FAILURE: Integrated in Jenkins build bookkeeper-master #1801 (See [https://builds.apache.org/job/bookkeeper-master/1801/]) BOOKKEEPER-759: Delay Ensemble Change & Disable Ensemble Change (sijie: [https://github.com/apache/bookkeeper/commit/5e399df67c2aa1e5f228c62ba8533ca3293ab147]) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandle.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PerChannelBookieClient.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/PendingAddOp.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/PendingWriteLacOp.java * (add) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestDisableEnsembleChange.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/RoundRobinDistributionScheduleTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieClient.java * (add) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestDelayEnsembleChange.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/util/BookKeeperConstants.java * (edit) bookkeeper-server/src/test/resources/log4j.properties * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/DistributionSchedule.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/BookieRecoveryTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeper.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/ReadLastConfirmedOp.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/ReadOnlyLedgerHandle.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/PendingReadLacOp.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BKException.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/RoundRobinDistributionSchedule.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/ClientConfiguration.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookKeeperClusterTestCase.java > bookkeeper: delay ensemble change if it doesn't break ack quorum requirement > > > Key: BOOKKEEPER-759 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-759 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-client >Reporter: Sijie Guo >Assignee: Sijie Guo > Fix For: 4.5.0 > > > flag to allow delay ensemble change. if that is set to change, will not do > ensemble change until it breaks ack quorum requirement -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (BOOKKEEPER-1092) Ledger Recovery - Add Test Case for Parallel Ledger Recovery
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16059384#comment-16059384 ] Hudson commented on BOOKKEEPER-1092: FAILURE: Integrated in Jenkins build bookkeeper-master #1792 (See [https://builds.apache.org/job/bookkeeper-master/1792/]) BOOKKEEPER-1092: Ledger Recovery (Part 5) - Add Test Case for Parallel (sijie: rev 256e7587331eddd51b7c05f27717c9d4dc848500) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeperClientStats.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerRecoveryOp.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/CleanupLedgerManager.java * (add) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/ParallelLedgerRecoveryTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeper.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookkeeperInternalCallbacks.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandle.java > Ledger Recovery - Add Test Case for Parallel Ledger Recovery > > > Key: BOOKKEEPER-1092 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1092 > Project: Bookkeeper > Issue Type: Sub-task > Components: bookkeeper-client >Reporter: Sijie Guo >Assignee: Sijie Guo > Fix For: 4.5.0 > > > Add test case for parallel ledger recovery -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (BOOKKEEPER-1093) Piggyback LAC on ReadResponse
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16059383#comment-16059383 ] Hudson commented on BOOKKEEPER-1093: FAILURE: Integrated in Jenkins build bookkeeper-master #1792 (See [https://builds.apache.org/job/bookkeeper-master/1792/]) BOOKKEEPER-1093: Piggyback LAC on ReadResponse (sijie: rev 90e4e46bccdd1a788e10056f4a3818aa2aead452) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PerChannelBookieClient.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerRecoveryOp.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieClientTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/ClientConfiguration.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/PendingReadOp.java * (add) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestPiggybackLAC.java > Piggyback LAC on ReadResponse > - > > Key: BOOKKEEPER-1093 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1093 > Project: Bookkeeper > Issue Type: Sub-task > Components: bookkeeper-client, bookkeeper-server >Reporter: Sijie Guo >Assignee: Sijie Guo > Fix For: 4.5.0 > > > Currently LAC is only updated when the reader explicitly calls > #readLastAddConfirmed(). In tailing-read use cases, it will not wise to keep > calling #readLastAddConfirmed, especially when the traffic is huge. > The idea here is piggy-back LAC along with the read responses. so the client > will get advanced LAC along with read responses. so it will reduce calling > #readLastAddConfirmed. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (BOOKKEEPER-1089) Ledger Recovery - allow batch reads in ledger recovery
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16059382#comment-16059382 ] Hudson commented on BOOKKEEPER-1089: FAILURE: Integrated in Jenkins build bookkeeper-master #1792 (See [https://builds.apache.org/job/bookkeeper-master/1792/]) BOOKKEEPER-1089: Ledger Recovery (part-4) - allow batch reads in ledger (sijie: rev 90a8f283999ddbfee818629e1be17102e63be22c) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeper.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/LedgerRecoveryTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerRecoveryOp.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/PendingReadOp.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeperClientStats.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/ClientConfiguration.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandle.java > Ledger Recovery - allow batch reads in ledger recovery > -- > > Key: BOOKKEEPER-1089 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1089 > Project: Bookkeeper > Issue Type: Sub-task > Components: bookkeeper-client >Reporter: Sijie Guo >Assignee: Sijie Guo > Fix For: 4.5.0 > > > bookkeeper recovery improvement (part-4): allow batch reading in ledger > recovery > - enable batch read in ledger recovery, so we could parallel reading to > improve recovery time. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (BOOKKEEPER-1096) When ledger is deleted, along with leaf node all the eligible branch nodes also should be deleted in ZooKeeper.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16047897#comment-16047897 ] Hudson commented on BOOKKEEPER-1096: SUCCESS: Integrated in Jenkins build bookkeeper-master #1783 (See [https://builds.apache.org/job/bookkeeper-master/1783/]) BOOKKEEPER-1096: recursive znode delete (sijie: rev 07852d35856dca232450135913090bac27b29abe) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/AbstractZkLedgerManager.java * (add) bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/LedgerMetadataCreationTest.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/LedgerManagerTestCase.java * (add) bookkeeper-server/src/test/java/org/apache/bookkeeper/util/TestZkUtils.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/util/ZkUtils.java > When ledger is deleted, along with leaf node all the eligible branch nodes > also should be deleted in ZooKeeper. > --- > > Key: BOOKKEEPER-1096 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1096 > Project: Bookkeeper > Issue Type: Improvement >Reporter: Charan Reddy Guttapalem >Assignee: Charan Reddy Guttapalem > Fix For: 4.5.0 > > > Currently when we delete a ledger, we delete just the leaf node in the ZK but > we ignore about the branch nodes. This is ok for FlatLedgerManager, but for > HierarchicalLedgerManagers, especially for LongHierarchicalLedgerManager, the > number of internal nodes gets blown up over time and we would get into ZK > capacity limitations. When ZK reaches the capacity limits, it will manifest > in very severe performance and stability issues of cluster. So for > HierarchicalLedgerManagers, when we delete a ledger we should optimistically > recursively delete the parent znodes as well if they don’t have anymore child > znodes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (BOOKKEEPER-1088) Ledger Recovery - Add a ReadEntryListener to callback on individual request
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045524#comment-16045524 ] Hudson commented on BOOKKEEPER-1088: FAILURE: Integrated in Jenkins build bookkeeper-master #1780 (See [https://builds.apache.org/job/bookkeeper-master/1780/]) BOOKKEEPER-1088: CompactionTest tests are broken because of (eolivelli: rev 13c5c88f63f4b7faa63c8c7e42d95f94c6a8d204) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/PendingReadOp.java > Ledger Recovery - Add a ReadEntryListener to callback on individual request > --- > > Key: BOOKKEEPER-1088 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1088 > Project: Bookkeeper > Issue Type: Sub-task > Components: bookkeeper-client >Reporter: Sijie Guo >Assignee: Sijie Guo > Fix For: 4.5.0 > > > bookkeeper recovery improvement (part-3): add a ReadEntryListener to callback > on individual request. > - add read entry listener which allow doing batch read, but callback on > individual entries in sequence. so in recovery op, we could issue batch > reads, then on each individual callback do add entry and stop when received > NoSuchEntry. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1087) Ledger Recovery - Add a parallel reading request in PendingReadOp
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16039380#comment-16039380 ] Hudson commented on BOOKKEEPER-1087: UNSTABLE: Integrated in Jenkins build bookkeeper-master #1776 (See [https://builds.apache.org/job/bookkeeper-master/1776/]) BOOKKEEPER-1087: Ledger Recovery (part 2) - Add a parallel reading (sijie: rev b3b958c2b6a7195b23477d181db3fea1e7d1c248) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandle.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerRecoveryOp.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/PendingReadOp.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/ClientConfiguration.java * (add) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestParallelRead.java > Ledger Recovery - Add a parallel reading request in PendingReadOp > - > > Key: BOOKKEEPER-1087 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1087 > Project: Bookkeeper > Issue Type: Sub-task > Components: bookkeeper-client >Reporter: Sijie Guo >Assignee: Sijie Guo > Fix For: 4.5.0 > > > bookkeeper recovery improvement (part-2): add a parallel reading request > in PendingReadOp > - add a parallel reading request in PendingReadOp > - allow PendingReadOp to configure whether to do parallel reading or not > - add flag in ClientConfiguration to allow configuring whether to do > parallel reading in LedgerRecoveryOp or not. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-944) Multiple issues and improvements to BK Compaction.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16039378#comment-16039378 ] Hudson commented on BOOKKEEPER-944: --- UNSTABLE: Integrated in Jenkins build bookkeeper-master #1776 (See [https://builds.apache.org/job/bookkeeper-master/1776/]) BOOKKEEPER-944: LowWaterMark Storage Threshold (sijie: rev efd8ec26926d2e75f51d6d576543a0dcc116b83f) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerDirsMonitor.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/IndexPersistenceMgrTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerDirsManager.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/ServerConfiguration.java * (edit) bookkeeper-server/conf/bk_server.conf * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/BookieInitializationTest.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/TestLedgerDirsManager.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/util/DiskChecker.java > Multiple issues and improvements to BK Compaction. > -- > > Key: BOOKKEEPER-944 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-944 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-server >Affects Versions: 4.4.0 >Reporter: Venkateswararao Jujjuri (JV) >Assignee: Venkateswararao Jujjuri (JV) > Fix For: 4.5.0 > > > We have identified multiple issues with BK compaction. > This issue is to list all of them in one Jira ticket. > 1. > MajorCompaction and MinorCompaction are very basic. Either they do it or > won’t do it. Proposal is to add Low Water Mark(LWM) and High Water Mark(HWM) > to the disk space. Have different compaction frequency and re-claim %s when > the disk space is < low water mark , > LWM < HWM, > HWM. > 2. > MajorCompaction and Minor Compactions are strictly frequency based. They > should at least time of the day based, and also run during low system load, > and if the system load raises, reduce the compaction depending on the disk > availability > 3. > Current code disables compaction when disk space grows beyond configured > threshold. There is no exit from this point. Have an option to keep reserved > space for compaction, at least 2 entryLog file sizes when > isForceGCAllowWhenNoSpace enabled. > 4. > Current code toggles READONLY status of the bookie as soon as it falls below > the disk storage threshold. Imagine if we keep 95% as the threshold, Bookie > becomes RW as soon as it falls below 95 % and few more writes pushes it above > 95 and it turns back to RONLY. Use a set of defines (another set of LWM/HWM?) > where Bookie turns RO on high end and won't become RW until it hits low end. > 5. > Current code never checks if the compaction is enabled or disabled once the > major/minor compaction is started. If the bookie goes > disk threshold (95%) > and at that compaction is going on, it never checks until it finishes but > there may not be disk available for compaction to take place. So check if > compaction is enabled after processing every EntryLog. > 6. > Current code changes the Bookie Cookie value even when new storage is added. > When the cookie changes Bookie becomes a new one, and BK cluster treats it as > new bookie. If we have mechanism to keep valid cookie even after adding > additional disk space, we may have a chance to bring the bookie back to > healthy mode and have compaction going. > 7. Bug > CheckPoint was never attempted to complete after once sync failure. There is > a TODO in the code for this area. > 8. > When the disk is above threshold, Bookie goes to RO. If we have to restart > the bookie, on the way back, bookie tries to create new entrylog and other > files, which will fail because disk usage is above threshold, hence bookie > refuses to come up. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-748) Move fence requests out of read threads
[ https://issues.apache.org/jira/browse/BOOKKEEPER-748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16039382#comment-16039382 ] Hudson commented on BOOKKEEPER-748: --- UNSTABLE: Integrated in Jenkins build bookkeeper-master #1776 (See [https://builds.apache.org/job/bookkeeper-master/1776/]) BOOKKEEPER-748: Move fence requests out of read threads (sijie: rev 58b92de8b5feee37f8bc9a99d8c98c2a9d9df383) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/ReadLacProcessorV3.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/BookKeeperServerStats.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieProtoEncoding.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieProtocol.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/WriteLacProcessorV3.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookkeeperInternalCallbacks.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieRequestProcessor.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/GetBookieInfoProcessorV3.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PacketProcessorBaseV3.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/RequestUtils.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerDescriptorImpl.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/ReadEntryProcessorV3.java > Move fence requests out of read threads > --- > > Key: BOOKKEEPER-748 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-748 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-server >Reporter: Sijie Guo >Assignee: Sijie Guo > Fix For: 4.5.0 > > > move fence requests out of read threads to address TODO. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1088) Ledger Recovery - Add a ReadEntryListener to callback on individual request
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16039381#comment-16039381 ] Hudson commented on BOOKKEEPER-1088: UNSTABLE: Integrated in Jenkins build bookkeeper-master #1776 (See [https://builds.apache.org/job/bookkeeper-master/1776/]) BOOKKEEPER-1088: Ledger Recovery (part-3) - Add a ReadEntryListener to (sijie: rev 5fe86525a9c823f79b3e97fd82ea4aa1c75c79eb) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookkeeperInternalCallbacks.java * (add) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestReadEntryListener.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/PendingReadOp.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/ListenerBasedPendingReadOp.java > Ledger Recovery - Add a ReadEntryListener to callback on individual request > --- > > Key: BOOKKEEPER-1088 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1088 > Project: Bookkeeper > Issue Type: Sub-task > Components: bookkeeper-client >Reporter: Sijie Guo >Assignee: Sijie Guo > Fix For: 4.5.0 > > > bookkeeper recovery improvement (part-3): add a ReadEntryListener to callback > on individual request. > - add read entry listener which allow doing batch read, but callback on > individual entries in sequence. so in recovery op, we could issue batch > reads, then on each individual callback do add entry and stop when received > NoSuchEntry. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-989) Enable travis CI for bookkeeper git
[ https://issues.apache.org/jira/browse/BOOKKEEPER-989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16034679#comment-16034679 ] Hudson commented on BOOKKEEPER-989: --- SUCCESS: Integrated in Jenkins build bookkeeper-master #1772 (See [https://builds.apache.org/job/bookkeeper-master/1772/]) BOOKKEEPER-989: Enable Travis CI for Apache BookKeeper (sijie: rev 55d1dc45c4761f150c59a34b94d21e39e69f872c) * (add) dev/ticktoc.sh * (add) .travis.yml > Enable travis CI for bookkeeper git > --- > > Key: BOOKKEEPER-989 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-989 > Project: Bookkeeper > Issue Type: Bug > Components: build >Reporter: Sijie Guo >Assignee: Sijie Guo > Fix For: 4.5.0 > > > Let's enable travis build for bookkeeper project. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1090) Use LOG.isDebugEnabled() to avoid unexpected allocations
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16034680#comment-16034680 ] Hudson commented on BOOKKEEPER-1090: SUCCESS: Integrated in Jenkins build bookkeeper-master #1772 (See [https://builds.apache.org/job/bookkeeper-master/1772/]) BOOKKEEPER-1090: Use LOG.isDebugEnabled() to avoid unexpected (eolivelli: rev 5f945f8a02f5171608810eb002443193c3eb058c) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerRecoveryOp.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/ExplicitLacFlushPolicy.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/sasl/SASLClientAuthProvider.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieServer.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/LedgerLayout.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/sasl/SaslClientState.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/ReadEntryProcessor.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/BookieShell.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/replication/ReplicationEnableCb.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/zookeeper/ZooKeeperWatcherBase.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/ReadLastConfirmedOp.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/IndexInMemPageMgr.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/ReadOnlyLedgerHandle.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/replication/ReplicationWorker.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/AuthHandler.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeper.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/EntryLogger.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandleAdv.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/util/LocalBookKeeper.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/FileInfo.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookieInfoReader.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Journal.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/GarbageCollectorThread.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/PendingAddOp.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/LedgerManagerFactory.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/ZkLedgerIdGenerator.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/AbstractZkLedgerManager.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/IndexPersistenceMgr.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/TopologyAwareEnsemblePlacementPolicy.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerMetadata.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/sasl/SASLClientProviderFactory.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/replication/Auditor.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/MSLedgerManagerFactory.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/SyncThread.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeperAdmin.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/sasl/SaslServerState.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/sasl/SASLBookieAuthProviderFactory.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/PendingReadOp.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/ReadEntryProcessorV3.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/ZkLedgerUnderreplicationManager.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/sasl/TGTRefreshThread.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PerChannelBookieClient.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/sasl/SASLBookieAuthProvider.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandle.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/PendingReadLacOp.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerCacheImpl.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/WeightedRandomSelection.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java * (edit)
[jira] [Commented] (BOOKKEEPER-1086) Ledger Recovery - Refactor PendingReadOp
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16034678#comment-16034678 ] Hudson commented on BOOKKEEPER-1086: SUCCESS: Integrated in Jenkins build bookkeeper-master #1772 (See [https://builds.apache.org/job/bookkeeper-master/1772/]) BOOKKEEPER-1086: Ledger Recovery - Refactor PendingReadOp (eolivelli: rev 5c81acaccfb0cb1260acdfd7d0bb95ae84f85654) * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestSpeculativeRead.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/PendingReadOp.java > Ledger Recovery - Refactor PendingReadOp > > > Key: BOOKKEEPER-1086 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1086 > Project: Bookkeeper > Issue Type: Sub-task > Components: bookkeeper-client >Reporter: Sijie Guo >Assignee: Sijie Guo > Fix For: 4.5.0 > > > bookkeeper recovery improvement (part-1): refactor PendingReadOp > this change is the first part of improving bookkeeper recovery. it is > basically a refactor change, which: > - abstract an interface for LedgerEntryRequest in PendingReadOp > - rename current implementation to SequenceReadRequest, which read the > entry in the sequence of quorum. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1056) Removed PacketHeader serialization/deserialization allocation
[
https://issues.apache.org/jira/browse/BOOKKEEPER-1056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16034677#comment-16034677
]
Hudson commented on BOOKKEEPER-1056:
SUCCESS: Integrated in Jenkins build bookkeeper-master #1772 (See
[https://builds.apache.org/job/bookkeeper-master/1772/])
BOOKKEEPER-1056: Removed PacketHeader serialization/deserialization (eolivelli:
rev a9fe7d94f1890f697633fdca8abdeff23903025f)
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieProtocol.java
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieProtoEncoding.java
> Removed PacketHeader serialization/deserialization allocation
> -
>
> Key: BOOKKEEPER-1056
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1056
> Project: Bookkeeper
> Issue Type: Improvement
>Reporter: Matteo Merli
>Assignee: Matteo Merli
>Priority: Minor
> Fix For: 4.5.0
>
>
> When parsing the request packet header, use static methods to avoid creating
> a {{PacketHeader}} instance.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1074) Remove JMX Bean
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16032966#comment-16032966 ] Hudson commented on BOOKKEEPER-1074: SUCCESS: Integrated in Jenkins build bookkeeper-master #1771 (See [https://builds.apache.org/job/bookkeeper-master/1771/]) BOOKKEEPER-1074: Remove JMX Bean (eolivelli: rev dd08ce1a644de00967dd5fe34ce15caa9789d775) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java * (delete) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/BookieBean.java * (delete) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerCacheMXBean.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieServer.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerCacheImpl.java * (delete) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieServerBean.java * (delete) bookkeeper-server/src/main/java/org/apache/bookkeeper/jmx/BKMBeanInfo.java * (delete) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerCacheBean.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/TestSyncThread.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/InterleavedLedgerStorage.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerCache.java * (delete) bookkeeper-server/src/main/java/org/apache/bookkeeper/jmx/BKMBeanRegistry.java * (delete) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieServerMXBean.java * (delete) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/BookieMXBean.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerStorage.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/LedgerManagerTestCase.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/GcLedgersTest.java > Remove JMX Bean > > > Key: BOOKKEEPER-1074 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1074 > Project: Bookkeeper > Issue Type: Bug > Components: bookkeeper-stats >Reporter: Sijie Guo >Assignee: Leigh Stewart > Fix For: 4.5.0 > > > JMX Bean was the old fashion used for exporting metrics. It was introduced > before stats-provider introduced. Now stats-provider is there. Let's remove > JMX Bean. > It is the change ported from Twitter branch. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1084) Make variables finale if necessary
[
https://issues.apache.org/jira/browse/BOOKKEEPER-1084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16032851#comment-16032851
]
Hudson commented on BOOKKEEPER-1084:
UNSTABLE: Integrated in Jenkins build bookkeeper-master #1770 (See
[https://builds.apache.org/job/bookkeeper-master/1770/])
BOOKKEEPER-1084: Make variables finale if necessary (eolivelli: rev
77a01dc7e6431092f82a7efb02789beb165b75e2)
* (edit)
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/LoopbackClient.java
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PerChannelBookieClient.java
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/streaming/LedgerOutputStream.java
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/PendingReadOp.java
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerEntry.java
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/streaming/LedgerInputStream.java
> Make variables finale if necessary
> --
>
> Key: BOOKKEEPER-1084
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1084
> Project: Bookkeeper
> Issue Type: Improvement
>Reporter: Sijie Guo
>Assignee: Sijie Guo
> Fix For: 4.5.0
>
>
> Merge commit
> {code}
> commit daffc8460670cd15994bb8650add3be5a9dc2eef
> Author: Sijie Guo
> Date: Thu Apr 18 17:54:55 2013 -0700
> make logger as a static variable
> not all logger in bookkeeper & hedwig are static. some class like
> PendnigReadOp and LedgerEntry would have lots of objects, it might be bad. so
> this task is to make logger as a static variable if it didn't.
> with benefit, cleaning up the imports when touching that file.
> RB_ID=141138
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1073) Several stats provider related changes.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16032848#comment-16032848 ] Hudson commented on BOOKKEEPER-1073: UNSTABLE: Integrated in Jenkins build bookkeeper-master #1770 (See [https://builds.apache.org/job/bookkeeper-master/1770/]) BOOKKEEPER-1073: Several stats provider related changes. (eolivelli: rev fc51f73cbc9cdaae9876bc08cafa3e7a80826207) * (edit) bookkeeper-stats-providers/twitter-ostrich-provider/src/main/java/org/apache/bookkeeper/stats/twitter/ostrich/OstrichProvider.java * (add) bookkeeper-stats-providers/twitter-finagle-provider/src/main/java/org/apache/bookkeeper/stats/twitter/finagle/CounterImpl.java * (edit) bookkeeper-stats-providers/twitter-ostrich-provider/src/main/java/org/apache/bookkeeper/stats/twitter/ostrich/OstrichStatsLoggerImpl.java * (edit) pom.xml * (add) bookkeeper-stats-providers/twitter-finagle-provider/src/main/java/org/apache/bookkeeper/stats/twitter/finagle/FinagleStatsLoggerImpl.java * (add) bookkeeper-stats/src/main/java/org/apache/bookkeeper/stats/CachingStatsProvider.java * (edit) bookkeeper-stats-providers/codahale-metrics-provider/src/main/java/org/apache/bookkeeper/stats/CodahaleStatsLogger.java * (edit) bookkeeper-stats/src/main/java/org/apache/bookkeeper/stats/StatsLogger.java * (edit) bookkeeper-stats-providers/twitter-science-provider/pom.xml * (edit) bookkeeper-stats-providers/twitter-ostrich-provider/src/main/java/org/apache/bookkeeper/stats/twitter/ostrich/OpStatsLoggerImpl.java * (add) bookkeeper-stats-providers/twitter-finagle-provider/pom.xml * (edit) bookkeeper-stats-providers/twitter-science-provider/src/main/java/org/apache/bookkeeper/stats/twitter/science/TwitterStatsProvider.java * (add) bookkeeper-stats-providers/twitter-finagle-provider/src/main/java/org/apache/bookkeeper/stats/twitter/finagle/FinagleStatsProvider.java * (edit) bookkeeper-stats-providers/pom.xml * (add) bookkeeper-stats-providers/twitter-finagle-provider/src/main/java/org/apache/bookkeeper/stats/twitter/finagle/OpStatsLoggerImpl.java * (add) bookkeeper-stats/src/main/java/org/apache/bookkeeper/stats/CachingStatsLogger.java * (edit) bookkeeper-stats-providers/prometheus-metrics-provider/src/main/java/org/apache/bookkeeper/stats/PrometheusStatsLogger.java * (edit) bookkeeper-stats/src/main/java/org/apache/bookkeeper/stats/NullStatsLogger.java * (edit) bookkeeper-stats-providers/twitter-science-provider/src/main/java/org/apache/bookkeeper/stats/twitter/science/TwitterStatsLoggerImpl.java > Several stats provider related changes. > --- > > Key: BOOKKEEPER-1073 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1073 > Project: Bookkeeper > Issue Type: Bug > Components: bookkeeper-stats >Reporter: Sijie Guo >Assignee: Leigh Stewart > Fix For: 4.5.0 > > > Merge changes from twitter's branch: > - add finagle stats provider > - provide the ability to remove gauge and scopes > - update jetty versions for twitter-sciences stats provider -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1083) Improvements on OrderedSafeExecutor
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16032852#comment-16032852 ] Hudson commented on BOOKKEEPER-1083: UNSTABLE: Integrated in Jenkins build bookkeeper-master #1770 (See [https://builds.apache.org/job/bookkeeper-master/1770/]) BOOKKEEPER-1083: Improvements on OrderedSafeExecutor (eolivelli: rev e33ec10aa400f32c2e0278c15ea80a0f624e5919) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/util/OrderedSafeExecutor.java > Improvements on OrderedSafeExecutor > --- > > Key: BOOKKEEPER-1083 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1083 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-client, bookkeeper-server >Reporter: Sijie Guo >Assignee: Sijie Guo > Fix For: 4.5.0 > > > Merge the the improvements made to OrderedSafeExecutor. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1085) Introduce the AlertStatsLogger
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16032854#comment-16032854 ] Hudson commented on BOOKKEEPER-1085: UNSTABLE: Integrated in Jenkins build bookkeeper-master #1770 (See [https://builds.apache.org/job/bookkeeper-master/1770/]) BOOKKEEPER-1085: Introduce the AlertStatsLogger (eolivelli: rev d1f37dafbb475d4d6aab4769335550428c680269) * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/stats/AlertStatsLogger.java > Introduce the AlertStatsLogger > -- > > Key: BOOKKEEPER-1085 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1085 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-stats >Reporter: Sijie Guo >Assignee: Robin Dhamankar > Fix For: 4.5.0 > > > Introduce the AlertStatsLogger used to increment a metric whenever an event > that should never happen is detected. Allow specifying an optional scope to > better classify the error conditions -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1078) Local BookKeeper enhancements for testability
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16032850#comment-16032850 ] Hudson commented on BOOKKEEPER-1078: UNSTABLE: Integrated in Jenkins build bookkeeper-master #1770 (See [https://builds.apache.org/job/bookkeeper-master/1770/]) BOOKKEEPER-1078: Local BookKeeper enhancements for testability (eolivelli: rev 0eca5ff57f6e19fe5829d5ef8df228411467401f) * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/shims/zk/ZooKeeperServerShim.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/shims/zk/ZooKeeperServerShimFactory.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/shims/zk/ZooKeeperServerShimImpl.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/util/LocalBookKeeper.java > Local BookKeeper enhancements for testability > - > > Key: BOOKKEEPER-1078 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1078 > Project: Bookkeeper > Issue Type: Test > Components: bookkeeper-server >Reporter: Sijie Guo >Assignee: Robin Dhamankar > Fix For: 4.5.0 > > > BookKeeper: Local Bookkeeper enhancements for testability > 1. Allow creating local bookies without always starting a zookeeper > server - This is required as tests may want to create and use their own > instance of a test zookeeper > 2. Allow using non default zookeeper host and more importantly non > default ZK port > 3. Allowing the caller to specify the initial port for the bookies > 4. Optionally shutdown bookies when the bookie thread exits -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1010) Bump up Guava version to 20.0
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16027432#comment-16027432 ] Hudson commented on BOOKKEEPER-1010: FAILURE: Integrated in Jenkins build bookkeeper-master #1762 (See [https://builds.apache.org/job/bookkeeper-master/1762/]) BOOKKEEPER-1010: Moving Guava to 20.0 (eolivelli: rev fd3331a2769a29c379ed63e21ed2dc3c0f85ba25) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Journal.java * (edit) pom.xml * (edit) compat-deps/bookkeeper-server-compat-4.0.0/pom.xml * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/replication/ReplicationWorker.java * (edit) bookkeeper-stats-providers/codahale-metrics-provider/src/main/java/org/apache/bookkeeper/stats/CodahaleMetricsProvider.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/replication/Auditor.java * (edit) compat-deps/bookkeeper-server-compat-4.2.0/pom.xml * (edit) compat-deps/bookkeeper-server-compat-4.1.0/pom.xml * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/metastore/Value.java > Bump up Guava version to 20.0 > - > > Key: BOOKKEEPER-1010 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1010 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-server >Reporter: Arvind Kandhare >Assignee: Enrico Olivelli > Fix For: 4.5.0 > > > Currently bookkeeper is using Guava 13.0. Guava has moved forward and at > version 20.0 there are some API changes. > This becomes an issue if a project using Guava 20.0 tries to run bookie > server inproc. Bumping the Guava version to 20.0 will fix that issue. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1077) BookKeeper: Local Bookie Journal and ledger paths
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16026296#comment-16026296 ] Hudson commented on BOOKKEEPER-1077: FAILURE: Integrated in Jenkins build bookkeeper-master #1761 (See [https://builds.apache.org/job/bookkeeper-master/1761/]) BOOKKEEPER-1077: Allow configuration journal/ledger paths for local (eolivelli: rev 6698912e4d35119a2e85a6ea3877c6771073489c) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/ServerConfiguration.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/util/LocalBookKeeper.java > BookKeeper: Local Bookie Journal and ledger paths > - > > Key: BOOKKEEPER-1077 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1077 > Project: Bookkeeper > Issue Type: Bug > Components: bookkeeper-server >Reporter: Sijie Guo >Assignee: Sijie Guo > Fix For: 4.5.0 > > > Use the journal directory and ledger paths specified in the configuration > file passed in to local bookie. Before this change local bookie was always > creating directories in the temp folder -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1075) BK LedgerMetadata: more memory-efficient parsing of configs
[
https://issues.apache.org/jira/browse/BOOKKEEPER-1075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16024724#comment-16024724
]
Hudson commented on BOOKKEEPER-1075:
FAILURE: Integrated in Jenkins build bookkeeper-master #1760 (See
[https://builds.apache.org/job/bookkeeper-master/1760/])
BOOKKEEPER-1075: BK LedgerMetadata: more memory-efficient parsing of
(eolivelli: rev 64bedc2f92d5bb17783c6ea2a2db7ea29170f264)
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerMetadata.java
> BK LedgerMetadata: more memory-efficient parsing of configs
> ---
>
> Key: BOOKKEEPER-1075
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1075
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client
>Reporter: Sijie Guo
>Assignee: Leigh Stewart
> Fix For: 4.5.0
>
>
> Looking at the most prevalent client-side memory allocations, I noticed that
> we allocate 4KB every time we open a ledger. This is caused by allocating a
> 4KB buffer (in TextFormat.toStringBuilder) to account for the maximum
> possible Protobufs message, which is unnecessary in our case: we know the
> exact size of the metadata ( << 500 B) and don't need to allocate more.
> TextFormat.merge(Readable, Message.Builder) is the current method we use.
> This changes to use TextFormat.merge(CharSequence, Message.Builder), which
> avoids the extra 4K allocation conversion + an extra StringBuilder.
> It is the contribution from Alex Yarmula
> {quote}
> commit 9d9d7dd26235a9beda4421b7bed750fea1789076
> Author: Alex Yarmula
> Date: Wed Sep 23 05:57:30 2015 -0700
> BK LedgerMetadata: more memory-efficient parsing of configs
> Looking at the most prevalent client-side memory allocations, I noticed
> that we allocate 4KB every time we open a ledger. This is caused by
> allocating a 4KB buffer (in TextFormat.toStringBuilder) to account for the
> maximum possible Protobufs message, which is unnecessary in our case: we know
> the exact size of the metadata ( << 500 B) and don't need to allocate more.
> TextFormat.merge(Readable, Message.Builder) is the current method we use.
> This changes to use TextFormat.merge(CharSequence, Message.Builder), which
> avoids the extra 4K allocation conversion + an extra StringBuilder.
> RB_ID=745700
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1068) Expose ByteBuf in LedgerEntry to avoid data copy
[
https://issues.apache.org/jira/browse/BOOKKEEPER-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16024723#comment-16024723
]
Hudson commented on BOOKKEEPER-1068:
FAILURE: Integrated in Jenkins build bookkeeper-master #1760 (See
[https://builds.apache.org/job/bookkeeper-master/1760/])
BOOKKEEPER-1068: Expose ByteBuf in LedgerEntry to avoid data copy (eolivelli:
rev 7b1eec47092d0de6776c5a89575dbfc678165ee7)
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/DigestManager.java
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandleAdv.java
* (edit)
bookkeeper-server/src/test/java/org/apache/bookkeeper/client/ClientUtil.java
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/PendingReadOp.java
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerEntry.java
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandle.java
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerFragmentReplicator.java
> Expose ByteBuf in LedgerEntry to avoid data copy
>
>
> Key: BOOKKEEPER-1068
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1068
> Project: Bookkeeper
> Issue Type: Improvement
>Reporter: Matteo Merli
>Assignee: Matteo Merli
> Fix For: 4.5.0
>
>
> To avoid copying the entries payloads when writing/reading on a ledger and
> having to allocate a lot of byte[] on the JVM heap, we need to accept Netty
> ByteBuf buffer.
> By passing a ByteBuf, an application can use a pooled buffer, pointing to
> direct memory, to the {{LedgerHandle.addEntry()}} and have the same buffer
> forwarded on the connection sockets to the bookies.
> The same thing on the read side, {{LedgerEntry}} exposes an additional
> {{getEntryBuffer()}} method that can be used to get the underlying buffer and
> possibly forward that to some other connection, with zero-copy behavior
> (excluding getting data in-out of the kernel).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-391) Support Kerberos authentication of bookkeeper
[ https://issues.apache.org/jira/browse/BOOKKEEPER-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16024722#comment-16024722 ] Hudson commented on BOOKKEEPER-391: --- FAILURE: Integrated in Jenkins build bookkeeper-master #1760 (See [https://builds.apache.org/job/bookkeeper-master/1760/]) BOOKKEEPER-391: Support Kerberos authentication of bookkeeper (eolivelli: rev 667390d1a6305c31608130929ba0d68a0b5a4763) * (add) bookkeeper-server/src/test/java/org/apache/bookkeeper/sasl/MD5DigestBookKeeperTest.java * (edit) bookkeeper-server/pom.xml * (add) bookkeeper-server/src/test/java/org/apache/bookkeeper/sasl/GSSAPIBookKeeperTest.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/sasl/SaslServerState.java * (add) bookkeeper-server/src/test/resources/jaas_md5.conf * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/sasl/JAASCredentialsContainer.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/sasl/SASLBookieAuthProviderFactory.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/sasl/SaslClientState.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/sasl/SASLClientAuthProvider.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/sasl/SaslConstants.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/sasl/SASLBookieAuthProvider.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/sasl/SASLClientProviderFactory.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/sasl/TGTRefreshThread.java > Support Kerberos authentication of bookkeeper > - > > Key: BOOKKEEPER-391 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-391 > Project: Bookkeeper > Issue Type: New Feature > Components: bookkeeper-client, bookkeeper-server >Reporter: Rakesh R >Assignee: Enrico Olivelli > Fix For: 4.5.0 > > > This JIRA to discuss authentication mechanism of bookie clients and server. > Assume ZK provides fully secured communication channel using Kerberos based > authentication and authorization model. We could also manage and renew users > authenticated to BK via Kerberos. There is currently no configuration or > hooks for the Bookie process to obtain Kerberos credentials. > Today an unauthenticated bookie client can easily establish connection with > the bookkeeper server. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1070) bk-merge-pr.py use apache-rat:check goal instead of rat:rat
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16017434#comment-16017434 ] Hudson commented on BOOKKEEPER-1070: FAILURE: Integrated in Jenkins build bookkeeper-master #1754 (See [https://builds.apache.org/job/bookkeeper-master/1754/]) BOOKKEEPER-1070: bk-merge-pr.py use apache-rat:check goal (eolivelli: rev 7da9ed293db87103efcc28b7e706c0b386131ec1) * (edit) dev/bk-merge-pr.py > bk-merge-pr.py use apache-rat:check goal instead of rat:rat > --- > > Key: BOOKKEEPER-1070 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1070 > Project: Bookkeeper > Issue Type: Task > Components: build >Affects Versions: 4.5.0 >Reporter: Enrico Olivelli >Assignee: Enrico Olivelli >Priority: Critical > Fix For: 4.5.0 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1008) Move to netty4
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012400#comment-16012400 ] Hudson commented on BOOKKEEPER-1008: FAILURE: Integrated in Jenkins build bookkeeper-master #1750 (See [https://builds.apache.org/job/bookkeeper-master/1750/]) BOOKKEEPER-1008: Netty 4.1 (sijie: rev 74f795136c1fff3badb29fc982d0cc2d43096b45) * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/proto/NetworkLessBookieTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/DefaultEnsemblePlacementPolicy.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/CRC32DigestManager.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/ReadLastConfirmedOp.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieRequestHandler.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/BookieRecoveryTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieProtoEncoding.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/PendingReadLacOp.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieClient.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerEntry.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookkeeperInternalCallbacks.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/ReadLacProcessorV3.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestGetBookieInfoTimeout.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/EnsemblePlacementPolicy.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/test/LoopbackClient.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestRackawareEnsemblePlacementPolicyUsingScript.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PacketProcessorBaseV3.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/BookKeeperTestClient.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/net/StabilizeNetworkTopology.java * (add) bookkeeper-server/src/test/java/org/apache/bookkeeper/util/DoubleByteBufTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieNettyServer.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/ReadEntryProcessorV3.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/PendingReadOp.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookKeeperClusterTestCase.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/PendingWriteLacOp.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/AuthHandler.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieRequestProcessor.java * (delete) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/ChannelManager.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PerChannelBookieClient.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeper.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/DigestManager.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/net/BookieSocketAddress.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestRegionAwareEnsemblePlacementPolicy.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandleAdv.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieReadWriteTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/RackawareEnsemblePlacementPolicy.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/WriteLacProcessorV3.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/RackawareEnsemblePlacementPolicyImpl.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/BookieInitializationTest.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieClientTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PerChannelBookieClientPool.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/DefaultPerChannelBookieClientPool.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/BookieJournalTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/WriteEntryProcessor.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerChecker.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/ExplicitLacFlushPolicy.java * (edit) bookkeeper-server/pom.xml * (edit) bookkeeper-benchmark/pom.xml * (edit)
[jira] [Commented] (BOOKKEEPER-1058) Ignore already deleted ledger on replication audit
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012398#comment-16012398 ] Hudson commented on BOOKKEEPER-1058: FAILURE: Integrated in Jenkins build bookkeeper-master #1750 (See [https://builds.apache.org/job/bookkeeper-master/1750/]) BOOKKEEPER-1058: Ignore already deleted ledger on replication audit (sijie: rev 811ece53a1c975c4e768422f3d622ac9de6b3e41) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/replication/BookieLedgerIndexer.java > Ignore already deleted ledger on replication audit > -- > > Key: BOOKKEEPER-1058 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1058 > Project: Bookkeeper > Issue Type: Bug >Reporter: Matteo Merli >Assignee: Matteo Merli > Fix For: 4.5.0 > > > Replication auditor should skip ledgers that were deleted since the auditing > was started. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1061) BookieWatcher should not do ZK blocking operations from ZK async callback thread
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012402#comment-16012402 ] Hudson commented on BOOKKEEPER-1061: FAILURE: Integrated in Jenkins build bookkeeper-master #1750 (See [https://builds.apache.org/job/bookkeeper-master/1750/]) BOOKKEEPER-1061: BookieWatcher should not do ZK blocking operations from (sijie: rev 5d43260e84c6121df72c0ee4c844651bfb726638) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookieWatcher.java > BookieWatcher should not do ZK blocking operations from ZK async callback > thread > > > Key: BOOKKEEPER-1061 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1061 > Project: Bookkeeper > Issue Type: Bug >Reporter: Matteo Merli >Assignee: Matteo Merli >Priority: Minor > Fix For: 4.5.0 > > > In some cases, the BookieWatcher can get the ZK event thread stuck. This > happens when a ZK blocking request is issued from a ZK callback thread. > We should decouple the blocking requests in a separate executor to avoid > deadlocking ZK client. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1069) If client uses V2 proto, set the connection to always decode V2 messages
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012404#comment-16012404 ] Hudson commented on BOOKKEEPER-1069: FAILURE: Integrated in Jenkins build bookkeeper-master #1750 (See [https://builds.apache.org/job/bookkeeper-master/1750/]) BOOKKEEPER-1069: If client uses V2 proto, set the connection to always (eolivelli: rev e44c7388399e5589cf44e38c58bb84c74da544af) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieProtoEncoding.java > If client uses V2 proto, set the connection to always decode V2 messages > > > Key: BOOKKEEPER-1069 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1069 > Project: Bookkeeper > Issue Type: Improvement >Reporter: Matteo Merli >Assignee: Matteo Merli >Priority: Minor > Fix For: 4.5.0 > > > Avoid handling parsing exception for each request and instead adapt to what > the client is sending. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1067) Add Prometheus stats provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012401#comment-16012401 ] Hudson commented on BOOKKEEPER-1067: FAILURE: Integrated in Jenkins build bookkeeper-master #1750 (See [https://builds.apache.org/job/bookkeeper-master/1750/]) BOOKKEEPER-1067: Add Prometheus stats provider (sijie: rev d863513719b696638e3369f4bbfde9abf5fd3d5f) * (edit) pom.xml * (edit) bookkeeper-benchmark/pom.xml * (add) bookkeeper-stats-providers/prometheus-metrics-provider/src/main/java/org/apache/bookkeeper/stats/PrometheusMetricsProvider.java * (edit) bookkeeper-stats/pom.xml * (add) bookkeeper-stats-providers/prometheus-metrics-provider/src/main/java/org/apache/bookkeeper/stats/PrometheusOpStatsLogger.java * (add) bookkeeper-stats-providers/prometheus-metrics-provider/pom.xml * (add) bookkeeper-stats-providers/prometheus-metrics-provider/src/main/java/org/apache/bookkeeper/stats/PrometheusStatsLogger.java * (edit) bookkeeper-stats-providers/pom.xml * (add) bookkeeper-stats-providers/prometheus-metrics-provider/src/main/java/org/apache/bookkeeper/stats/PrometheusCounter.java * (edit) bookkeeper-server/pom.xml > Add Prometheus stats provider > - > > Key: BOOKKEEPER-1067 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1067 > Project: Bookkeeper > Issue Type: New Feature >Reporter: Matteo Merli >Assignee: Matteo Merli >Priority: Minor > Fix For: 4.5.0 > > > Prometheus (https://prometheus.io) is a metrics collection system, similar > but much more flexible than graphite. > It would be good to expose the Bookie and BookKeeper client stats directly so > that a Prometheus instance can collect them (and also check the process > status and add alerts). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1048) Use ByteBuf in LedgerStorageInterface
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012403#comment-16012403 ] Hudson commented on BOOKKEEPER-1048: FAILURE: Integrated in Jenkins build bookkeeper-master #1750 (See [https://builds.apache.org/job/bookkeeper-master/1750/]) BOOKKEEPER-1048: Use ByteBuf in LedgerStorage interface (mmerli: rev 0f81461d2d1dc5cf9db4de9a46599d7d64e3dac6) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/ReadEntryProcessorV3.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/WriteEntryProcessor.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/WriteEntryProcessorV3.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerDescriptorReadOnlyImpl.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieProtoEncoding.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/LedgerRecoveryTest.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/BookieJournalTest.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/LedgerManagerTestCase.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/EntryMemTable.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/InterleavedLedgerStorage.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/ReadEntryProcessor.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/CompactionTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerCacheImpl.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieProtocol.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/GcLedgersTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/SortedLedgerStorage.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/LedgerCacheTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerStorage.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/WriteLacProcessorV3.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/FileInfo.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/test/ConcurrentLedgerTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/IndexPersistenceMgr.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/BookKeeperCloseTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Journal.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/replication/AuditorPeriodicCheckTest.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/proto/TestPerChannelBookieClient.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerDescriptorImpl.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/EntryKeyValue.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/ReadLacProcessorV3.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/LedgerCloseTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/ResponseBuilder.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerDescriptor.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/EntryLogTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerCache.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/TestSyncThread.java > Use ByteBuf in LedgerStorageInterface > - > > Key: BOOKKEEPER-1048 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1048 > Project: Bookkeeper > Issue Type: Improvement >Reporter: Matteo Merli >Assignee: Matteo Merli > Fix For: 4.5.0 > > > To pass ref-counted buffer from Netty directly to the storage and the > Journal, we need to have LedgerStorage to accept ByteBuf instead of ByteBuffer -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1066) Introduce GrowableArrayBlockingQueue
[
https://issues.apache.org/jira/browse/BOOKKEEPER-1066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16009349#comment-16009349
]
Hudson commented on BOOKKEEPER-1066:
FAILURE: Integrated in Jenkins build bookkeeper-master #1746 (See
[https://builds.apache.org/job/bookkeeper-master/1746/])
BOOKKEEPER-1066: Introduce GrowableArrayBlockingQueue (eolivelli: rev
e2894a9e9946b4cc4647c280094bcb2eb1a521ea)
* (add)
bookkeeper-server/src/main/java/org/apache/bookkeeper/util/collections/GrowableArrayBlockingQueue.java
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/util/MathUtils.java
* (add)
bookkeeper-server/src/test/java/org/apache/bookkeeper/util/collections/GrowableArrayBlockingQueueTest.java
> Introduce GrowableArrayBlockingQueue
>
>
> Key: BOOKKEEPER-1066
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1066
> Project: Bookkeeper
> Issue Type: Improvement
>Reporter: Matteo Merli
>Assignee: Matteo Merli
>Priority: Minor
> Fix For: 4.5.0
>
>
> In multiple places, (eg: journal, ordered executor, etc..), we are using
> {{LinkedBlockingQueue}} instances to pass objects between threads.
> The {{LinkedBlockingQueue}} differs from the {{ArrayBlockingQueue}} in that
> it doesn't require to define a max queue size, though, being implemented with
> a linked list, it requires to allocates list nodes each time an item is added.
> We can use a {{GrowableArrayBlockingQueue}} that behaves in the same way as
> the {{LinkedBlockingQueue}}, but it's implemented with an array that can be
> resized when the queue reaches the capacity.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1063) Use executure.execute() instead of submit() to avoid creation of unused FutureTask
[
https://issues.apache.org/jira/browse/BOOKKEEPER-1063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16009351#comment-16009351
]
Hudson commented on BOOKKEEPER-1063:
FAILURE: Integrated in Jenkins build bookkeeper-master #1746 (See
[https://builds.apache.org/job/bookkeeper-master/1746/])
BOOKKEEPER-1063: Use executor.execute() instead of submit() (eolivelli: rev
5130d3596f7fb862e38910bf0a0a1dae3ed892e8)
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/SortedLedgerStorage.java
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Journal.java
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/HierarchicalLedgerManager.java
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/util/OrderedSafeExecutor.java
> Use executure.execute() instead of submit() to avoid creation of unused
> FutureTask
> --
>
> Key: BOOKKEEPER-1063
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1063
> Project: Bookkeeper
> Issue Type: Improvement
>Reporter: Matteo Merli
>Assignee: Matteo Merli
>Priority: Minor
> Fix For: 4.5.0
>
>
> When submitting tasks to an executor, if the {{FutureTask}} object is not
> being used we should use {{execute()}} instead of {{submit()}} in order to
> avoid the task object allocation.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1064) ConcurrentModificationException in AuditorLedgerCheckerTest
[
https://issues.apache.org/jira/browse/BOOKKEEPER-1064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16009350#comment-16009350
]
Hudson commented on BOOKKEEPER-1064:
FAILURE: Integrated in Jenkins build bookkeeper-master #1746 (See
[https://builds.apache.org/job/bookkeeper-master/1746/])
BOOKKEEPER-1064: ConcurrentModificationException in (eolivelli: rev
d7f7b922f3ea0471452115c6271fd18ab63d3505)
* (edit)
bookkeeper-server/src/test/java/org/apache/bookkeeper/replication/AuditorLedgerCheckerTest.java
> ConcurrentModificationException in AuditorLedgerCheckerTest
> ---
>
> Key: BOOKKEEPER-1064
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1064
> Project: Bookkeeper
> Issue Type: Test
>Reporter: Matteo Merli
>Assignee: Matteo Merli
>Priority: Minor
> Fix For: 4.5.0
>
>
> As seen in:
> https://builds.apache.org/job/bookkeeper-master-git-pullrequest/371/
> The test is iterating over a hash map that gets updated by a different
> thread. The map needs to be concurrent.
> {noformat}
> java.util.ConcurrentModificationException
> at
> org.apache.bookkeeper.replication.AuditorLedgerCheckerTest.stopAuditorElectors(AuditorLedgerCheckerTest.java:130)
> at
> org.apache.bookkeeper.replication.AuditorLedgerCheckerTest.tearDown(AuditorLedgerCheckerTest.java:114)
> {noformat}
> All subsequent tests in the same class are failing because of the 1st test
> not cleaning up properly.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1059) Upgrade to SLF4J-1.7.25
[
https://issues.apache.org/jira/browse/BOOKKEEPER-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006396#comment-16006396
]
Hudson commented on BOOKKEEPER-1059:
SUCCESS: Integrated in Jenkins build bookkeeper-master #1744 (See
[https://builds.apache.org/job/bookkeeper-master/1744/])
BOOKKEEPER-1059: Upgrade to SLF4J-1.7.25 (mmerli: rev
fc4800544bcd60c66fa682e6ffcbf2789b7fbb74)
* (edit) bookkeeper-server/pom.xml
> Upgrade to SLF4J-1.7.25
> ---
>
> Key: BOOKKEEPER-1059
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1059
> Project: Bookkeeper
> Issue Type: Task
>Reporter: Matteo Merli
>Assignee: Matteo Merli
>Priority: Minor
> Fix For: 4.5.0
>
>
> BookKeeper is currently using SLF4J 1.6.4. By upgrading to 1.7x, the most
> visible change would be to be able to pass a variable number of arguments
> without needing to wrap them into a:
> {code:java}
> log.info("msg: {} {} {}", new Object[] {x, y, x});
> // versus
> log.info("msg: {} {} {}", x, y, x);
> {code}
> SLF4J 1.7 has been around since 2012.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1054) Add gitignore file
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006398#comment-16006398 ] Hudson commented on BOOKKEEPER-1054: SUCCESS: Integrated in Jenkins build bookkeeper-master #1744 (See [https://builds.apache.org/job/bookkeeper-master/1744/]) BOOKKEEPER-1054: Add gitignore file (mmerli: rev eeaddeda66fc687d20c2309a8cbc109c5dfeed89) * (add) .gitignore > Add gitignore file > -- > > Key: BOOKKEEPER-1054 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1054 > Project: Bookkeeper > Issue Type: Task >Reporter: Matteo Merli >Assignee: Matteo Merli >Priority: Trivial > Fix For: 4.5.0 > > > We should have a .gitignore file to hide all build generated files. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1053) Upgrade RAT maven version to 0.12 and ignore Eclipse project files
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004726#comment-16004726 ] Hudson commented on BOOKKEEPER-1053: SUCCESS: Integrated in Jenkins build bookkeeper-master #1743 (See [https://builds.apache.org/job/bookkeeper-master/1743/]) BOOKKEEPER-1053: Upgrade RAT maven version to 0.12 and ignore Eclipse (mmerli: rev d8a7ac310947049f65687e33d92b847c44c91a39) * (edit) pom.xml * (edit) bookkeeper-server/pom.xml > Upgrade RAT maven version to 0.12 and ignore Eclipse project files > -- > > Key: BOOKKEEPER-1053 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1053 > Project: Bookkeeper > Issue Type: Improvement >Reporter: Matteo Merli >Assignee: Matteo Merli >Priority: Trivial > Fix For: 4.5.0 > > > RAT maven plugin should ignore existing Eclipse project files. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1052) Print autorecovery enabled or not in bookie shell
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004725#comment-16004725 ] Hudson commented on BOOKKEEPER-1052: SUCCESS: Integrated in Jenkins build bookkeeper-master #1743 (See [https://builds.apache.org/job/bookkeeper-master/1743/]) BOOKKEEPER-1052: Print autorecovery enabled or not in bookie shell (mmerli: rev 5531014ab1cf282d35d4d53db975192b853bf7a6) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/BookieShell.java > Print autorecovery enabled or not in bookie shell > - > > Key: BOOKKEEPER-1052 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1052 > Project: Bookkeeper > Issue Type: Improvement >Reporter: Matteo Merli >Assignee: Matteo Merli >Priority: Minor > Fix For: 4.5.0 > > > Print the current status of auto-recovery, whether it's enabled or disabled. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1050) Cache journalFormatVersionToWrite when starting Journal
[
https://issues.apache.org/jira/browse/BOOKKEEPER-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004723#comment-16004723
]
Hudson commented on BOOKKEEPER-1050:
SUCCESS: Integrated in Jenkins build bookkeeper-master #1743 (See
[https://builds.apache.org/job/bookkeeper-master/1743/])
BOOKKEEPER-1050: Cache journalFormatVersionToWrite when starting Journal
(mmerli: rev 61a5446b8923e5ce0b2a46891a385fd480275bac)
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Journal.java
> Cache journalFormatVersionToWrite when starting Journal
> ---
>
> Key: BOOKKEEPER-1050
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1050
> Project: Bookkeeper
> Issue Type: Improvement
>Reporter: Matteo Merli
>Assignee: Matteo Merli
>Priority: Minor
> Fix For: 4.5.0
>
>
> Reading the journal version format from {{ServiceConfiguration}} each time is
> inefficient.
> {{ServiceConfiguration}} is based on Java properties which is based on a
> String to object hashtable. Each read implies acquiring a mutex and
> converting from object to int.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1051) Fast shutdown for GarbageCollectorThread
[
https://issues.apache.org/jira/browse/BOOKKEEPER-1051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004724#comment-16004724
]
Hudson commented on BOOKKEEPER-1051:
SUCCESS: Integrated in Jenkins build bookkeeper-master #1743 (See
[https://builds.apache.org/job/bookkeeper-master/1743/])
BOOKKEEPER-1051: Fast shutdown for GarbageCollectorThread (mmerli: rev
1a2e017b65812e6e50975213a365aa37eba9fa96)
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/GarbageCollectorThread.java
> Fast shutdown for GarbageCollectorThread
>
>
> Key: BOOKKEEPER-1051
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1051
> Project: Bookkeeper
> Issue Type: Improvement
>Reporter: Matteo Merli
>Assignee: Matteo Merli
>Priority: Minor
> Fix For: 4.5.0
>
>
> Several unit tests are taking very long time to complete (eg:
> {{BookieLedgerIndexTest}} taking ~10 minutes).
> The reason is that these tests are playing with the ZK quorum shutting it
> down and after the test succeeds, the shutdown phase is taking long time,
> since we try to do graceful shutdown with 1min wait time.
> I think is better to interrupt immediately the garbage collector thread when
> shutting down the bookie.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1046) Avoid long to Long conversion in OrderedSafeExecutor task submit
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998339#comment-15998339 ] Hudson commented on BOOKKEEPER-1046: FAILURE: Integrated in Jenkins build bookkeeper-master #1738 (See [https://builds.apache.org/job/bookkeeper-master/1738/]) BOOKKEEPER-1046: Avoid long to Long conversion in OrderedSafeExecutor (eolivelli: rev 25c113f62a40bff6ff91cf82f699fd1d35b102fb) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/util/OrderedSafeExecutor.java > Avoid long to Long conversion in OrderedSafeExecutor task submit > > > Key: BOOKKEEPER-1046 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1046 > Project: Bookkeeper > Issue Type: Improvement >Reporter: Matteo Merli >Assignee: Matteo Merli >Priority: Trivial > Fix For: 4.5.0 > > > When submitting tasks to an OrderedSafeExecutor, most of the time a ledger id > is being passed. Given that the method accepts and Object, the primitive long > is boxed into a Long allocated on the heap. > Added specific method overload to directly accept longs as the key in the > OrderedSafeExecutor. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1047) Add missing error code in ZK setData return path
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998341#comment-15998341 ] Hudson commented on BOOKKEEPER-1047: FAILURE: Integrated in Jenkins build bookkeeper-master #1738 (See [https://builds.apache.org/job/bookkeeper-master/1738/]) BOOKKEEPER-1047: Add missing error code in ZK setData return path (eolivelli: rev d5af77c05256d5942c19aff2654c5d8f5ac1eb79) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/AbstractZkLedgerManager.java > Add missing error code in ZK setData return path > > > Key: BOOKKEEPER-1047 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1047 > Project: Bookkeeper > Issue Type: Bug >Reporter: Matteo Merli >Assignee: Matteo Merli >Priority: Trivial > Fix For: 4.5.0 > > > The log warning is not printing the error code returned by ZooKeeper -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1045) Execute tests in different JVM processes
[
https://issues.apache.org/jira/browse/BOOKKEEPER-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996752#comment-15996752
]
Hudson commented on BOOKKEEPER-1045:
SUCCESS: Integrated in Jenkins build bookkeeper-master #1737 (See
[https://builds.apache.org/job/bookkeeper-master/1737/])
BOOKKEEPER-1045: Execute tests in different JVM processes (eolivelli: rev
5d282dceae140577b97a12db5b2b531c7d84e985)
* (edit) pom.xml
> Execute tests in different JVM processes
>
>
> Key: BOOKKEEPER-1045
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1045
> Project: Bookkeeper
> Issue Type: Test
>Reporter: Matteo Merli
>Assignee: Matteo Merli
>Priority: Minor
> Fix For: 4.5.0
>
>
> The current Maven Surefire configuration is using:
> {code:xml}
> always
> {code}
> This is a deprecated config and apparently it's not creating new processes
> for each test as intended.
> Currently the tests are leaking a big number of files and threads due to
> several reasons:
> * Tests that instantiate bookies and call shutdown() without calling start()
> before are creating and initializing the ledger storage but not closing it,
> leaking threads and several fds
> * ZooKeeperClient sometimes doesn't shutdown the zk handle if the test
> completes too quickly, leaking sockets.
> * Several tests are passing bad config, so the bookie/client start gets
> exception (on purpose) and then doesn't clean up some partial objects.
> * ...
> That make running the test suite to be dependent on ulimit of the machine.
> Until we can fix (almost) all the test to do proper cleanup, we should make
> maven to run tests in separated processes.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1019) Support for reading entries after LAC (causal consistency driven by out-of-band communications)
[
https://issues.apache.org/jira/browse/BOOKKEEPER-1019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994117#comment-15994117
]
Hudson commented on BOOKKEEPER-1019:
FAILURE: Integrated in Jenkins build bookkeeper-master #1734 (See
[https://builds.apache.org/job/bookkeeper-master/1734/])
BOOKKEEPER-1019: Support for reading entries after LAC (eolivelli: rev
9c79e078b8cfefc24251aefcb727760fb99229ed)
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandle.java
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/ClientConfiguration.java
* (edit)
bookkeeper-server/src/test/java/org/apache/bookkeeper/client/BookKeeperTest.java
> Support for reading entries after LAC (causal consistency driven by
> out-of-band communications)
> ---
>
> Key: BOOKKEEPER-1019
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1019
> Project: Bookkeeper
> Issue Type: New Feature
> Components: bookkeeper-client
>Affects Versions: 4.4.0
>Reporter: Enrico Olivelli
>Assignee: Enrico Olivelli
> Fix For: 4.5.0
>
>
> Currently we check in asyncReadEntries that the range of entries is within
> the range 0LastAddConfirmed.
> This is because the LAC guarantees that the client can read only entries that
> have been acked from the writer.
> The LAC protocol is very useful when there is not direct communication
> between "writers" and "readers".
> I have an use case in which the "writer" blocks until the write is acked
> (like addEntry) and then it takes the returned id (ledgerId + entryid) and
> passes it to a "reader" which in turn tries to read the entry.
> This communication is done out-of-band in respect to BookKeeper and we can
> assume that the entries has been stored in a durable way (the write as been
> acked by a quorum of bookies).
> As the 'reader' as received a confirmation the the writer as successifully
> written the entry it can read it without waiting for the piggyback of the LAC
> of the standard bookkeeper protocol.
> This is the normal way of working with transactional databases or with
> filesystems.
> This is kind of "causal consistency".
> The idea is to add a configuration option to relax the check in
> asyncreadEntries
> this is 4.4 version:
> {code}
> if (lastEntry > lastAddConfirmed) {
> LOG.error("ReadException on ledgerId:{} firstEntry:{}
> lastEntry:{}",
> new Object[] { ledgerId, firstEntry, lastEntry });
> cb.readComplete(BKException.Code.ReadException, this, null, ctx);
> return;
> }
> {code}
> this is my proposal:
> {code}
> if (lastEntry > lastAddConfirmed &&
> !allowReadingAfterLastAddConfirmed) {
> LOG.error("ReadException on ledgerId:{} firstEntry:{}
> lastEntry:{}",
> new Object[] { ledgerId, firstEntry, lastEntry });
> cb.readComplete(BKException.Code.ReadException, this, null, ctx);
> return;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-552) 64 Bits Ledger ID Generation
[ https://issues.apache.org/jira/browse/BOOKKEEPER-552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994118#comment-15994118 ] Hudson commented on BOOKKEEPER-552: --- FAILURE: Integrated in Jenkins build bookkeeper-master #1734 (See [https://builds.apache.org/job/bookkeeper-master/1734/]) BOOKKEEPER-552: 64 Bits Ledger ID Generation (mmerli: rev 057af8dbce6c08794eb8b46ca52ca13f222d9bbb) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/AbstractConfiguration.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BKException.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/AbstractHierarchicalLedgerManager.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/util/StringUtils.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/BookieRecoveryTest.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/test/MultiLedgerManagerTestCase.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/LedgerManagerFactory.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/ZkLedgerIdGenerator.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/HierarchicalLedgerManager.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/LegacyHierarchicalLedgerManager.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/HierarchicalLedgerManagerFactory.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/test/MultiLedgerManagerMultiDigestTestCase.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/test/TestBackwardCompat.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/LongZkLedgerIdGenerator.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/FlatLedgerManager.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/LongHierarchicalLedgerManager.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/LegacyHierarchicalLedgerManagerFactory.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/AbstractZkLedgerManager.java * (add) bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/TestLongZkLedgerIdGenerator.java > 64 Bits Ledger ID Generation > > > Key: BOOKKEEPER-552 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-552 > Project: Bookkeeper > Issue Type: Sub-task > Components: bookkeeper-client, bookkeeper-server, hedwig-server >Reporter: Jiannan Wang >Assignee: Kyle Nusbaum > Fix For: 4.5.0 > > Attachments: BOOKKEEPER-552.patch, BOOKKEEPER-552.patch, > BOOKKEEPER-552.patch > > > This task aims to find and implement 64 bits global unique ledger id > generation mechanisms. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1043) Upgrade Apache Parent Pom Reference to latest version
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15987190#comment-15987190 ] Hudson commented on BOOKKEEPER-1043: FAILURE: Integrated in Jenkins build bookkeeper-master #1727 (See [https://builds.apache.org/job/bookkeeper-master/1727/]) BOOKKEEPER-1043: Upgrade Apache Parent Pom Reference to latest version (eolivelli: rev c94bbbacd7dcf369c51480a21ad8e55f1af4c9a6) * (edit) pom.xml > Upgrade Apache Parent Pom Reference to latest version > - > > Key: BOOKKEEPER-1043 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1043 > Project: Bookkeeper > Issue Type: Task > Components: build >Affects Versions: 4.5.0 >Reporter: Enrico Olivelli >Assignee: Enrico Olivelli > Fix For: 4.5.0 > > > Latest version is 18 > http://repo.maven.apache.org/maven2/org/apache/apache/ -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1038) Fix findbugs warnings and upgrade to 3.0.4
[
https://issues.apache.org/jira/browse/BOOKKEEPER-1038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15978745#comment-15978745
]
Hudson commented on BOOKKEEPER-1038:
FAILURE: Integrated in Jenkins build bookkeeper-master #1726 (See
[https://builds.apache.org/job/bookkeeper-master/1726/])
BOOKKEEPER-1038: Fix findbugs warnings and upgrade to 3.0.4 (sijie: rev
83d5d9e2a51c8c135bfc97d3831beccebe11a69a)
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookieInfoReader.java
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PerChannelBookieClient.java
* (edit) pom.xml
> Fix findbugs warnings and upgrade to 3.0.4
> --
>
> Key: BOOKKEEPER-1038
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1038
> Project: Bookkeeper
> Issue Type: Task
> Components: build
>Affects Versions: 4.5.0
>Reporter: Enrico Olivelli
>Assignee: Enrico Olivelli
> Fix For: 4.5.0
>
>
> This findbugs errors are on master, they have to be fixed
> {code}
> INFO] --- findbugs-maven-plugin:3.0.3:check (default-cli) @ bookkeeper-server
> ---
> [INFO] BugInstance size is 2
> [INFO] Error size is 0
> [INFO] Total bugs: 2
> [INFO] Dead store to requested in
> org.apache.bookkeeper.client.BookieInfoReader.getReadWriteBookieInfo(Collection)
> [org.apache.bookkeeper.client.BookieInfoReader] At
> BookieInfoReader.java:[line 166] DLS_DEAD_LOCAL_STORE
> [INFO] org.apache.bookkeeper.proto.PerChannelBookieClient$V2CompletionKey
> overrides equals in PerChannelBookieClient$CompletionKey and may not be
> symmetric
> [org.apache.bookkeeper.proto.PerChannelBookieClient$V2CompletionKey] At
> PerChannelBookieClient.java:[lines 1720-1724]
> EQ_OVERRIDING_EQUALS_NOT_SYMMETRIC
> [INFO]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1039) bk-merge-pr.py ask to run findbugs and rat before merge
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15974677#comment-15974677 ] Hudson commented on BOOKKEEPER-1039: FAILURE: Integrated in Jenkins build bookkeeper-master #1724 (See [https://builds.apache.org/job/bookkeeper-master/1724/]) BOOKKEEPER-1039: bk-merge-pr.py ask to run findbugs and rat before merge (sijie: rev de59bd2ad038428cfa61cfdefe1266f3356e1937) * (edit) dev/bk-merge-pr.py > bk-merge-pr.py ask to run findbugs and rat before merge > --- > > Key: BOOKKEEPER-1039 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1039 > Project: Bookkeeper > Issue Type: Improvement > Components: build >Affects Versions: 4.5.0 >Reporter: Enrico Olivelli >Assignee: Enrico Olivelli >Priority: Critical > Fix For: 4.5.0 > > > The bk-merge-pr.py script currently asks for running "mvn clean install". > The full test suite is very slow. > Many times we are merging patches which break findbugs and rat check > I would like to add a step like > "Do you want to validate findbugs and rat after the merge? (y/n)" -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1018) Allow client to select older V2 protocol (no protobuf)
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15969947#comment-15969947 ] Hudson commented on BOOKKEEPER-1018: FAILURE: Integrated in Jenkins build bookkeeper-master #1719 (See [https://builds.apache.org/job/bookkeeper-master/1719/]) BOOKKEEPER-1018: Allow client to select older V2 protocol (no protobuf) (family: rev f74d07d6b96ba3e6f1270d053fa36676cf680304) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/AuthHandler.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/ClientConfiguration.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PerChannelBookieClient.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieProtocol.java > Allow client to select older V2 protocol (no protobuf) > -- > > Key: BOOKKEEPER-1018 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1018 > Project: Bookkeeper > Issue Type: Bug > Components: bookkeeper-server >Reporter: Govind Menon >Assignee: Govind Menon > Fix For: 4.5.0 > > > Oriignally done by Matteo Merli -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-997) Wire protocol change for supporting long poll
[ https://issues.apache.org/jira/browse/BOOKKEEPER-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965866#comment-15965866 ] Hudson commented on BOOKKEEPER-997: --- FAILURE: Integrated in Jenkins build bookkeeper-master #1716 (See [https://builds.apache.org/job/bookkeeper-master/1716/]) BOOKKEEPER-997: Wire protocol change for supporting long poll (sijie: rev 24fae0322327d8fd75c30524546924f27e278448) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookkeeperProtocol.java * (edit) bookkeeper-server/src/main/proto/BookkeeperProtocol.proto > Wire protocol change for supporting long poll > - > > Key: BOOKKEEPER-997 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-997 > Project: Bookkeeper > Issue Type: Sub-task > Components: bookkeeper-client, bookkeeper-server >Reporter: Sijie Guo >Assignee: Robin Dhamankar > Fix For: 4.5.0 > > > In order to support piggybacking lac and long poll, there will be changes to > new fields added to the wire protocol. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1022) Make BookKeeperAdmin implement AutoCloseable
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965865#comment-15965865 ] Hudson commented on BOOKKEEPER-1022: FAILURE: Integrated in Jenkins build bookkeeper-master #1716 (See [https://builds.apache.org/job/bookkeeper-master/1716/]) BOOKKEEPER-1022: Make BookKeeperAdmin implement AutoCloseable (sijie: rev 12bb13cdb1feca0fa6739f8e6287c326fb5fdb94) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeperAdmin.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/BookKeeperCloseTest.java > Make BookKeeperAdmin implement AutoCloseable > > > Key: BOOKKEEPER-1022 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1022 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-client >Affects Versions: 4.4.0 >Reporter: Enrico Olivelli >Assignee: Enrico Olivelli >Priority: Minor > Fix For: 4.5.0 > > > as BookKeeper and LedgerHandler are AutoCloseable it would be good to make > BookKeeperAdmin AutoCloseable too -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1020) Fix Explicit LAC tests on master
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955141#comment-15955141 ] Hudson commented on BOOKKEEPER-1020: FAILURE: Integrated in Jenkins build bookkeeper-master #1708 (See [https://builds.apache.org/job/bookkeeper-master/1708/]) BOOKKEEPER-1020: Fix Explicit LAC tests on master (sijie: rev 9836c87dc2c082c46692d60c60cf9e23c7b0b06c) * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/BookKeeperTest.java > Fix Explicit LAC tests on master > > > Key: BOOKKEEPER-1020 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1020 > Project: Bookkeeper > Issue Type: Bug >Affects Versions: 4.5.0 >Reporter: Enrico Olivelli >Assignee: Enrico Olivelli >Priority: Blocker > Fix For: 4.5.0 > > > Tests in error: > BookKeeperTest.testReadHandleWithExplicitLAC:404 » TestTimedOut test timed > out... > BookKeeperTest.testReadHandleWithExplicitLAC:404 » TestTimedOut test timed > out... -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1018) Allow client to select older V2 protocol (no protobuf)
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955142#comment-15955142 ] Hudson commented on BOOKKEEPER-1018: FAILURE: Integrated in Jenkins build bookkeeper-master #1708 (See [https://builds.apache.org/job/bookkeeper-master/1708/]) BOOKKEEPER-1018: Allow client to select older V2 protocol (no protobuf) (sijie: rev 9001e300ce0d5d2655d437e3eaa52f91487caed6) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PerChannelBookieClient.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/AuthHandler.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieProtocol.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/ClientConfiguration.java BOOKKEEPER-1018: Revert ": Allow client to select older V2 protocol (no… (sijie: rev f30f60889e0810a47797daf4107c0d9bc2ee998c) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/AuthHandler.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PerChannelBookieClient.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieProtocol.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/ClientConfiguration.java > Allow client to select older V2 protocol (no protobuf) > -- > > Key: BOOKKEEPER-1018 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1018 > Project: Bookkeeper > Issue Type: Bug > Components: bookkeeper-server >Reporter: Govind Menon >Assignee: Govind Menon > Fix For: 4.5.0 > > > Oriignally done by Matteo Merli -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-390) Provide support for ZooKeeper authentication
[ https://issues.apache.org/jira/browse/BOOKKEEPER-390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15947210#comment-15947210 ] Hudson commented on BOOKKEEPER-390: --- FAILURE: Integrated in Jenkins build bookkeeper-master #1702 (See [https://builds.apache.org/job/bookkeeper-master/1702/]) BOOKKEEPER-390: Provide support for ZooKeeper authentication (sijie: rev b30b527ba995978694ebf735c02b508374001021) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/util/LocalBookKeeper.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/AbstractConfiguration.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/TestZkLedgerIdGenerator.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/FlatLedgerManagerFactory.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookieWatcher.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/HierarchicalLedgerManagerFactory.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/ZkLedgerIdGenerator.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeperAdmin.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/LedgerManagerFactory.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/LedgerLayoutTest.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/TestGcOverreplicatedLedger.java * (add) bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/EnableZkSecurityBasicTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/MSLedgerManagerFactory.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/TestLedgerManager.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/LedgerLayout.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/replication/AuditorElector.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/ScanAndCompareGarbageCollector.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/util/ZkUtils.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Cookie.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/AbstractZkLedgerManager.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/ZkLedgerUnderreplicationManager.java > Provide support for ZooKeeper authentication > > > Key: BOOKKEEPER-390 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-390 > Project: Bookkeeper > Issue Type: New Feature > Components: bookkeeper-client, bookkeeper-server >Affects Versions: 4.0.0 >Reporter: Rakesh R >Assignee: Enrico Olivelli > Fix For: 4.5.0 > > Attachments: BOOKKEEPER-390-Acl-draftversion.patch, > BOOKKEEPER-390-Authentication-interfaces-draftversion.patch > > > This JIRA adds support for protecting the state of Bookkeeper znodes on a > multi-tenant ZooKeeper cluster. > Use case: When user tries to run a ZK cluster in multitenant mode, where > more than one client service would like to share a single ZK service instance > (cluster). In this case the client services typically want to protect their > data (ZK znodes) from access by other services (tenants) on the cluster. Say > you are running BK, HBase or ZKFC instances, etc... having > authentication/authorization on the znodes is important for both security and > helping to ensure that services don't interact negatively (touch each other's > data). > Presently Bookkeeper does not have support for authentication or > authorization while accessing to ZK. This should be added to the BK > clients/server that are accessing the ZK cluster. In general it means calling > addAuthInfo once after a session is established -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1004) Allow bookie garbage collection to be triggered manually from tests
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15947209#comment-15947209 ] Hudson commented on BOOKKEEPER-1004: FAILURE: Integrated in Jenkins build bookkeeper-master #1702 (See [https://builds.apache.org/job/bookkeeper-master/1702/]) BOOKKEEPER-1004: Allow bookie garbage collection to be triggered manu… (sijie: rev 825e0e7b4a27730d85afbe823a405ba333de3a65) * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/BookieAccessor.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/CompactionTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/GarbageCollectorThread.java > Allow bookie garbage collection to be triggered manually from tests > --- > > Key: BOOKKEEPER-1004 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1004 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-server >Affects Versions: 4.5.0, 4.6.0 >Reporter: Govind Menon >Priority: Minor > Labels: test > Fix For: 4.5.0 > > Original Estimate: 24h > Remaining Estimate: 24h > > The current gc tests rely on waiting on a timeout for gc to run. It's > never certain whether it has run or not or if it's still running. > This patch allows tests to trigger a gc run and gives the client > a future to know when it has completed. The gc algorithm is unchangedI but > now it runs in a scheduled executor rather than as a > Thread. > This work was originally done by Ivan Kelly and I am just pushing it back to > open source -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-852) Release LedgerDescriptor and master-key objects when not used anymore
[ https://issues.apache.org/jira/browse/BOOKKEEPER-852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15947208#comment-15947208 ] Hudson commented on BOOKKEEPER-852: --- FAILURE: Integrated in Jenkins build bookkeeper-master #1702 (See [https://builds.apache.org/job/bookkeeper-master/1702/]) BOOKKEEPER-852: Release LedgerDescriptor and master-key objects when not (sijie: rev bf8ef14bc1bfdde6c6c8edcd7f380b4beca023f6) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/InterleavedLedgerStorage.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/GcLedgersTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/HandleFactoryImpl.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/LedgerManagerTestCase.java * (add) bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/LedgerStorageTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/TestSyncThread.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerStorage.java > Release LedgerDescriptor and master-key objects when not used anymore > - > > Key: BOOKKEEPER-852 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-852 > Project: Bookkeeper > Issue Type: Bug >Affects Versions: 4.3.0 >Reporter: Matteo Merli >Assignee: Matteo Merli > Fix For: 4.5.0 > > Attachments: > 0001-BOOKKEEPER-852-Release-LedgerDescriptor-and-master-k.patch > > > Maps with ledger descriptors and master-keys are not cleaned after a ledger > gets deleted. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1001) Make LocalBookiesRegistry.isLocalBookie() public
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940315#comment-15940315 ] Hudson commented on BOOKKEEPER-1001: FAILURE: Integrated in Jenkins build bookkeeper-master #1696 (See [https://builds.apache.org/job/bookkeeper-master/1696/]) BOOKKEEPER-1001: Make LocalBookiesRegistry.isLocalBookie() public (sijie: rev 986110c4d03a38db81c166216cf4db81a9cb5cb7) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/LocalBookiesRegistry.java * (add) bookkeeper-server/src/test/java/org/apache/bookkeeper/test/LocalBookiesRegistryTest.java > Make LocalBookiesRegistry.isLocalBookie() public > > > Key: BOOKKEEPER-1001 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1001 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-server >Affects Versions: 4.4.0 >Reporter: Francesco Caliumi >Priority: Minor > Labels: easyfix, newbie > Fix For: 4.5.0 > > Original Estimate: 0.25h > Remaining Estimate: 0.25h > > This function is useful for applications using an embedded local bookie, for > example in order to create a custom EnsamblePlacementPolicy which prefers the > local bookie if present. > This is an easy fix, I will file a PR. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1013) Fix findbugs errors on latest master
[
https://issues.apache.org/jira/browse/BOOKKEEPER-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940314#comment-15940314
]
Hudson commented on BOOKKEEPER-1013:
FAILURE: Integrated in Jenkins build bookkeeper-master #1696 (See
[https://builds.apache.org/job/bookkeeper-master/1696/])
BOOKKEEPER-1013: Fix findbugs errors on latest master (sijie: rev
3d3340be7d754af1738a117ee07e3bed04ec5547)
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/ServerConfiguration.java
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/ReadOnlyLedgerHandle.java
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/CheckpointSourceList.java
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/FileInfo.java
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieServer.java
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandle.java
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/util/OrderedSafeExecutor.java
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PerChannelBookieClient.java
> Fix findbugs errors on latest master
>
>
> Key: BOOKKEEPER-1013
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1013
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
>Reporter: Sijie Guo
>Assignee: Sijie Guo
> Fix For: 4.5.0
>
>
> The multiple journal dirs change and explict-lac change introduced a few
> findbug errors.
> {code}
> [INFO] BugInstance size is 8
> [INFO] Error size is 0
> [INFO] Total bugs: 8
> [INFO] org.apache.bookkeeper.bookie.CheckpointSourceList$CheckpointList
> defines compareTo(Object) and uses Object.equals()
> [org.apache.bookkeeper.bookie.CheckpointSourceList$CheckpointList] At
> CheckpointSourceList.java:[line 53] EQ_COMPARETO_USE_OBJECT_EQUALS
> [INFO] Dead store to ledgerId in
> org.apache.bookkeeper.bookie.FileInfo.setExplicitLac(ByteBuffer)
> [org.apache.bookkeeper.bookie.FileInfo] At FileInfo.java:[line 140]
> DLS_DEAD_LOCAL_STORE
> [INFO] Inconsistent synchronization of
> org.apache.bookkeeper.bookie.FileInfo.explicitLac; locked 85% of time
> [org.apache.bookkeeper.bookie.FileInfo,
> org.apache.bookkeeper.bookie.FileInfo, org.apache.bookkeeper.bookie.FileInfo,
> org.apache.bookkeeper.bookie.FileInfo, org.apache.bookkeeper.bookie.FileInfo,
> org.apache.bookkeeper.bookie.FileInfo, org.apache.bookkeeper.bookie.FileInfo,
> org.apache.bookkeeper.bookie.FileInfo, org.apache.bookkeeper.bookie.FileInfo,
> org.apache.bookkeeper.bookie.FileInfo, org.apache.bookkeeper.bookie.FileInfo,
> org.apache.bookkeeper.bookie.FileInfo, org.apache.bookkeeper.bookie.FileInfo,
> org.apache.bookkeeper.bookie.FileInfo] Unsynchronized access at
> FileInfo.java:[line 118]Unsynchronized access at FileInfo.java:[line
> 145]Synchronized access at FileInfo.java:[line 121]Synchronized access at
> FileInfo.java:[line 122]Synchronized access at FileInfo.java:[line
> 123]Synchronized access at FileInfo.java:[line 124]Synchronized access at
> FileInfo.java:[line 125]Synchronized access at FileInfo.java:[line
> 134]Synchronized access at FileInfo.java:[line 137]Synchronized access at
> FileInfo.java:[line 138]Synchronized access at FileInfo.java:[line
> 140]Synchronized access at FileInfo.java:[line 135]Synchronized access at
> FileInfo.java:[line 141]Synchronized access at FileInfo.java:[line 143]
> IS2_INCONSISTENT_SYNC
> [INFO] Inconsistent synchronization of
> org.apache.bookkeeper.client.LedgerHandle.length; locked 90% of time
> [org.apache.bookkeeper.client.ReadOnlyLedgerHandle$MetadataUpdater,
> org.apache.bookkeeper.client.LedgerHandle,
> org.apache.bookkeeper.client.LedgerHandle,
> org.apache.bookkeeper.client.LedgerHandle,
> org.apache.bookkeeper.client.LedgerHandle,
> org.apache.bookkeeper.client.LedgerHandle,
> org.apache.bookkeeper.client.LedgerHandle,
> org.apache.bookkeeper.client.LedgerHandle,
> org.apache.bookkeeper.client.LedgerHandle,
> org.apache.bookkeeper.client.LedgerRecoveryOp,
> org.apache.bookkeeper.client.LedgerHandle$2] Unsynchronized access at
> ReadOnlyLedgerHandle.java:[line 59]Synchronized access at
> LedgerHandle.java:[line 259]Synchronized access at LedgerHandle.java:[line
> 777]Synchronized access at LedgerHandle.java:[line 777]Synchronized access at
> LedgerHandle.java:[line 1360]Synchronized access at LedgerHandle.java:[line
> 249]Synchronized access at LedgerHandle.java:[line 249]Synchronized access at
> LedgerHandle.java:[line 250]Synchronized access at LedgerHandle.java:[line
> 976]Synchronized access at LedgerRecoveryOp.java:[line 151]Synchronized
> access at LedgerHandle.java:[line 379] IS2_INCONSISTENT_SYNC
> [INFO] Redundant nullcheck of journalDirNames, which is known
[jira] [Commented] (BOOKKEEPER-970) Bump zookeeper version to 3.5
[ https://issues.apache.org/jira/browse/BOOKKEEPER-970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940316#comment-15940316 ] Hudson commented on BOOKKEEPER-970: --- FAILURE: Integrated in Jenkins build bookkeeper-master #1696 (See [https://builds.apache.org/job/bookkeeper-master/1696/]) BOOKKEEPER-970: Bump the zookeeper version to 3.5 (sijie: rev 1e4ccaf167936fb4ad0de8c29ecf840bed3e89ed) * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/zookeeper/TestZooKeeperClient.java * (edit) bookkeeper-server/pom.xml * (edit) pom.xml * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/zookeeper/ZooKeeperClient.java * (edit) bookkeeper-benchmark/pom.xml * (add) bookkeeper-server/src/test/java/org/apache/bookkeeper/zookeeper/TestRetryPolicy.java > Bump zookeeper version to 3.5 > - > > Key: BOOKKEEPER-970 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-970 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-client, bookkeeper-server >Reporter: Sijie Guo >Assignee: Sijie Guo > Fix For: 4.5.0 > > > in DL, we need to leverage the asynchronous version of 'multi' in zookeeper. > so this jira is to bump the zookeeper version to 3.5 to support async multi. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-1007) Explicit LAC: make the interval configurable in milliseconds instead of seconds
[ https://issues.apache.org/jira/browse/BOOKKEEPER-1007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15938276#comment-15938276 ] Hudson commented on BOOKKEEPER-1007: FAILURE: Integrated in Jenkins build bookkeeper-master #1695 (See [https://builds.apache.org/job/bookkeeper-master/1695/]) BOOKKEEPER-1007: Explicit LAC: make the interval configurable in ms (sijie: rev f20568eaa25e0efe6926ede9030a210382e3005b) * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/BookKeeperTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/ClientConfiguration.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/ExplicitLacFlushPolicy.java > Explicit LAC: make the interval configurable in milliseconds instead of > seconds > --- > > Key: BOOKKEEPER-1007 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1007 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-client >Affects Versions: 4.5.0 >Reporter: Enrico Olivelli >Assignee: Enrico Olivelli >Priority: Critical > Fix For: 4.5.0 > > Original Estimate: 1h > Remaining Estimate: 1h > > in BOOKKEEPER-874 we introduced the "Explicit LAC", but the interval is > configurable in "seconds", and I need to achieve sub-second latency. > So my proposal is to make it configurable in "milliseconds", since 4.5.0 has > not yet been released I think it is better to change it as soon as possible -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-996) Apache Rat Check Failures
[
https://issues.apache.org/jira/browse/BOOKKEEPER-996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15848357#comment-15848357
]
Hudson commented on BOOKKEEPER-996:
---
FAILURE: Integrated in Jenkins build bookkeeper-master #1645 (See
[https://builds.apache.org/job/bookkeeper-master/1645/])
BOOKKEEPER-996: LongHierarchicalLedgerManagerFactory is missing license (sijie:
rev 91db1254c1118c4efa1c520f61850a3382f41fff)
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/LongHierarchicalLedgerManagerFactory.java
> Apache Rat Check Failures
> -
>
> Key: BOOKKEEPER-996
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-996
> Project: Bookkeeper
> Issue Type: Task
> Components: build
>Reporter: Sijie Guo
>Assignee: Sijie Guo
> Fix For: 4.5.0
>
>
> {code}
> [ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.7:check
> (default-cli) on project bookkeeper-server: Too many unapproved licenses: 1
> -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (BOOKKEEPER-943) Reduce log level of AbstractZkLedgerManager for register/unregister ReadOnlyLedgerHandle
[
https://issues.apache.org/jira/browse/BOOKKEEPER-943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15818293#comment-15818293
]
Hudson commented on BOOKKEEPER-943:
---
FAILURE: Integrated in Jenkins build bookkeeper-master #1624 (See
[https://builds.apache.org/job/bookkeeper-master/1624/])
BOOKKEEPER-943: Reduce log level of AbstractZkLedgerManager (sijie: rev
43971c3b9e4a9ac37ee07b398ed5506e040f9310)
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/AbstractZkLedgerManager.java
> Reduce log level of AbstractZkLedgerManager for register/unregister
> ReadOnlyLedgerHandle
>
>
> Key: BOOKKEEPER-943
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-943
> Project: Bookkeeper
> Issue Type: Wish
> Components: bookkeeper-client
>Affects Versions: 4.4.0
>Reporter: Enrico Olivelli
>Assignee: Enrico Olivelli
>Priority: Critical
> Fix For: 4.5.0
>
>
> Every time a ledger is opened/closed at least two lines at level INFO are
> written to the application log.
> In the case of "tailing" a ledger I open/close ledgers many times an so
> application log are filled up of this kind of lines:
> {code}
> Registered ledger metadata listener ReadOnlyLedgerHandle(lid = 29152, id =
> 1405806205) on ledger 29152.
> Unregistered ledger metadata listener ReadOnlyLedgerHandle(lid = 29152, id =
> 1405806205) on ledger 29152.
> {code}
> I would like to decrease the log level to "TRACE" instead of "INFO" in
> AbstractZkLedgerManager.java
> {code}
> LOG.info("Registered ledger metadata listener {} on ledger {}.", listener,
> ledgerId);
> LOG.info("Unregistered ledger metadata listener {} on ledger {}.", listener,
> ledgerId);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-992) ReadLog Command Enhancement
[ https://issues.apache.org/jira/browse/BOOKKEEPER-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15818292#comment-15818292 ] Hudson commented on BOOKKEEPER-992: --- FAILURE: Integrated in Jenkins build bookkeeper-master #1624 (See [https://builds.apache.org/job/bookkeeper-master/1624/]) BOOKKEEPER-992: ReadLog Command Enhancement (sijie: rev 15d6f600e7158e2eecb0ae7231455305e5d1ceaa) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/BookieShell.java > ReadLog Command Enhancement > --- > > Key: BOOKKEEPER-992 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-992 > Project: Bookkeeper > Issue Type: New Feature >Reporter: Charan Reddy Guttapalem >Assignee: Charan Reddy Guttapalem >Priority: Minor > Fix For: 4.5.0 > > > - Take arguments lid and eid and prints just that entry > - Take arguments of position range and print entries > which are located in that position range -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-964) Add concurrent maps and sets for primitive types
[
https://issues.apache.org/jira/browse/BOOKKEEPER-964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15770028#comment-15770028
]
Hudson commented on BOOKKEEPER-964:
---
SUCCESS: Integrated in Jenkins build bookkeeper-master #1604 (See
[https://builds.apache.org/job/bookkeeper-master/1604/])
BOOKKEEPER-964: Ignore findbugs warnings (sijie: rev
f53cbf82758af7f620ecd1506c76801bd0e75c86)
* (edit) bookkeeper-server/src/main/resources/findbugsExclude.xml
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieServer.java
> Add concurrent maps and sets for primitive types
>
>
> Key: BOOKKEEPER-964
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-964
> Project: Bookkeeper
> Issue Type: Improvement
>Reporter: Matteo Merli
>Assignee: Matteo Merli
> Fix For: 4.5.0
>
>
> In BookKeeper there are many instances of maps and sets that use ledger id
> and entry ids as keys or values. JDK concurrent collections have the overhead
> of boxing all the primitive values into objects (eg: long --> Long) that would
> need to be allocated from the heap. In addition to that, JDK map
> implementations
> are closed hash tables and they will require at least one more allocation to
> hold
> the linked-list/tree node.
> There are already available libraries that offer primitive collections with
> zero-allocation, but none of these support concurrent maps/sets.
> We have added a handful of specializations, all based on the same
> implementation
> idea. We have a hash table which is broken down into multiple sections. Each
> sections, on its own, is an open hash table with linear probing, protected by
> a stamped lock.
> All insertions, lookups and iterations on these collections are allocation
> free.
> {noformat}
> ConcurrentLongHashMap: Map
> ConcurrentLongHashSet: Set
> ConcurrentLongLongHashMap: Map
> ConcurrentLongLongPairHashMap: Map< Pair, Pair >
> ConcurrentOpenHashMap: Map
> ConcurrentOpenHashSet: Set
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-986) Handle Memtable flush failure
[ https://issues.apache.org/jira/browse/BOOKKEEPER-986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15767024#comment-15767024 ] Hudson commented on BOOKKEEPER-986: --- FAILURE: Integrated in Jenkins build bookkeeper-master #1603 (See [https://builds.apache.org/job/bookkeeper-master/1603/]) BOOKKEEPER-986: Handle memtable flush failure (vjujjuri: rev 6e738d0c02ebe6710a24aad505e2fd9386bb8821) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/SortedLedgerStorage.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/LedgerCacheTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/EntryMemTable.java > Handle Memtable flush failure > - > > Key: BOOKKEEPER-986 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-986 > Project: Bookkeeper > Issue Type: Bug >Reporter: Charan Reddy Guttapalem >Assignee: Charan Reddy Guttapalem >Priority: Critical > Fix For: 4.5.0 > > > Consider the following scenario > - currently bookie is in writemode but it is close to the diskusagethreshold > and also entrymemtable is close to skipListSizeLimit > - it received a addentry call, and now the size of the entrymemtable is > greater than skipListSizeLimit > - so onSizeLimitReached of SortedLedgerStorage will be called and now lets > assume that memtable flush has failed because of diskusagethreshold > - because of previous step, bookie would turn to readonly > - now lets assume after certain rounds of compaction bookie reclaimed > sufficient storage and went back to writemode > - now bookie would be receiving addentry calls > - because of outstanding snapshot from previous memtable flush failure, it > will never retry to flush memtable > - now the memory start keep going up and up and the process memory gets > bloated -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-983) BookieShell Command for LedgerDelete
[ https://issues.apache.org/jira/browse/BOOKKEEPER-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15767023#comment-15767023 ] Hudson commented on BOOKKEEPER-983: --- FAILURE: Integrated in Jenkins build bookkeeper-master #1603 (See [https://builds.apache.org/job/bookkeeper-master/1603/]) BOOKKEEPER-983: BookieShell LedgerDelete Command (vjujjuri: rev 9a506c261d3702ddd3f39556f4331b91da952a6e) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/BookieShell.java > BookieShell Command for LedgerDelete > > > Key: BOOKKEEPER-983 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-983 > Project: Bookkeeper > Issue Type: New Feature >Reporter: Charan Reddy Guttapalem >Assignee: Charan Reddy Guttapalem >Priority: Minor > Fix For: 4.5.0 > > > BookieShell Command for LedgerDelete -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-966) change the bookieServer cmdline to make conf-file and option co-exist
[ https://issues.apache.org/jira/browse/BOOKKEEPER-966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15756970#comment-15756970 ] Hudson commented on BOOKKEEPER-966: --- FAILURE: Integrated in Jenkins build bookkeeper-master #1599 (See [https://builds.apache.org/job/bookkeeper-master/1599/]) BOOKKEEPER-966: change bookieServer cmdline to make conf-file and option (sijie: rev 73edd4a99191d3c710b24a85eed3e2dbcd0a93bc) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieServer.java > change the bookieServer cmdline to make conf-file and option co-exist > - > > Key: BOOKKEEPER-966 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-966 > Project: Bookkeeper > Issue Type: Improvement >Reporter: Jia Zhai >Assignee: Jia Zhai > Fix For: 4.5.0 > > > Currently, when using bookieServer cmdline to start a bookie, you will either > give it a cofiguration file by "-c booke.conf"; or add some options like > " " in a fix > sequential. > It may not satisfy some of the requirement. So changed it to be co-exist for > configuration file and options. > By this change, it will first use settings in configuration file; and then > use options to overwrite some of the settings, if there are some options > provided. > Here is an example after this change: > BookieServer -c bookie.conf -z localhost:2181 -m /bookkeeper/ledgers -p 3181 > -j /mnt/journal -l "/mnt/ledger1 /mnt/ledger2 /mnt/ledger3” > Here, in this command: > -z is for “Zookeeper client instance”; > -m is for "Zookeeper ledgers root path for bookies"; > -p is for "bookie service port exported"; > -j is for "bookie journal directory"; > -l is for "bookie ledgers directories". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-967) Create new testsuite for testing RackAwareEnsemblePlacementPolicy using ScriptBasedMapping.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15756971#comment-15756971 ] Hudson commented on BOOKKEEPER-967: --- FAILURE: Integrated in Jenkins build bookkeeper-master #1599 (See [https://builds.apache.org/job/bookkeeper-master/1599/]) BOOKKEEPER-967: New testsuite for RackPlacement (sijie: rev 4bb57ef0b1b72eb1118963efc3ab9f0e9bec0e1c) * (add) bookkeeper-server/src/test/resources/networkmappingscript.sh * (add) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestRackawareEnsemblePlacementPolicyUsingScript.java > Create new testsuite for testing RackAwareEnsemblePlacementPolicy using > ScriptBasedMapping. > --- > > Key: BOOKKEEPER-967 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-967 > Project: Bookkeeper > Issue Type: Test >Reporter: Charan Reddy Guttapalem >Assignee: Charan Reddy Guttapalem >Priority: Minor > Fix For: 4.5.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-971) update bk codahale stats provider version
[ https://issues.apache.org/jira/browse/BOOKKEEPER-971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15756966#comment-15756966 ] Hudson commented on BOOKKEEPER-971: --- FAILURE: Integrated in Jenkins build bookkeeper-master #1599 (See [https://builds.apache.org/job/bookkeeper-master/1599/]) BOOKKEEPER-971: update bk codahale stats provider version (sijie: rev be99f024df5d53b3988f585ab46d6b604592acf1) * (edit) bookkeeper-stats-providers/codahale-metrics-provider/pom.xml * (edit) bookkeeper-stats-providers/codahale-metrics-provider/src/main/java/org/apache/bookkeeper/stats/CodahaleMetricsProvider.java > update bk codahale stats provider version > - > > Key: BOOKKEEPER-971 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-971 > Project: Bookkeeper > Issue Type: Improvement >Reporter: Jia Zhai >Assignee: Jia Zhai > Fix For: 4.5.0 > > > update bk stats provider: from codahale to yammer. > Currently io.dropwizard.metrics 3.1.0 is used most widely. will change > version to this version, and run the test. > And would like to change CodahaleMetricsProvider.getMetrics() to public, > since this would be used outside package. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-987) BookKeeper build is broken due to the shade plugin for commit ecbb053e6e
[
https://issues.apache.org/jira/browse/BOOKKEEPER-987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15756969#comment-15756969
]
Hudson commented on BOOKKEEPER-987:
---
FAILURE: Integrated in Jenkins build bookkeeper-master #1599 (See
[https://builds.apache.org/job/bookkeeper-master/1599/])
BOOKKEEPER-987: BookKeeper build is broken due to the shade plugin (sijie: rev
d2e82d4942be079cbfdb179e9a3fe3e574beff8d)
* (edit) bookkeeper-server/pom.xml
> BookKeeper build is broken due to the shade plugin for commit ecbb053e6e
>
>
> Key: BOOKKEEPER-987
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-987
> Project: Bookkeeper
> Issue Type: Bug
> Components: build
>Affects Versions: 4.5.0
>Reporter: Enrico Olivelli
>Assignee: Enrico Olivelli
>Priority: Blocker
> Fix For: 4.5.0
>
>
> Step to reproduce:
> - clone BookKeeper repository from GitHub
> - checkout master (or at least commit
> ecbb053e6e873859507e247cae727f4bc8b9f7fa)
> - mvn clean install -DskipTests -X
> The fix is to upgrade the shade plugin to the latest version (2.4.3)
> this is the error
> {code}
> [INFO] Excluding log4j:log4j:jar:1.2.15 from the shaded jar.
> [INFO] Minimizing jar
> org.apache.bookkeeper:bookkeeper-server:jar:4.5.0-SNAPSHOT
> [INFO]
>
> [INFO] Reactor Summary:
> [INFO]
> [INFO] bookkeeper . SUCCESS [ 0.428
> s]
> [INFO] compability dependencies ... SUCCESS [ 0.016
> s]
> [INFO] bookkeeper-server-compat400 SUCCESS [ 2.772
> s]
> [INFO] bookkeeper-server-compat410 SUCCESS [ 1.159
> s]
> [INFO] bookkeeper-server-compat420 SUCCESS [ 1.370
> s]
> [INFO] Stats API for bookkeeper ... SUCCESS [ 0.422
> s]
> [INFO] bookkeeper-server .. FAILURE [ 4.002
> s]
> [INFO] bookkeeper-benchmark ... SKIPPED
> [INFO] Stats provider for twitter-stats package ... SKIPPED
> [INFO] Stats provider for twitter-ostrich package . SKIPPED
> [INFO] Stats provider for codahale metrics SKIPPED
> [INFO] bookkeeper-stats-providers . SKIPPED
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 10.328 s
> [INFO] Finished at: 2016-12-16T17:06:08+01:00
> [INFO] Final Memory: 46M/998M
> [INFO]
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-shade-plugin:2.1:shade (default) on project
> bookkeeper-server: Error creating shaded jar: 46848 -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal org.apache.maven.plugins:maven-shade-plugin:2.1:shade (default) on
> project bookkeeper-server: Error creating shaded jar: 46848
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
> at
> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
> at
> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
> at
>
[jira] [Commented] (BOOKKEEPER-946) Provide an option to delay auto recovery of lost bookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15756972#comment-15756972 ] Hudson commented on BOOKKEEPER-946: --- FAILURE: Integrated in Jenkins build bookkeeper-master #1599 (See [https://builds.apache.org/job/bookkeeper-master/1599/]) BOOKKEEPER-946: Provide an option to delay auto recovery of lost bookies (sijie: rev 669ab4ac32bcbf6b3d883a07ed942d36d25b8a6e) * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/replication/AuditorLedgerCheckerTest.java > Provide an option to delay auto recovery of lost bookies > > > Key: BOOKKEEPER-946 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-946 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-server >Affects Versions: 4.5.0 >Reporter: Rithin Shetty >Assignee: Rithin Shetty > Fix For: 4.5.0 > > Attachments: > org.apache.bookkeeper.replication.AuditorLedgerCheckerTest-output.txt, > org.apache.bookkeeper.replication.AuditorLedgerCheckerTest-output.txt > > > If auto recovery is enabled, and a bookie goes down for upgrade or even if it > looses zk connection intermittently, the auditor detects it as a lost bookie > and starts under replication detection and the replication workers on other > bookie nodes start replicating the under replicated ledgers. All of this > stops once the bookie comes up but by then a few ledgers would get > replicated. Given the fact that we have multiple copies of data, it is > probably not necessary to start the recovery as soon as a bookie goes down. > We can probably wait for an hour or so and then start recovery. This should > cover cases like planned upgrade, intermittent network connectivity loss, > etc. The amount of time to wait can be an option and the default would be to > not wait at all(i.e. retain current behavior). > Of course, if more than one bookie goes down within a short interval, we > could decide to start auto recovery without waiting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-948) Provide an option to add more ledger/index directories to a bookie
[ https://issues.apache.org/jira/browse/BOOKKEEPER-948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15708561#comment-15708561 ] Hudson commented on BOOKKEEPER-948: --- FAILURE: Integrated in Jenkins build bookkeeper-master #1582 (See [https://builds.apache.org/job/bookkeeper-master/1582/]) BOOKKEEPER-948: Provide an option to add more ledger/index directorie… (sijie: rev 66515e74d3fb03d21cad4eccb446a90ab7e94597) * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/CookieTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/BookieShell.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/ServerConfiguration.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Cookie.java * (edit) bookkeeper-server/conf/bk_server.conf > Provide an option to add more ledger/index directories to a bookie > -- > > Key: BOOKKEEPER-948 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-948 > Project: Bookkeeper > Issue Type: New Feature > Components: bookkeeper-server >Affects Versions: 4.5.0 >Reporter: Rithin Shetty >Assignee: Rithin Shetty > Fix For: 4.5.0 > > Original Estimate: 336h > Remaining Estimate: 336h > > Addition of new ledger or index directories to an existing bookie is > disallowed via the cookie check in the bookie start path. Any attempt to add > new storage is rejected and the bookie doesn't come up. We have a need to add > additional storage to a bookie. This jira is for providing an option to add > additional storage to a bookie. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-912) Allow EnsemblePlacementPolicy to choose bookies using ledger custom data (multitenancy support)
[ https://issues.apache.org/jira/browse/BOOKKEEPER-912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15676677#comment-15676677 ] Hudson commented on BOOKKEEPER-912: --- FAILURE: Integrated in Jenkins build bookkeeper-master #1570 (See [https://builds.apache.org/job/bookkeeper-master/1570/]) BOOKKEEPER-912: Allow EnsemblePlacementPolicy to choose bookies using (sijie: rev 95ea4815b5313ea2ab02e5192c9f577d1c0fe24e) * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestRegionAwareEnsemblePlacementPolicy.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerCreateOp.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LocalBookieEnsemblePlacementPolicy.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/RackawareEnsemblePlacementPolicyImpl.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/RegionAwareEnsemblePlacementPolicy.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/EnsemblePlacementPolicy.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookieWatcher.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandle.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestRackawareEnsemblePlacementPolicy.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/RackawareEnsemblePlacementPolicy.java * (add) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/GenericEnsemblePlacementPolicyTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/DefaultEnsemblePlacementPolicy.java > Allow EnsemblePlacementPolicy to choose bookies using ledger custom data > (multitenancy support) > --- > > Key: BOOKKEEPER-912 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-912 > Project: Bookkeeper > Issue Type: New Feature > Components: bookkeeper-client >Affects Versions: 4.4.0 >Reporter: Enrico Olivelli >Assignee: Enrico Olivelli > Fix For: 4.5.0 > > > I would like to restrict the set of bookies to be used for a specific ledger. > Actually a single EnsemblePlacementPolicy is used for all the ledgers. > This is because I want to create a ledger only using a dedicated set of > machines/bookies which are dedicated to the 'tenant' for which I'm creating > the ledger. > We can add an optional (byte[]) parameter to asyncCreateLedger which in turn > is to be passed to the configured EnsemblePlacementPolicy which in turn will > be able to decide which are the most suitable bookies for the tenant. > This parameter must be stored on ledger metadata, in order to be used in > replaceBookie. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-965) Long Poll: Changes to the Write Path
[ https://issues.apache.org/jira/browse/BOOKKEEPER-965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15676676#comment-15676676 ] Hudson commented on BOOKKEEPER-965: --- FAILURE: Integrated in Jenkins build bookkeeper-master #1570 (See [https://builds.apache.org/job/bookkeeper-master/1570/]) BOOKKEEPER-965: Long Polling Part I: Changes in the write path (sijie: rev 9359d682a1598e30364eca6021d976f911e055b2) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerDescriptorImpl.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/FileInfo.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/SortedLedgerStorage.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/LedgerManagerTestCase.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerCacheImpl.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerStorage.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/InterleavedLedgerStorage.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerDescriptor.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/IndexPersistenceMgr.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerCache.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/TestSyncThread.java > Long Poll: Changes to the Write Path > > > Key: BOOKKEEPER-965 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-965 > Project: Bookkeeper > Issue Type: Sub-task > Components: bookkeeper-client, bookkeeper-server >Reporter: Robin Dhamankar >Assignee: Robin Dhamankar > Fix For: 4.5.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-957) issues in LedgerHandleAdv
[ https://issues.apache.org/jira/browse/BOOKKEEPER-957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15650886#comment-15650886 ] Hudson commented on BOOKKEEPER-957: --- SUCCESS: Integrated in Jenkins build bookkeeper-master #1560 (See [https://builds.apache.org/job/bookkeeper-master/1560/]) BOOKKEEPER-957: LedgerHandleAdv fixes (sijie: rev 8f0bed9f8401001abd683c420757610996cf96d8) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandleAdv.java > issues in LedgerHandleAdv > -- > > Key: BOOKKEEPER-957 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-957 > Project: Bookkeeper > Issue Type: Bug >Reporter: Charan Reddy Guttapalem >Assignee: Charan Reddy Guttapalem > > 1) In LedgerHandleAdv in doAsyncAddEntry method, addToLength(length) call is > not synchronized > In LedgerHandleAdv in doAsyncAddEntry method, call to addToLength(length) is > not synchronized. > addEntry method is supposed to be thread safe, so when addEntry method of > LedgerHandleAdv is called concurrently from multiple threads, addToLength > will be messed up because its call is not synchrnoized. > in LedgerHandle it is synchronized over 'this'. > 2) In LedgerHandleAdv, in asyncAddEntry it is incorrect to add 'op' to > 'pendingAddOps' before calling doAsyncAddEntry > In LedgerHandleAdv, in asyncAddEntry, it is incorrect to add 'op' to > 'pendingAddOps' before calling doAsyncAddEntry. > Consider the following example, > Lets say "asyncAddEntry(final long entryId, final byte[] data, final int > offset, final int length, final AddCallback cb, final Object ctx)" is called > with incorrect arguments, and following condition is failed "(offset < 0 || > length < 0 || (offset + length) > data.length)". Then as expected we would > get ArrayIndexOutOfBoundsException, but now if we try to call asyncAddEntry > with correct arguments for that entry, then it will fail with > "DuplicateEntryIdException", since the op is added already in the previous > call, which shouldn't have happened. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-946) Provide an option to delay auto recovery of lost bookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15650885#comment-15650885 ] Hudson commented on BOOKKEEPER-946: --- SUCCESS: Integrated in Jenkins build bookkeeper-master #1560 (See [https://builds.apache.org/job/bookkeeper-master/1560/]) BOOKKEEPER-946: Provide an option to delay auto recovery of lost bookies (sijie: rev 0abf37c64ced0fe49a6470bc0e2be632e47902d6) * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/replication/AuditorLedgerCheckerTest.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/ServerConfiguration.java * (edit) bookkeeper-server/conf/bk_server.conf * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/replication/ReplicationStats.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/replication/Auditor.java > Provide an option to delay auto recovery of lost bookies > > > Key: BOOKKEEPER-946 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-946 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-server >Affects Versions: 4.5.0 >Reporter: Rithin Shetty >Assignee: Rithin Shetty >Priority: Minor > Fix For: 4.5.0 > > Attachments: > org.apache.bookkeeper.replication.AuditorLedgerCheckerTest-output.txt, > org.apache.bookkeeper.replication.AuditorLedgerCheckerTest-output.txt > > > If auto recovery is enabled, and a bookie goes down for upgrade or even if it > looses zk connection intermittently, the auditor detects it as a lost bookie > and starts under replication detection and the replication workers on other > bookie nodes start replicating the under replicated ledgers. All of this > stops once the bookie comes up but by then a few ledgers would get > replicated. Given the fact that we have multiple copies of data, it is > probably not necessary to start the recovery as soon as a bookie goes down. > We can probably wait for an hour or so and then start recovery. This should > cover cases like planned upgrade, intermittent network connectivity loss, > etc. The amount of time to wait can be an option and the default would be to > not wait at all(i.e. retain current behavior). > Of course, if more than one bookie goes down within a short interval, we > could decide to start auto recovery without waiting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-952) Fix RegionAwarePlacementPolicy
[ https://issues.apache.org/jira/browse/BOOKKEEPER-952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15595019#comment-15595019 ] Hudson commented on BOOKKEEPER-952: --- FAILURE: Integrated in Jenkins build bookkeeper-master #1541 (See [https://builds.apache.org/job/bookkeeper-master/1541/]) BOOKKEEPER-952: Fix RegionAwareEnsemblePlacementPolicy (sijie: rev 4383b0b171d5fdfcdfa55c196998bbc6b7fdc260) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/RegionAwareEnsemblePlacementPolicy.java > Fix RegionAwarePlacementPolicy > -- > > Key: BOOKKEEPER-952 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-952 > Project: Bookkeeper > Issue Type: Bug > Components: bookkeeper-server >Reporter: Yiming Zang >Assignee: Yiming Zang > Fix For: 4.5.0 > > > TestRegionAwareEnsemblePlacementPolicy#testNewEnsembleWithThreeRegions is > failing because RegionAwareEnsemblePlacementPolicy is not working as expected. > The current allocation policy is not evenly across the regions. The write > quorum size is also not calculated correctly. > We need to fix the placement policy and then fix the the test case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-924) addEntry() is susceptible to spurious wakeups
[
https://issues.apache.org/jira/browse/BOOKKEEPER-924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15571862#comment-15571862
]
Hudson commented on BOOKKEEPER-924:
---
FAILURE: Integrated in Jenkins build bookkeeper-master #1533 (See
[https://builds.apache.org/job/bookkeeper-master/1533/])
BOOKKEEPER-924: addEntry() is susceptible to spurious wakeups (sijie: rev
bf4a4d6a07f9d615752054c6743035cebd86716e)
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeper.java
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeperAdmin.java
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandleAdv.java
* (edit)
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandle.java
* (add)
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/SynchCallbackUtils.java
> addEntry() is susceptible to spurious wakeups
> -
>
> Key: BOOKKEEPER-924
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-924
> Project: Bookkeeper
> Issue Type: Bug
>Affects Versions: 4.3.2
>Reporter: Venkateswararao Jujjuri (JV)
>Assignee: Matteo Merli
>Priority: Critical
> Fix For: 4.5.0
>
>
> LedgerHandle sync interface heavily depends on SyncCounter to convert async
> interfaces
> into sync interfaces.
> Usaylly
> SyncCounter.inc()
> asyncCall()
> SyncCOunter.block(0)
> The block code is.
>synchronized void block(int limit) throws InterruptedException {
> while (i > limit) {
> int prev = i;
> wait();
> if (i == prev) {
> break;
> }
> }
> }
> Since 'i' is going to be same as 'prev' on spurious wakeup, and wait() can
> return on spurious wakeups.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-612) RegionAwarePlacement Policy
[ https://issues.apache.org/jira/browse/BOOKKEEPER-612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15571861#comment-15571861 ] Hudson commented on BOOKKEEPER-612: --- FAILURE: Integrated in Jenkins build bookkeeper-master #1533 (See [https://builds.apache.org/job/bookkeeper-master/1533/]) BOOKKEEPER-612: Region aware placement (sijie: rev bbd1eb8d8560b03834175fbd996b85237df09f5c) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/net/NodeBase.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeper.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/net/CachedDNSToSwitchMapping.java * (add) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestRegionAwareEnsemblePlacementPolicy.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/EnsemblePlacementPolicy.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/RegionAwareEnsemblePlacementPolicy.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/net/Node.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/net/DNSToSwitchMapping.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/net/StabilizeNetworkTopology.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestRackawareEnsemblePlacementPolicy.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/UpdateLedgerCmdTest.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/RackawareEnsemblePlacementPolicyImpl.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/net/BookieSocketAddress.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerCreateOp.java * (edit) bookkeeper-server/pom.xml * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/DefaultEnsemblePlacementPolicy.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/net/ScriptBasedMapping.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/RackawareEnsemblePlacementPolicy.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/net/NetUtils.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookieWatcher.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/util/BookKeeperConstants.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LocalBookieEnsemblePlacementPolicy.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/net/AbstractDNSToSwitchMapping.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/net/DNS.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PerChannelBookieClient.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/ClientConfiguration.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/ITopologyAwareEnsemblePlacementPolicy.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/net/NetworkTopology.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/client/UpdateLedgerOpTest.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/util/StaticDNSResolver.java * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandle.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/net/NetworkTopologyImpl.java * (add) bookkeeper-server/src/main/java/org/apache/bookkeeper/client/TopologyAwareEnsemblePlacementPolicy.java > RegionAwarePlacement Policy > --- > > Key: BOOKKEEPER-612 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-612 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-client >Reporter: Sijie Guo >Assignee: Robin Dhamankar > Fix For: 4.5.0 > > > after BOOKKEEPER-592, it would be better to let ack quorum cover at least of > two racks, so a rack failure would not make an entry unavailable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-937) Upgrade protobuf to 2.6
[ https://issues.apache.org/jira/browse/BOOKKEEPER-937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15424476#comment-15424476 ] Hudson commented on BOOKKEEPER-937: --- SUCCESS: Integrated in Jenkins build bookkeeper-master #1476 (See [https://builds.apache.org/job/bookkeeper-master/1476/]) BOOKKEEPER-937: Upgrade protobuf to 2.6 (sijie: rev 28f23e80bc1f64a22b45c52166cd51d526eacb5b) * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/DataFormats.java * (edit) compat-deps/bookkeeper-server-compat-4.1.0/pom.xml * (edit) bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookkeeperProtocol.java * (edit) bookkeeper-server/src/test/java/org/apache/bookkeeper/proto/TestDataFormats.java * (edit) bookkeeper-server/pom.xml * (edit) pom.xml * (edit) bookkeeper-server/src/test/proto/TestDataFormats.proto * (edit) compat-deps/bookkeeper-server-compat-4.2.0/pom.xml * (edit) compat-deps/bookkeeper-server-compat-4.0.0/pom.xml > Upgrade protobuf to 2.6 > --- > > Key: BOOKKEEPER-937 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-937 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-server >Affects Versions: 4.5.0 >Reporter: Andrey Yegorov >Priority: Minor > Labels: protobuf > Fix For: 4.5.0 > > > I had to update protobuf definition for some internal experiments and found > that working with protobuf 2.4 is rather inconvenient. It cannot be installed > with brew on mac and building it on mac always result is build errors hence > leaves an option of switching to linux to run protoc. > I decided to upgrade to 2.6 instead. It is compatible with 2.4 on wire and > shaded so should not create any problems. All tests passed. > Please ignore changes in java files in attached patch during review; these > are auto-generated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-941) Introduce Feature Switches For controlling client and server behavior
[ https://issues.apache.org/jira/browse/BOOKKEEPER-941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15401147#comment-15401147 ] Hudson commented on BOOKKEEPER-941: --- SUCCESS: Integrated in bookkeeper-master #1459 (See [https://builds.apache.org/job/bookkeeper-master/1459/]) BOOKKEEPER-941: Feature Switches for controling client and server (sijie: rev c43858b5095318b50f4c7f86fc33b0a9e454bee5) * bookkeeper-server/src/main/java/org/apache/bookkeeper/feature/SettableFeatureProvider.java * bookkeeper-server/src/main/java/org/apache/bookkeeper/feature/FixedValueFeature.java * bookkeeper-server/src/main/java/org/apache/bookkeeper/feature/FeatureProvider.java * bookkeeper-server/src/main/java/org/apache/bookkeeper/feature/CacheableFeatureProvider.java * bookkeeper-server/src/main/java/org/apache/bookkeeper/feature/Feature.java * bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/AbstractConfiguration.java * bookkeeper-server/src/main/java/org/apache/bookkeeper/feature/SettableFeature.java > Introduce Feature Switches For controlling client and server behavior > - > > Key: BOOKKEEPER-941 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-941 > Project: Bookkeeper > Issue Type: New Feature >Reporter: Robin Dhamankar >Assignee: Robin Dhamankar >Priority: Minor > Fix For: 4.5.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-933) ClientConfiguration always inherits System properties
[
https://issues.apache.org/jira/browse/BOOKKEEPER-933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15401146#comment-15401146
]
Hudson commented on BOOKKEEPER-933:
---
SUCCESS: Integrated in bookkeeper-master #1459 (See
[https://builds.apache.org/job/bookkeeper-master/1459/])
BOOKKEEPER-933: ClientConfiguration always inherits System properties (sijie:
rev 8324632b2983beba1528ab53a999340756c7f563)
*
bookkeeper-server/src/test/java/org/apache/bookkeeper/conf/NoSystemPropertiesConfigurationTest.java
*
bookkeeper-server/src/test/java/org/apache/bookkeeper/conf/SystemPropertiesConfigurationTest.java
*
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/ConfigurationTest.java
*
bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/AbstractConfiguration.java
> ClientConfiguration always inherits System properties
> -
>
> Key: BOOKKEEPER-933
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-933
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client
>Reporter: Enrico Olivelli
>Assignee: Enrico Olivelli
> Fix For: 4.5.0
>
>
> ClientConfiguration is always adding System.getProperties()
> {code}
> protected AbstractConfiguration() {
> super();
> // add configuration for system properties
> addConfiguration(new SystemConfiguration());
> }
> {code}
> This is useful is some cases but sometimes it is very harmful because the
> user could modify the client behaviour inadvertently, for instance::
> throttle, readTimeout
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-927) Extend BOOKKEEPER-886 to LedgerHandleAdv too (BOOKKEEPER-886: Allow to disable ledgers operation throttling)
[ https://issues.apache.org/jira/browse/BOOKKEEPER-927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15393760#comment-15393760 ] Hudson commented on BOOKKEEPER-927: --- FAILURE: Integrated in bookkeeper-master #1454 (See [https://builds.apache.org/job/bookkeeper-master/1454/]) BOOKKEEPER-927: LedgerHandleAdv to handle disabled ledgers operation ⦠(vjujjuri: rev 1f7e47005588eedd879d0b9d163b4f125c6a0d59) * bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandleAdv.java > Extend BOOKKEEPER-886 to LedgerHandleAdv too (BOOKKEEPER-886: Allow to > disable ledgers operation throttling) > > > Key: BOOKKEEPER-927 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-927 > Project: Bookkeeper > Issue Type: Bug > Components: bookkeeper-client >Affects Versions: 4.3.2 >Reporter: Venkateswararao Jujjuri (JV) >Assignee: Venkateswararao Jujjuri (JV) >Priority: Minor > Fix For: 4.5.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-932) Move to JDK 8
[ https://issues.apache.org/jira/browse/BOOKKEEPER-932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371822#comment-15371822 ] Hudson commented on BOOKKEEPER-932: --- FAILURE: Integrated in bookkeeper-master #1438 (See [https://builds.apache.org/job/bookkeeper-master/1438/]) BOOKKEEPER-932: Move to JDK 8 (sijie: rev d5cea821c2201db77bd75a9bc48ff4bf9f483ce2) * pom.xml > Move to JDK 8 > - > > Key: BOOKKEEPER-932 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-932 > Project: Bookkeeper > Issue Type: Story >Reporter: Matteo Merli >Assignee: Matteo Merli >Priority: Minor > Fix For: 4.5.0 > > > We should move BookKeeper to compile with Java8 source level compatibility, > to take advantage of new Java features. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-908) Case to handle BKLedgerExistException
[
https://issues.apache.org/jira/browse/BOOKKEEPER-908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371823#comment-15371823
]
Hudson commented on BOOKKEEPER-908:
---
FAILURE: Integrated in bookkeeper-master #1438 (See
[https://builds.apache.org/job/bookkeeper-master/1438/])
BOOKKEEPER-908: BKLedgerExistException creation is missing (sijie: rev
e5f3afa5ef84dbc466562664f1f8b0cd60779c02)
* bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BKException.java
> Case to handle BKLedgerExistException
> -
>
> Key: BOOKKEEPER-908
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-908
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client
>Affects Versions: 4.3.2
>Reporter: Venkateswararao Jujjuri (JV)
>Assignee: Venkateswararao Jujjuri (JV)
>Priority: Minor
> Labels: patch
> Fix For: 4.5.0
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ---
> a/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BKException.java
> +++
> b/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BKException.java
> @@ -100,6 +102,8 @@ public abstract class BKException extends Exception {
> return new BKDuplicateEntryIdException();
> case Code.TimeoutException:
> return new BKTimeoutException();
> +case Code.LedgerExistException:
> +return new BKLedgerExistException();
> default:
> return new BKUnexpectedConditionException();
> }
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-922) Create a generic (K,V) map to store ledger metadata
[ https://issues.apache.org/jira/browse/BOOKKEEPER-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371821#comment-15371821 ] Hudson commented on BOOKKEEPER-922: --- FAILURE: Integrated in bookkeeper-master #1438 (See [https://builds.apache.org/job/bookkeeper-master/1438/]) BOOKKEEPER-922: CreaterId in ledger metadata (sijie: rev 6d71b8292bc255e21f2dcf048be191650c01b4dc) * bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/DataFormats.java * bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerMetadata.java * bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerCreateOp.java * bookkeeper-server/src/main/proto/DataFormats.proto * bookkeeper-server/src/main/java/org/apache/bookkeeper/client/BookKeeper.java * bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandle.java * bookkeeper-server/src/test/java/org/apache/bookkeeper/client/BookieWriteLedgerTest.java > Create a generic (K,V) map to store ledger metadata > --- > > Key: BOOKKEEPER-922 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-922 > Project: Bookkeeper > Issue Type: Improvement > Components: bookkeeper-client >Affects Versions: 4.3.2 >Reporter: Venkateswararao Jujjuri (JV) >Assignee: Rithin Shetty > Fix For: 4.5.0 > > > We have introduced ctime into ledger metadata through > https://issues.apache.org/jira/browse/BOOKKEEPER-879. > In the same token we would like to introduce createrId also. This can be a > 128 bit UUID. Caller can write tools to group ledgers by createrId, in the > future we can even enhance this to run queries based on createrId. > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-926) Compacted entries are not properly synced before updating index
[
https://issues.apache.org/jira/browse/BOOKKEEPER-926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15270692#comment-15270692
]
Hudson commented on BOOKKEEPER-926:
---
SUCCESS: Integrated in bookkeeper-master #1368 (See
[https://builds.apache.org/job/bookkeeper-master/1368/])
BOOKKEEPER-926: Compacted entries are not properly synced before (mmerli: rev
d32010f5fcc6a040a56dc8b983cc14d107cff2df)
*
bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/GarbageCollectorThread.java
*
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/CompactionTest.java
> Compacted entries are not properly synced before updating index
> ---
>
> Key: BOOKKEEPER-926
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-926
> Project: Bookkeeper
> Issue Type: Bug
>Affects Versions: 4.3.1
>Reporter: Matteo Merli
>Assignee: Matteo Merli
> Fix For: 4.4.0
>
>
> I have identified a couple of issues in Bookie compaction code.
> h4. Compacted entries are not properly synced when index is updated
> When compacting, we read "active" entries from an entry log and we re-append
> to current entry log. After compacting a number of entries, by default 100K,
> or at the very end, we need to update the index pointing to the new entry log
> and offset.
> Before updating the index, we need to wait for this entries to be flushed and
> fsynced, otherwise a bookie crash might leave the index updated, pointing to
> an invalid offset.
> The current code that is supposed to wait until flushed is:
> {code:java}
> // GarbageCollectorThread.java:178
> EntryLocation lastOffset = offsets.get(offsets.size()-1);
> long lastOffsetLogId = EntryLogger.logIdForOffset(lastOffset.location);
> while (lastOffsetLogId < entryLogger.getLeastUnflushedLogId() && running) {
> synchronized (flushLock) {
> flushLock.wait(1000);
> }
> lastOffset = offsets.get(offsets.size()-1);
> lastOffsetLogId = EntryLogger.logIdForOffset(lastOffset.location);
> }
> // update the index
> {code}
> The condition {{lastOffsetLogId < entryLogger.getLeastUnflushedLogId()}} is
> wrong, because if the last compacted entry was written in an earlier entry
> log than the least unflushed log, it means that the entries are already
> flushed and thus we don't need to wait.
> In the normal case what happens is that {{lastOffsetLogId} is actually the
> current entryLog and it's equal to {{entryLogger.getLeastUnflushedLogId()}},
> so we don't wait. But, in this case the entries appended to the current
> entrylog are not flushed nor synced, hence the problem.
> h4. Exception during index flush
> Having an exception when updating the index, combined with the above issue,
> makes the bookie GC to stop indefinitely.
> What happens is that the offset list is not cleared, and at the next bookie
> GC iteration it will find the old compacted entries in that list, for which
> now the entryLogId is less than the current log id, and that makes the while
> loop to never exit.
> Another problem is that, any IOException during the index flush, will make
> the GC thread to bail out and it will not remove even the entry logs that
> were compacted and flushed. Next time, these entry logs will be compacted
> again.
> h4. Proposed solution
> I think the best solution is to trigger the {{entryLogger.flush()}} form the
> bookie GC thread before updating the indexes. That would simplify the code
> and I don't see any disadvantages in doing that.
> Another improvement would be to delete compacted entry logs individually, as
> soon as the compacted data is flush, without waiting the end of the whole
> compaction process.
> The advantages are :
> * If compaction stop halfway, at least we don't have to re-compact what we
> just compacted
> * Compaction won't require significant space overhead. Today a major
> compaction can end up reappending a large amount of data and then deleting
> all the entry logs at the very end, requiring twice the size of the active
> data set to be stored on disk at some point in time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
