[distributedlog] branch master updated: ISSUE #213: Correct logger name in BKLogWriteHandler
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/distributedlog.git
The following commit(s) were added to refs/heads/master by this push:
new 487896b ISSUE #213: Correct logger name in BKLogWriteHandler
487896b is described below
commit 487896b7ac4ad2c7e097846f031f8e2a9ef4e9d6
Author: Shoukun Huai
AuthorDate: Wed Oct 18 12:14:58 2017 -0700
ISSUE #213: Correct logger name in BKLogWriteHandler
correct logger name in BKLogWriteHandler
Descriptions of the changes in this PR:
(PR description content here)...
Author: Shoukun Huai
Reviewers: Sijie Guo
This closes #215 from shoukunhuai/issue-213, closes #213
---
.../src/main/java/org/apache/distributedlog/BKLogWriteHandler.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/distributedlog-core/src/main/java/org/apache/distributedlog/BKLogWriteHandler.java
b/distributedlog-core/src/main/java/org/apache/distributedlog/BKLogWriteHandler.java
index 18ec549..955fafe 100644
---
a/distributedlog-core/src/main/java/org/apache/distributedlog/BKLogWriteHandler.java
+++
b/distributedlog-core/src/main/java/org/apache/distributedlog/BKLogWriteHandler.java
@@ -83,7 +83,7 @@ import org.slf4j.LoggerFactory;
*
*/
class BKLogWriteHandler extends BKLogHandler {
-static final Logger LOG = LoggerFactory.getLogger(BKLogReadHandler.class);
+static final Logger LOG = LoggerFactory.getLogger(BKLogWriteHandler.class);
private static Transaction.OpListener
NULL_OP_LISTENER =
new Transaction.OpListener() {
--
To stop receiving notification emails like this one, please contact
['"distributedlog-commits@bookkeeper.apache.org"
'].
[distributedlog] branch master updated: ISSUE173: Implement AutoCloseable in BKDistributedLogNamespace
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/distributedlog.git
The following commit(s) were added to refs/heads/master by this push:
new 51a0873 ISSUE173: Implement AutoCloseable in BKDistributedLogNamespace
51a0873 is described below
commit 51a08735d57b7c11b05afe9e03c6ef69ae3706d0
Author: Arvin
AuthorDate: Wed Oct 18 12:16:58 2017 -0700
ISSUE173: Implement AutoCloseable in BKDistributedLogNamespace
Descriptions of the changes in this PR:
Implement AutoCloseable in BKDistributedLogNamespace
Author: Arvin
Reviewers: Sijie Guo
This closes #214 from ArvinDevel/issue173
---
.../java/org/apache/distributedlog/BKDistributedLogNamespace.java| 5 +++--
.../main/java/org/apache/distributedlog/api/namespace/Namespace.java | 3 ++-
2 files changed, 5 insertions(+), 3 deletions(-)
diff --git
a/distributedlog-core/src/main/java/org/apache/distributedlog/BKDistributedLogNamespace.java
b/distributedlog-core/src/main/java/org/apache/distributedlog/BKDistributedLogNamespace.java
index bc6cc8b..b756feb 100644
---
a/distributedlog-core/src/main/java/org/apache/distributedlog/BKDistributedLogNamespace.java
+++
b/distributedlog-core/src/main/java/org/apache/distributedlog/BKDistributedLogNamespace.java
@@ -298,16 +298,17 @@ public class BKDistributedLogNamespace implements
Namespace {
/**
* Close the distributed log manager factory, freeing any resources it may
hold.
+ * close the resource in reverse order v.s. in which they are started
*/
@Override
public void close() {
if (!closed.compareAndSet(false, true)) {
return;
}
-// shutdown the driver
-Utils.close(driver);
// close the write limiter
this.writeLimiter.close();
+// shutdown the driver
+Utils.close(driver);
// Shutdown the schedulers
SchedulerUtils.shutdownScheduler(scheduler,
conf.getSchedulerShutdownTimeoutMs(),
TimeUnit.MILLISECONDS);
diff --git
a/distributedlog-core/src/main/java/org/apache/distributedlog/api/namespace/Namespace.java
b/distributedlog-core/src/main/java/org/apache/distributedlog/api/namespace/Namespace.java
index cad0f97..8766f17 100644
---
a/distributedlog-core/src/main/java/org/apache/distributedlog/api/namespace/Namespace.java
+++
b/distributedlog-core/src/main/java/org/apache/distributedlog/api/namespace/Namespace.java
@@ -70,7 +70,7 @@ import org.apache.distributedlog.namespace.NamespaceDriver;
*/
@Public
@Evolving
-public interface Namespace {
+public interface Namespace extends AutoCloseable{
/**
* Get the namespace driver used by this namespace.
@@ -187,6 +187,7 @@ public interface Namespace {
/**
* Close the namespace.
*/
+@Override
void close();
}
--
To stop receiving notification emails like this one, please contact
['"distributedlog-commits@bookkeeper.apache.org"
'].
[distributedlog] branch master updated (51a0873 -> 1b42f5b)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/distributedlog.git. from 51a0873 ISSUE173: Implement AutoCloseable in BKDistributedLogNamespace add 146979c Fix the flaky test TestBKDistributedLogManager.deleteDuringRead new 1b42f5b Issue 201: Fix the flaky test TestBKDistributedLogManager.deleteDuringRead The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../distributedlog/ReadAheadEntryReader.java | 6 .../TestBKDistributedLogManager.java | 35 ++ 2 files changed, 22 insertions(+), 19 deletions(-) -- To stop receiving notification emails like this one, please contact ['"distributedlog-commits@bookkeeper.apache.org" '].
[distributedlog] 01/01: Issue 201: Fix the flaky test TestBKDistributedLogManager.deleteDuringRead
This is an automated email from the ASF dual-hosted git repository. sijie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/distributedlog.git commit 1b42f5b4f00b1445d097963967619e5d4e6f452f Merge: 51a0873 146979c Author: Sijie Guo AuthorDate: Wed Oct 18 13:06:39 2017 -0700 Issue 201: Fix the flaky test TestBKDistributedLogManager.deleteDuringRead This closes #204, #201 .../distributedlog/ReadAheadEntryReader.java | 6 .../TestBKDistributedLogManager.java | 35 ++ 2 files changed, 22 insertions(+), 19 deletions(-) -- To stop receiving notification emails like this one, please contact "distributedlog-commits@bookkeeper.apache.org" .
[distributedlog] 01/01: Issue 205: Avoid copying bytebuf for constructing log record to write
This is an automated email from the ASF dual-hosted git repository. sijie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/distributedlog.git commit 07e2a8a2ae3d7217ce1ea695251a9d9903f08721 Merge: 1b42f5b c2cd6d2 Author: Sijie Guo AuthorDate: Wed Oct 18 13:07:54 2017 -0700 Issue 205: Avoid copying bytebuf for constructing log record to write Descriptions of the changes in this PR: - avoid copying memory for writer facing constructor for log records - construct a ByteBuf by copying ByteBuffer for reader facing constructors This closes #205, #206 .../src/main/java/org/apache/distributedlog/LogRecord.java | 6 ++ .../src/main/java/org/apache/distributedlog/LogRecordWithDLSN.java | 6 +- 2 files changed, 7 insertions(+), 5 deletions(-) -- To stop receiving notification emails like this one, please contact "distributedlog-commits@bookkeeper.apache.org" .
[distributedlog] branch master updated (1b42f5b -> 07e2a8a)
This is an automated email from the ASF dual-hosted git repository. sijie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/distributedlog.git. from 1b42f5b Issue 201: Fix the flaky test TestBKDistributedLogManager.deleteDuringRead add c2cd6d2 Avoid copying bytebuf for constructing log record to write new 07e2a8a Issue 205: Avoid copying bytebuf for constructing log record to write The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../src/main/java/org/apache/distributedlog/LogRecord.java | 6 ++ .../src/main/java/org/apache/distributedlog/LogRecordWithDLSN.java | 6 +- 2 files changed, 7 insertions(+), 5 deletions(-) -- To stop receiving notification emails like this one, please contact ['"distributedlog-commits@bookkeeper.apache.org" '].
[distributedlog] branch master updated: ISSUE #207: Support getFirstLogRecord
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/distributedlog.git
The following commit(s) were added to refs/heads/master by this push:
new 442e000 ISSUE #207: Support getFirstLogRecord
442e000 is described below
commit 442e000201af0a1e028919f835fadeabc9baaf08
Author: Sijie Guo
AuthorDate: Wed Oct 18 13:09:34 2017 -0700
ISSUE #207: Support getFirstLogRecord
Descriptions of the changes in this PR:
- support getFirstLogRecord
Author: Sijie Guo
Reviewers: Jia Zhai
This closes #208 from sijie/3_get_firstrecord_pr, closes #207
---
.../distributedlog/BKDistributedLogManager.java| 13 ++-
.../apache/distributedlog/LogSegmentMetadata.java | 25 --
.../distributedlog/api/DistributedLogManager.java | 24 +
3 files changed, 59 insertions(+), 3 deletions(-)
diff --git
a/distributedlog-core/src/main/java/org/apache/distributedlog/BKDistributedLogManager.java
b/distributedlog-core/src/main/java/org/apache/distributedlog/BKDistributedLogManager.java
index d5d1477..5ca76d3 100644
---
a/distributedlog-core/src/main/java/org/apache/distributedlog/BKDistributedLogManager.java
+++
b/distributedlog-core/src/main/java/org/apache/distributedlog/BKDistributedLogManager.java
@@ -273,7 +273,8 @@ class BKDistributedLogManager implements
DistributedLogManager {
return Utils.ioResult(getLogSegmentsAsync());
}
-protected CompletableFuture>
getLogSegmentsAsync() {
+@Override
+public CompletableFuture> getLogSegmentsAsync() {
final BKLogReadHandler readHandler = createReadHandler();
return readHandler.readLogSegmentsFromStore(
LogSegmentMetadata.COMPARATOR,
@@ -901,6 +902,16 @@ class BKDistributedLogManager implements
DistributedLogManager {
return getFirstRecordAsyncInternal().thenApply(RECORD_2_DLSN_FUNCTION);
}
+@Override
+public LogRecordWithDLSN getFirstLogRecord() throws IOException {
+return Utils.ioResult(getFirstRecordAsyncInternal());
+}
+
+@Override
+public CompletableFuture getFirstLogRecordAsync() {
+return getFirstRecordAsyncInternal();
+}
+
private CompletableFuture getFirstRecordAsyncInternal()
{
return processReaderOperation(new Function>() {
@Override
diff --git
a/distributedlog-core/src/main/java/org/apache/distributedlog/LogSegmentMetadata.java
b/distributedlog-core/src/main/java/org/apache/distributedlog/LogSegmentMetadata.java
index c8907b5..0e4a2f0 100644
---
a/distributedlog-core/src/main/java/org/apache/distributedlog/LogSegmentMetadata.java
+++
b/distributedlog-core/src/main/java/org/apache/distributedlog/LogSegmentMetadata.java
@@ -83,8 +83,8 @@ public class LogSegmentMetadata {
/**
* TruncationStatus.
*/
-public enum TruncationStatus {
-ACTIVE (0), PARTIALLY_TRUNCATED(1), TRUNCATED (2);
+public enum TruncationStatus {
+UNKNOWN(-1), ACTIVE (0), PARTIALLY_TRUNCATED(1), TRUNCATED (2);
private final int value;
private TruncationStatus(int value) {
@@ -503,6 +503,19 @@ public class LogSegmentMetadata {
? startSequenceId : Long.MIN_VALUE +
(getLogSegmentSequenceNumber() << 32L);
}
+public TruncationStatus getTruncationStatus() {
+switch ((int) (status & METADATA_TRUNCATION_STATUS_MASK)) {
+case 0:
+return TruncationStatus.ACTIVE;
+case 1:
+return TruncationStatus.PARTIALLY_TRUNCATED;
+case 2:
+return TruncationStatus.TRUNCATED;
+default:
+return TruncationStatus.UNKNOWN;
+}
+}
+
public boolean isTruncated() {
return ((status & METADATA_TRUNCATION_STATUS_MASK)
== TruncationStatus.TRUNCATED.value);
@@ -530,6 +543,14 @@ public class LogSegmentMetadata {
return minActiveDLSN;
}
+public long getMinActiveEntryId() {
+return minActiveDLSN.getEntryId();
+}
+
+public long getMinActiveSlotId() {
+return minActiveDLSN.getSlotId();
+}
+
public DLSN getFirstDLSN() {
return new DLSN(getLogSegmentSequenceNumber(), 0, 0);
}
diff --git
a/distributedlog-core/src/main/java/org/apache/distributedlog/api/DistributedLogManager.java
b/distributedlog-core/src/main/java/org/apache/distributedlog/api/DistributedLogManager.java
index fb9c6be..db4b65e 100644
---
a/distributedlog-core/src/main/java/org/apache/distributedlog/api/DistributedLogManager.java
+++
b/distributedlog-core/src/main/java/org/apache/distributedlog/api/DistributedLogManager.java
@@ -71,6 +71,14 @@ public interface DistributedLogManager extends
AsyncCloseable, Closeable {
*/
List getLogSegments() throws IOException;
+
+/**
+ * Get the log segments asyn
[distributedlog] branch master updated: ISSUE #211: Support listing logs by prefix
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/distributedlog.git
The following commit(s) were added to refs/heads/master by this push:
new 3610f0f ISSUE #211: Support listing logs by prefix
3610f0f is described below
commit 3610f0f1083329a9baa7fbe8a3002905594cec92
Author: Sijie Guo
AuthorDate: Wed Oct 18 15:14:16 2017 -0700
ISSUE #211: Support listing logs by prefix
Descriptions of the changes in this PR:
- extend `getLogs` to `getLogs(prefix)`, so it provides a filesystem
`listFiles`-like semantic.
Author: Sijie Guo
Reviewers: Jia Zhai
This closes #212 from sijie/5_support_list_logs_by_prefix_pr, closes #211
---
.../distributedlog/BKDistributedLogNamespace.java | 9 -
.../apache/distributedlog/api/namespace/Namespace.java | 10 ++
.../apache/distributedlog/impl/ZKLogMetadataStore.java | 10 --
.../impl/federated/FederatedZKLogMetadataStore.java| 7 ++-
.../distributedlog/metadata/LogMetadataStore.java | 4 +++-
.../distributedlog/impl/TestZKLogMetadataStore.java| 18 +++---
.../federated/TestFederatedZKLogMetadataStore.java | 8
7 files changed, 54 insertions(+), 12 deletions(-)
diff --git
a/distributedlog-core/src/main/java/org/apache/distributedlog/BKDistributedLogNamespace.java
b/distributedlog-core/src/main/java/org/apache/distributedlog/BKDistributedLogNamespace.java
index b756feb..cd5a17a 100644
---
a/distributedlog-core/src/main/java/org/apache/distributedlog/BKDistributedLogNamespace.java
+++
b/distributedlog-core/src/main/java/org/apache/distributedlog/BKDistributedLogNamespace.java
@@ -214,7 +214,14 @@ public class BKDistributedLogNamespace implements
Namespace {
@Override
public Iterator getLogs() throws IOException {
checkState();
-return Utils.ioResult(driver.getLogMetadataStore().getLogs());
+return Utils.ioResult(driver.getLogMetadataStore().getLogs(""));
+}
+
+@Override
+public Iterator getLogs(String logNamePrefix) throws IOException {
+checkState();
+logNamePrefix = validateAndNormalizeName(logNamePrefix);
+return
Utils.ioResult(driver.getLogMetadataStore().getLogs(logNamePrefix));
}
@Override
diff --git
a/distributedlog-core/src/main/java/org/apache/distributedlog/api/namespace/Namespace.java
b/distributedlog-core/src/main/java/org/apache/distributedlog/api/namespace/Namespace.java
index 8766f17..dafc099 100644
---
a/distributedlog-core/src/main/java/org/apache/distributedlog/api/namespace/Namespace.java
+++
b/distributedlog-core/src/main/java/org/apache/distributedlog/api/namespace/Namespace.java
@@ -163,6 +163,16 @@ public interface Namespace extends AutoCloseable{
Iterator getLogs()
throws IOException;
+/**
+ * Retrieve the logs under a given logNamePrefix.
+ *
+ * @param logNamePrefix log name prefix
+ * @return iterator of the logs under the log name prefix
+ * @throws IOException when encountered issues with backend.
+ */
+Iterator getLogs(String logNamePrefix)
+throws IOException;
+
//
// Methods for namespace
//
diff --git
a/distributedlog-core/src/main/java/org/apache/distributedlog/impl/ZKLogMetadataStore.java
b/distributedlog-core/src/main/java/org/apache/distributedlog/impl/ZKLogMetadataStore.java
index 43fed26..32e7ab7 100644
---
a/distributedlog-core/src/main/java/org/apache/distributedlog/impl/ZKLogMetadataStore.java
+++
b/distributedlog-core/src/main/java/org/apache/distributedlog/impl/ZKLogMetadataStore.java
@@ -25,6 +25,7 @@ import java.net.URI;
import java.util.Iterator;
import java.util.List;
import java.util.concurrent.CompletableFuture;
+import org.apache.commons.lang.StringUtils;
import org.apache.distributedlog.DistributedLogConfiguration;
import org.apache.distributedlog.ZooKeeperClient;
import org.apache.distributedlog.callback.NamespaceListener;
@@ -70,9 +71,14 @@ public class ZKLogMetadataStore implements LogMetadataStore {
}
@Override
-public CompletableFuture> getLogs() {
+public CompletableFuture> getLogs(String logNamePrefix) {
final CompletableFuture> promise = new
CompletableFuture>();
-final String nsRootPath = namespace.getPath();
+final String nsRootPath;
+if (StringUtils.isEmpty(logNamePrefix)) {
+nsRootPath = namespace.getPath();
+} else {
+nsRootPath = namespace.getPath() + "/" + logNamePrefix;
+}
try {
final ZooKeeper zk = zkc.get();
zk.sync(nsRootPath, new AsyncCallback.VoidCallback() {
diff --git
a/distributedlog-core/src/main/java/org/apache/distributedlog/impl/federated/FederatedZKLogMetadataStore.java
b/distributedlog-core/src/main/java/org/apache/distributedlog/impl/federated/Federa
