[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[ https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15162955#comment-15162955 ] Hudson commented on BOOKKEEPER-879: --- FAILURE: Integrated in bookkeeper-master #1296 (See [https://builds.apache.org/job/bookkeeper-master/1296/]) BOOKKEEPER-879: Record ledger creation time (sijie: rev bac3ed3603f9dc9864beb610bea215c186ef4fab) * bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerMetadata.java * bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/MSLedgerManagerFactory.java * bookkeeper-server/src/test/java/org/apache/bookkeeper/client/ListLedgersTest.java * bookkeeper-server/src/main/proto/DataFormats.proto * bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/AbstractZkLedgerManager.java * bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/DataFormats.java > Record ledger creation time > --- > > Key: BOOKKEEPER-879 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879 > Project: Bookkeeper > Issue Type: New Feature > Components: bookkeeper-client, bookkeeper-server >Affects Versions: 4.3.1 >Reporter: Enrico Olivelli >Assignee: Enrico Olivelli > Fix For: 4.4.0 > > > I think that a creation timestamp would be very useful and it does not cost > very much. It would be an immutable value. > Nowadays (on 4.3.1) I must keep that info together with the id of the ledger, > for instance in zookeeper, but if that reference gets lost there is now way > to know how old a ledger is. > I think that this timestamp should be captured on client while calling > createLedger or asyncCreateLedger. > In addition to this very common field maybe it would be useful to add a > custom byte[] field named "custom client data" in order to let the client > 'describe' the ledger without the need of extra data on Zookeeper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[ https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15159364#comment-15159364 ] Sijie Guo commented on BOOKKEEPER-879: -- Thanks [~eolivelli] for the change and thanks [~fpj] for the reviews. > Record ledger creation time > --- > > Key: BOOKKEEPER-879 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879 > Project: Bookkeeper > Issue Type: New Feature > Components: bookkeeper-client, bookkeeper-server >Affects Versions: 4.3.1 >Reporter: Enrico Olivelli >Assignee: Enrico Olivelli > Fix For: 4.4.0 > > > I think that a creation timestamp would be very useful and it does not cost > very much. It would be an immutable value. > Nowadays (on 4.3.1) I must keep that info together with the id of the ledger, > for instance in zookeeper, but if that reference gets lost there is now way > to know how old a ledger is. > I think that this timestamp should be captured on client while calling > createLedger or asyncCreateLedger. > In addition to this very common field maybe it would be useful to add a > custom byte[] field named "custom client data" in order to let the client > 'describe' the ledger without the need of extra data on Zookeeper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[ https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15159355#comment-15159355 ] ASF GitHub Bot commented on BOOKKEEPER-879: --- Github user asfgit closed the pull request at: https://github.com/apache/bookkeeper/pull/2 > Record ledger creation time > --- > > Key: BOOKKEEPER-879 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879 > Project: Bookkeeper > Issue Type: New Feature > Components: bookkeeper-client, bookkeeper-server >Affects Versions: 4.3.1 >Reporter: Enrico Olivelli > Fix For: 4.4.0 > > > I think that a creation timestamp would be very useful and it does not cost > very much. It would be an immutable value. > Nowadays (on 4.3.1) I must keep that info together with the id of the ledger, > for instance in zookeeper, but if that reference gets lost there is now way > to know how old a ledger is. > I think that this timestamp should be captured on client while calling > createLedger or asyncCreateLedger. > In addition to this very common field maybe it would be useful to add a > custom byte[] field named "custom client data" in order to let the client > 'describe' the ledger without the need of extra data on Zookeeper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[ https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15146488#comment-15146488 ] ASF GitHub Bot commented on BOOKKEEPER-879: --- Github user fpj commented on the pull request: https://github.com/apache/bookkeeper/pull/2#issuecomment-183871751 +1, LGTM @eolivelli @sijie > Record ledger creation time > --- > > Key: BOOKKEEPER-879 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879 > Project: Bookkeeper > Issue Type: New Feature > Components: bookkeeper-client, bookkeeper-server >Affects Versions: 4.3.1 >Reporter: Enrico Olivelli > Fix For: 4.4.0 > > > I think that a creation timestamp would be very useful and it does not cost > very much. It would be an immutable value. > Nowadays (on 4.3.1) I must keep that info together with the id of the ledger, > for instance in zookeeper, but if that reference gets lost there is now way > to know how old a ledger is. > I think that this timestamp should be captured on client while calling > createLedger or asyncCreateLedger. > In addition to this very common field maybe it would be useful to add a > custom byte[] field named "custom client data" in order to let the client > 'describe' the ledger without the need of extra data on Zookeeper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[ https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15138463#comment-15138463 ] ASF GitHub Bot commented on BOOKKEEPER-879: --- Github user sijie commented on the pull request: https://github.com/apache/bookkeeper/pull/2#issuecomment-181741858 @fpj what's your opinion about this? > Record ledger creation time > --- > > Key: BOOKKEEPER-879 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879 > Project: Bookkeeper > Issue Type: New Feature > Components: bookkeeper-client, bookkeeper-server >Affects Versions: 4.3.1 >Reporter: Enrico Olivelli > Fix For: 4.4.0 > > > I think that a creation timestamp would be very useful and it does not cost > very much. It would be an immutable value. > Nowadays (on 4.3.1) I must keep that info together with the id of the ledger, > for instance in zookeeper, but if that reference gets lost there is now way > to know how old a ledger is. > I think that this timestamp should be captured on client while calling > createLedger or asyncCreateLedger. > In addition to this very common field maybe it would be useful to add a > custom byte[] field named "custom client data" in order to let the client > 'describe' the ledger without the need of extra data on Zookeeper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[ https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15133124#comment-15133124 ] Enrico Olivelli commented on BOOKKEEPER-879: Ok. I updated the pull request > Record ledger creation time > --- > > Key: BOOKKEEPER-879 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879 > Project: Bookkeeper > Issue Type: New Feature > Components: bookkeeper-client, bookkeeper-server >Affects Versions: 4.3.1 >Reporter: Enrico Olivelli > Fix For: 4.4.0 > > > I think that a creation timestamp would be very useful and it does not cost > very much. It would be an immutable value. > Nowadays (on 4.3.1) I must keep that info together with the id of the ledger, > for instance in zookeeper, but if that reference gets lost there is now way > to know how old a ledger is. > I think that this timestamp should be captured on client while calling > createLedger or asyncCreateLedger. > In addition to this very common field maybe it would be useful to add a > custom byte[] field named "custom client data" in order to let the client > 'describe' the ledger without the need of extra data on Zookeeper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[ https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15128804#comment-15128804 ] Sijie Guo commented on BOOKKEEPER-879: -- no. I mean using guava Optional class. so MSLedgerManagerFactory could return Optional.absent(). > Record ledger creation time > --- > > Key: BOOKKEEPER-879 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879 > Project: Bookkeeper > Issue Type: New Feature > Components: bookkeeper-client, bookkeeper-server >Affects Versions: 4.3.1 >Reporter: Enrico Olivelli > Fix For: 4.4.0 > > > I think that a creation timestamp would be very useful and it does not cost > very much. It would be an immutable value. > Nowadays (on 4.3.1) I must keep that info together with the id of the ledger, > for instance in zookeeper, but if that reference gets lost there is now way > to know how old a ledger is. > I think that this timestamp should be captured on client while calling > createLedger or asyncCreateLedger. > In addition to this very common field maybe it would be useful to add a > custom byte[] field named "custom client data" in order to let the client > 'describe' the ledger without the need of extra data on Zookeeper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[ https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15127944#comment-15127944 ] Enrico Olivelli commented on BOOKKEEPER-879: I'm sorry, what do you mean with "Optional" ? do you mean an overloaded method which does not take that argument ? I cannot find any usage of an "Optional" class in bookkeeper code > Record ledger creation time > --- > > Key: BOOKKEEPER-879 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879 > Project: Bookkeeper > Issue Type: New Feature > Components: bookkeeper-client, bookkeeper-server >Affects Versions: 4.3.1 >Reporter: Enrico Olivelli > Fix For: 4.4.0 > > > I think that a creation timestamp would be very useful and it does not cost > very much. It would be an immutable value. > Nowadays (on 4.3.1) I must keep that info together with the id of the ledger, > for instance in zookeeper, but if that reference gets lost there is now way > to know how old a ledger is. > I think that this timestamp should be captured on client while calling > createLedger or asyncCreateLedger. > In addition to this very common field maybe it would be useful to add a > custom byte[] field named "custom client data" in order to let the client > 'describe' the ledger without the need of extra data on Zookeeper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[
https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15127852#comment-15127852
]
ASF GitHub Bot commented on BOOKKEEPER-879:
---
Github user sijie commented on a diff in the pull request:
https://github.com/apache/bookkeeper/pull/2#discussion_r51534872
--- Diff:
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/AbstractZkLedgerManager.java
---
@@ -355,7 +355,7 @@ public void processResult(int rc, String path, Object
ctx, byte[] data, Stat sta
LedgerMetadata metadata;
try {
-metadata = LedgerMetadata.parseConfig(data, new
ZkVersion(stat.getVersion()));
+metadata = LedgerMetadata.parseConfig(data, new
ZkVersion(stat.getVersion()),stat.getCtime());
--- End diff --
@fpj when stat will be null? does it return null when rc code is OK?
> Record ledger creation time
> ---
>
> Key: BOOKKEEPER-879
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879
> Project: Bookkeeper
> Issue Type: New Feature
> Components: bookkeeper-client, bookkeeper-server
>Affects Versions: 4.3.1
>Reporter: Enrico Olivelli
> Fix For: 4.4.0
>
>
> I think that a creation timestamp would be very useful and it does not cost
> very much. It would be an immutable value.
> Nowadays (on 4.3.1) I must keep that info together with the id of the ledger,
> for instance in zookeeper, but if that reference gets lost there is now way
> to know how old a ledger is.
> I think that this timestamp should be captured on client while calling
> createLedger or asyncCreateLedger.
> In addition to this very common field maybe it would be useful to add a
> custom byte[] field named "custom client data" in order to let the client
> 'describe' the ledger without the need of extra data on Zookeeper.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[
https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15127789#comment-15127789
]
ASF GitHub Bot commented on BOOKKEEPER-879:
---
Github user sijie commented on a diff in the pull request:
https://github.com/apache/bookkeeper/pull/2#discussion_r51531399
--- Diff:
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerMetadata.java
---
@@ -308,11 +319,13 @@ long getNextEnsembleChange(long entryId) {
*byte array to parse
* @param version
*version of the ledger metadata
+ * @param zkCtime
+ *ctime read from Zookeeper Stat, used for legacy ledgers
* @return LedgerConfig
* @throws IOException
* if the given byte[] cannot be parsed
*/
-public static LedgerMetadata parseConfig(byte[] bytes, Version
version) throws IOException {
+public static LedgerMetadata parseConfig(byte[] bytes, Version
version, Long zkCtime) throws IOException {
--- End diff --
change 'zkCtime' to 'msCtime' (stands for metadata store creation time)?
since the ledger metadata here isn't actually tight with zookeeper time.
And I'd prefer Optional rather than Long.
> Record ledger creation time
> ---
>
> Key: BOOKKEEPER-879
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879
> Project: Bookkeeper
> Issue Type: New Feature
> Components: bookkeeper-client, bookkeeper-server
>Affects Versions: 4.3.1
>Reporter: Enrico Olivelli
> Fix For: 4.4.0
>
>
> I think that a creation timestamp would be very useful and it does not cost
> very much. It would be an immutable value.
> Nowadays (on 4.3.1) I must keep that info together with the id of the ledger,
> for instance in zookeeper, but if that reference gets lost there is now way
> to know how old a ledger is.
> I think that this timestamp should be captured on client while calling
> createLedger or asyncCreateLedger.
> In addition to this very common field maybe it would be useful to add a
> custom byte[] field named "custom client data" in order to let the client
> 'describe' the ledger without the need of extra data on Zookeeper.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[
https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15125101#comment-15125101
]
ASF GitHub Bot commented on BOOKKEEPER-879:
---
Github user eolivelli commented on a diff in the pull request:
https://github.com/apache/bookkeeper/pull/2#discussion_r51351686
--- Diff:
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/AbstractZkLedgerManager.java
---
@@ -355,7 +355,7 @@ public void processResult(int rc, String path, Object
ctx, byte[] data, Stat sta
LedgerMetadata metadata;
try {
-metadata = LedgerMetadata.parseConfig(data, new
ZkVersion(stat.getVersion()));
+metadata = LedgerMetadata.parseConfig(data, new
ZkVersion(stat.getVersion()),stat.getCtime());
--- End diff --
I added an extra check on stat object, the result will the same as of the
IOException (unreadable data in this case), what do you think ?
> Record ledger creation time
> ---
>
> Key: BOOKKEEPER-879
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879
> Project: Bookkeeper
> Issue Type: New Feature
> Components: bookkeeper-client, bookkeeper-server
>Affects Versions: 4.3.1
>Reporter: Enrico Olivelli
> Fix For: 4.4.0
>
>
> I think that a creation timestamp would be very useful and it does not cost
> very much. It would be an immutable value.
> Nowadays (on 4.3.1) I must keep that info together with the id of the ledger,
> for instance in zookeeper, but if that reference gets lost there is now way
> to know how old a ledger is.
> I think that this timestamp should be captured on client while calling
> createLedger or asyncCreateLedger.
> In addition to this very common field maybe it would be useful to add a
> custom byte[] field named "custom client data" in order to let the client
> 'describe' the ledger without the need of extra data on Zookeeper.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[
https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15124982#comment-15124982
]
Flavio Junqueira commented on BOOKKEEPER-879:
-
I think you need to bump the metadata version number and you could check the
version and call methods with different signatures to deal with the creation
time. Also, it is not pretty that {{MSLedgerManagerFactory}} isn't really
setting the field, perhaps we should have a default way of generating a
creation time, like having the client doing it or using a zk znode for that
(e.g., create a znode, get the ctime, delete the znode).
> Record ledger creation time
> ---
>
> Key: BOOKKEEPER-879
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879
> Project: Bookkeeper
> Issue Type: New Feature
> Components: bookkeeper-client, bookkeeper-server
>Affects Versions: 4.3.1
>Reporter: Enrico Olivelli
> Fix For: 4.4.0
>
>
> I think that a creation timestamp would be very useful and it does not cost
> very much. It would be an immutable value.
> Nowadays (on 4.3.1) I must keep that info together with the id of the ledger,
> for instance in zookeeper, but if that reference gets lost there is now way
> to know how old a ledger is.
> I think that this timestamp should be captured on client while calling
> createLedger or asyncCreateLedger.
> In addition to this very common field maybe it would be useful to add a
> custom byte[] field named "custom client data" in order to let the client
> 'describe' the ledger without the need of extra data on Zookeeper.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[
https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15124979#comment-15124979
]
ASF GitHub Bot commented on BOOKKEEPER-879:
---
Github user fpj commented on a diff in the pull request:
https://github.com/apache/bookkeeper/pull/2#discussion_r51347727
--- Diff:
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/AbstractZkLedgerManager.java
---
@@ -355,7 +355,7 @@ public void processResult(int rc, String path, Object
ctx, byte[] data, Stat sta
LedgerMetadata metadata;
try {
-metadata = LedgerMetadata.parseConfig(data, new
ZkVersion(stat.getVersion()));
+metadata = LedgerMetadata.parseConfig(data, new
ZkVersion(stat.getVersion()),stat.getCtime());
--- End diff --
You're not covering all error codes and stat could be null, in which case
it would throw an NPE.
> Record ledger creation time
> ---
>
> Key: BOOKKEEPER-879
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879
> Project: Bookkeeper
> Issue Type: New Feature
> Components: bookkeeper-client, bookkeeper-server
>Affects Versions: 4.3.1
>Reporter: Enrico Olivelli
> Fix For: 4.4.0
>
>
> I think that a creation timestamp would be very useful and it does not cost
> very much. It would be an immutable value.
> Nowadays (on 4.3.1) I must keep that info together with the id of the ledger,
> for instance in zookeeper, but if that reference gets lost there is now way
> to know how old a ledger is.
> I think that this timestamp should be captured on client while calling
> createLedger or asyncCreateLedger.
> In addition to this very common field maybe it would be useful to add a
> custom byte[] field named "custom client data" in order to let the client
> 'describe' the ledger without the need of extra data on Zookeeper.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[
https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15124971#comment-15124971
]
ASF GitHub Bot commented on BOOKKEEPER-879:
---
Github user fpj commented on a diff in the pull request:
https://github.com/apache/bookkeeper/pull/2#discussion_r51347497
--- Diff:
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/MSLedgerManagerFactory.java
---
@@ -408,7 +408,7 @@ public void complete(int rc, Versioned value,
Object ctx) {
LedgerMetadata metadata;
try {
metadata = LedgerMetadata
-
.parseConfig(value.getValue().getField(META_FIELD), value.getVersion());
+
.parseConfig(value.getValue().getField(META_FIELD), value.getVersion(),null);
--- End diff --
Space after comma.
> Record ledger creation time
> ---
>
> Key: BOOKKEEPER-879
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879
> Project: Bookkeeper
> Issue Type: New Feature
> Components: bookkeeper-client, bookkeeper-server
>Affects Versions: 4.3.1
>Reporter: Enrico Olivelli
> Fix For: 4.4.0
>
>
> I think that a creation timestamp would be very useful and it does not cost
> very much. It would be an immutable value.
> Nowadays (on 4.3.1) I must keep that info together with the id of the ledger,
> for instance in zookeeper, but if that reference gets lost there is now way
> to know how old a ledger is.
> I think that this timestamp should be captured on client while calling
> createLedger or asyncCreateLedger.
> In addition to this very common field maybe it would be useful to add a
> custom byte[] field named "custom client data" in order to let the client
> 'describe' the ledger without the need of extra data on Zookeeper.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[
https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15124970#comment-15124970
]
ASF GitHub Bot commented on BOOKKEEPER-879:
---
Github user fpj commented on a diff in the pull request:
https://github.com/apache/bookkeeper/pull/2#discussion_r51347443
--- Diff:
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/AbstractZkLedgerManager.java
---
@@ -355,7 +355,7 @@ public void processResult(int rc, String path, Object
ctx, byte[] data, Stat sta
LedgerMetadata metadata;
try {
-metadata = LedgerMetadata.parseConfig(data, new
ZkVersion(stat.getVersion()));
+metadata = LedgerMetadata.parseConfig(data, new
ZkVersion(stat.getVersion()),stat.getCtime());
--- End diff --
Space after comma.
> Record ledger creation time
> ---
>
> Key: BOOKKEEPER-879
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879
> Project: Bookkeeper
> Issue Type: New Feature
> Components: bookkeeper-client, bookkeeper-server
>Affects Versions: 4.3.1
>Reporter: Enrico Olivelli
> Fix For: 4.4.0
>
>
> I think that a creation timestamp would be very useful and it does not cost
> very much. It would be an immutable value.
> Nowadays (on 4.3.1) I must keep that info together with the id of the ledger,
> for instance in zookeeper, but if that reference gets lost there is now way
> to know how old a ledger is.
> I think that this timestamp should be captured on client while calling
> createLedger or asyncCreateLedger.
> In addition to this very common field maybe it would be useful to add a
> custom byte[] field named "custom client data" in order to let the client
> 'describe' the ledger without the need of extra data on Zookeeper.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[
https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15124969#comment-15124969
]
ASF GitHub Bot commented on BOOKKEEPER-879:
---
Github user fpj commented on a diff in the pull request:
https://github.com/apache/bookkeeper/pull/2#discussion_r51347435
--- Diff:
bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerMetadata.java
---
@@ -351,7 +364,12 @@ public static LedgerMetadata parseConfig(byte[] bytes,
Version version) throws I
LedgerMetadataFormat.Builder builder =
LedgerMetadataFormat.newBuilder();
TextFormat.merge(reader, builder);
LedgerMetadataFormat data = builder.build();
-lc.writeQuorumSize = data.getQuorumSize();
+lc.writeQuorumSize = data.getQuorumSize();
+if (data.hasCtime()) {
+lc.ctime=data.getCtime();
--- End diff --
Space around "=" sign to be consistent across the code.
> Record ledger creation time
> ---
>
> Key: BOOKKEEPER-879
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879
> Project: Bookkeeper
> Issue Type: New Feature
> Components: bookkeeper-client, bookkeeper-server
>Affects Versions: 4.3.1
>Reporter: Enrico Olivelli
> Fix For: 4.4.0
>
>
> I think that a creation timestamp would be very useful and it does not cost
> very much. It would be an immutable value.
> Nowadays (on 4.3.1) I must keep that info together with the id of the ledger,
> for instance in zookeeper, but if that reference gets lost there is now way
> to know how old a ledger is.
> I think that this timestamp should be captured on client while calling
> createLedger or asyncCreateLedger.
> In addition to this very common field maybe it would be useful to add a
> custom byte[] field named "custom client data" in order to let the client
> 'describe' the ledger without the need of extra data on Zookeeper.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[ https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15123448#comment-15123448 ] Enrico Olivelli commented on BOOKKEEPER-879: Is it Ok ? > Record ledger creation time > --- > > Key: BOOKKEEPER-879 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879 > Project: Bookkeeper > Issue Type: New Feature > Components: bookkeeper-client, bookkeeper-server >Affects Versions: 4.3.1 >Reporter: Enrico Olivelli > Fix For: 4.4.0 > > > I think that a creation timestamp would be very useful and it does not cost > very much. It would be an immutable value. > Nowadays (on 4.3.1) I must keep that info together with the id of the ledger, > for instance in zookeeper, but if that reference gets lost there is now way > to know how old a ledger is. > I think that this timestamp should be captured on client while calling > createLedger or asyncCreateLedger. > In addition to this very common field maybe it would be useful to add a > custom byte[] field named "custom client data" in order to let the client > 'describe' the ledger without the need of extra data on Zookeeper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[ https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15105345#comment-15105345 ] Enrico Olivelli commented on BOOKKEEPER-879: I updated the pull request https://github.com/apache/bookkeeper/pull/2 > Record ledger creation time > --- > > Key: BOOKKEEPER-879 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879 > Project: Bookkeeper > Issue Type: New Feature > Components: bookkeeper-client, bookkeeper-server >Affects Versions: 4.3.1 >Reporter: Enrico Olivelli > Fix For: 4.4.0 > > > I think that a creation timestamp would be very useful and it does not cost > very much. It would be an immutable value. > Nowadays (on 4.3.1) I must keep that info together with the id of the ledger, > for instance in zookeeper, but if that reference gets lost there is now way > to know how old a ledger is. > I think that this timestamp should be captured on client while calling > createLedger or asyncCreateLedger. > In addition to this very common field maybe it would be useful to add a > custom byte[] field named "custom client data" in order to let the client > 'describe' the ledger without the need of extra data on Zookeeper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[ https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15098352#comment-15098352 ] Sijie Guo commented on BOOKKEEPER-879: -- [~eolivelli] This option 1 looks reasonable to me. Since this field is the one that used for backward compatibility. And also not all metadata store backends support (ctime for updates). so the signature could be #parseConfig(bytes, version, Optional ctime). > Record ledger creation time > --- > > Key: BOOKKEEPER-879 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879 > Project: Bookkeeper > Issue Type: New Feature > Components: bookkeeper-client, bookkeeper-server >Affects Versions: 4.3.1 >Reporter: Enrico Olivelli > Fix For: 4.4.0 > > > I think that a creation timestamp would be very useful and it does not cost > very much. It would be an immutable value. > Nowadays (on 4.3.1) I must keep that info together with the id of the ledger, > for instance in zookeeper, but if that reference gets lost there is now way > to know how old a ledger is. > I think that this timestamp should be captured on client while calling > createLedger or asyncCreateLedger. > In addition to this very common field maybe it would be useful to add a > custom byte[] field named "custom client data" in order to let the client > 'describe' the ledger without the need of extra data on Zookeeper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[ https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15097788#comment-15097788 ] Enrico Olivelli commented on BOOKKEEPER-879: I have to change LedgerMetadata.parseConfig in order to take the ZK ctime field, I have some options: 1) change the signature and add an explicit "long ctime" 2) change the Version structure (for ZKVersion we will get the ctime from the Stat) the option 2 is not good because ctime is not about versioning, since it is immutable for the option 1 I hit anyway a problem on MSLedgerManagerFactory, can I let it to be 0 for MSLedgerManagerFactory ? > Record ledger creation time > --- > > Key: BOOKKEEPER-879 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879 > Project: Bookkeeper > Issue Type: New Feature > Components: bookkeeper-client, bookkeeper-server >Affects Versions: 4.3.1 >Reporter: Enrico Olivelli > Fix For: 4.4.0 > > > I think that a creation timestamp would be very useful and it does not cost > very much. It would be an immutable value. > Nowadays (on 4.3.1) I must keep that info together with the id of the ledger, > for instance in zookeeper, but if that reference gets lost there is now way > to know how old a ledger is. > I think that this timestamp should be captured on client while calling > createLedger or asyncCreateLedger. > In addition to this very common field maybe it would be useful to add a > custom byte[] field named "custom client data" in order to let the client > 'describe' the ledger without the need of extra data on Zookeeper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[ https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095922#comment-15095922 ] Sijie Guo commented on BOOKKEEPER-879: -- [~eolivelli] I've made the comments in your pull request. The overall change looks good to me. [~mmerli] what is your opinion on this? > Record ledger creation time > --- > > Key: BOOKKEEPER-879 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879 > Project: Bookkeeper > Issue Type: New Feature > Components: bookkeeper-client, bookkeeper-server >Affects Versions: 4.3.1 >Reporter: Enrico Olivelli > Fix For: 4.4.0 > > > I think that a creation timestamp would be very useful and it does not cost > very much. It would be an immutable value. > Nowadays (on 4.3.1) I must keep that info together with the id of the ledger, > for instance in zookeeper, but if that reference gets lost there is now way > to know how old a ledger is. > I think that this timestamp should be captured on client while calling > createLedger or asyncCreateLedger. > In addition to this very common field maybe it would be useful to add a > custom byte[] field named "custom client data" in order to let the client > 'describe' the ledger without the need of extra data on Zookeeper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[ https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15053544#comment-15053544 ] Sijie Guo commented on BOOKKEEPER-879: -- Let's focus on ctime right now. I could see ctime is useful in lots of use cases. > Record ledger creation time > --- > > Key: BOOKKEEPER-879 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879 > Project: Bookkeeper > Issue Type: New Feature > Components: bookkeeper-client, bookkeeper-server >Affects Versions: 4.3.1 >Reporter: Enrico Olivelli > > I think that a creation timestamp would be very useful and it does not cost > very much. It would be an immutable value. > Nowadays (on 4.3.1) I must keep that info together with the id of the ledger, > for instance in zookeeper, but if that reference gets lost there is now way > to know how old a ledger is. > I think that this timestamp should be captured on client while calling > createLedger or asyncCreateLedger. > In addition to this very common field maybe it would be useful to add a > custom byte[] field named "custom client data" in order to let the client > 'describe' the ledger without the need of extra data on Zookeeper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[ https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15052581#comment-15052581 ] Enrico Olivelli commented on BOOKKEEPER-879: if we keep mtime inside LedgerMetadata, so not reading from znode, it has to be updated every time that LedgerMetadata change, I don't know Bookkeeper internals very much, but I think that it is a non trivial implementation because we need to decide which mutation changes the mtime. Otherwise we can update it at every update on the znode (before calling writeLedgerMetadata or inside writeLedgerMetadata?) but the value that we are going to write will be different from the value that the caller is requesting to be written. I don't known how to handle isConflictWith for changes on mtime honestly I would prefer to leave mtime to another issue and implement only ctime either as serialized LedgerMetadata field or as derived LedgerMetadata field (read from znode and passed to the parseConfig method) I we use the znode-only approach new clients will be able to known ctime for old ledgers and this feature for me will be very valuable (because I have in production thousands of ledgers) > Record ledger creation time > --- > > Key: BOOKKEEPER-879 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879 > Project: Bookkeeper > Issue Type: New Feature > Components: bookkeeper-client, bookkeeper-server >Affects Versions: 4.3.1 >Reporter: Enrico Olivelli > > I think that a creation timestamp would be very useful and it does not cost > very much. It would be an immutable value. > Nowadays (on 4.3.1) I must keep that info together with the id of the ledger, > for instance in zookeeper, but if that reference gets lost there is now way > to know how old a ledger is. > I think that this timestamp should be captured on client while calling > createLedger or asyncCreateLedger. > In addition to this very common field maybe it would be useful to add a > custom byte[] field named "custom client data" in order to let the client > 'describe' the ledger without the need of extra data on Zookeeper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[ https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15052582#comment-15052582 ] Enrico Olivelli commented on BOOKKEEPER-879: if we keep mtime inside LedgerMetadata, so not reading from znode, it has to be updated every time that LedgerMetadata change, I don't know Bookkeeper internals very much, but I think that it is a non trivial implementation because we need to decide which mutation changes the mtime. Otherwise we can update it at every update on the znode (before calling writeLedgerMetadata or inside writeLedgerMetadata?) but the value that we are going to write will be different from the value that the caller is requesting to be written. I don't known how to handle isConflictWith for changes on mtime honestly I would prefer to leave mtime to another issue and implement only ctime either as serialized LedgerMetadata field or as derived LedgerMetadata field (read from znode and passed to the parseConfig method) I we use the znode-only approach new clients will be able to known ctime for old ledgers and this feature for me will be very valuable (because I have in production thousands of ledgers) > Record ledger creation time > --- > > Key: BOOKKEEPER-879 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879 > Project: Bookkeeper > Issue Type: New Feature > Components: bookkeeper-client, bookkeeper-server >Affects Versions: 4.3.1 >Reporter: Enrico Olivelli > > I think that a creation timestamp would be very useful and it does not cost > very much. It would be an immutable value. > Nowadays (on 4.3.1) I must keep that info together with the id of the ledger, > for instance in zookeeper, but if that reference gets lost there is now way > to know how old a ledger is. > I think that this timestamp should be captured on client while calling > createLedger or asyncCreateLedger. > In addition to this very common field maybe it would be useful to add a > custom byte[] field named "custom client data" in order to let the client > 'describe' the ledger without the need of extra data on Zookeeper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[ https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15035431#comment-15035431 ] Sijie Guo commented on BOOKKEEPER-879: -- it would be not tight with the zookeeper details on exposing metadata like ctime, mtime. so I think it makes sense to have such information in the ledger metadata, but we just need to take care of backward compatability as what [~merlimat] suggested: 1) probably fallback to use zookeeper znode ctime if this field is missing. 2) make sure we test backward compatability on TestCompact test cases, make sure new client could read old metadata and old client could read new metadata too. > Record ledger creation time > --- > > Key: BOOKKEEPER-879 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879 > Project: Bookkeeper > Issue Type: New Feature > Components: bookkeeper-client, bookkeeper-server >Affects Versions: 4.3.1 >Reporter: Enrico Olivelli > > I think that a creation timestamp would be very useful and it does not cost > very much. It would be an immutable value. > Nowadays (on 4.3.1) I must keep that info together with the id of the ledger, > for instance in zookeeper, but if that reference gets lost there is now way > to know how old a ledger is. > I think that this timestamp should be captured on client while calling > createLedger or asyncCreateLedger. > In addition to this very common field maybe it would be useful to add a > custom byte[] field named "custom client data" in order to let the client > 'describe' the ledger without the need of extra data on Zookeeper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[
https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15031636#comment-15031636
]
Enrico Olivelli commented on BOOKKEEPER-879:
Maybe be it would be BookKeeperAdmin#describeLedger or
Bookkeeper#describeLedger or LedgerHandle#describeLedger
LedgerInfo describeLedger(long lId) throws InterruptedException, BKException
class LedgerInfo {
long ledgerId;
long ctime;
long mtime;
}
> Record ledger creation time
> ---
>
> Key: BOOKKEEPER-879
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879
> Project: Bookkeeper
> Issue Type: New Feature
> Components: bookkeeper-client, bookkeeper-server
>Affects Versions: 4.3.1
>Reporter: Enrico Olivelli
>
> I think that a creation timestamp would be very useful and it does not cost
> very much. It would be an immutable value.
> Nowadays (on 4.3.1) I must keep that info together with the id of the ledger,
> for instance in zookeeper, but if that reference gets lost there is now way
> to know how old a ledger is.
> I think that this timestamp should be captured on client while calling
> createLedger or asyncCreateLedger.
> In addition to this very common field maybe it would be useful to add a
> custom byte[] field named "custom client data" in order to let the client
> 'describe' the ledger without the need of extra data on Zookeeper.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[ https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028540#comment-15028540 ] Enrico Olivelli commented on BOOKKEEPER-879: I agree that the ZK backup/restore as I describe is not feasible. If we are going to use the ZK metadata (z-node ctime/mtime) maybe it would be better not to include this info in LedgerMetadata but to add a new data structure like LedgerInfo (maybe a better name). ctime and mtime are only for descriptive purposes and are not useful to Bookkeeper Internals. We can simply add a new method to BookKeeperAdmin, in this case the LedgerHandle and other internals will not be touched by this new feature, and there will be not impact on compatibility of older clients > Record ledger creation time > --- > > Key: BOOKKEEPER-879 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879 > Project: Bookkeeper > Issue Type: New Feature > Components: bookkeeper-client, bookkeeper-server >Affects Versions: 4.3.1 >Reporter: Enrico Olivelli > > I think that a creation timestamp would be very useful and it does not cost > very much. It would be an immutable value. > Nowadays (on 4.3.1) I must keep that info together with the id of the ledger, > for instance in zookeeper, but if that reference gets lost there is now way > to know how old a ledger is. > I think that this timestamp should be captured on client while calling > createLedger or asyncCreateLedger. > In addition to this very common field maybe it would be useful to add a > custom byte[] field named "custom client data" in order to let the client > 'describe' the ledger without the need of extra data on Zookeeper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[ https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027717#comment-15027717 ] Matteo Merli commented on BOOKKEEPER-879: - I have a couple of concerns on the pull request: # If we record "ctime", it might be useful to expose other informations that can be currently get from the z-node stats, like the last modification timestamp "mtime" and how many changes there were on the ledger metadata (z-node version) # When this feature is introduced, older ledger will not have the "ctime" field. What should I expect to get in that case? How would I know whether a ledger was created before or after the feature was rolled out? (This can also be at different times for different hosts) Can we fallback to z-node information if the field is not there? # We need to ensure that older clients are able to de-serialize the message metadata correctly (In the past I've seen errors with protocol buffer text format when new field are introduced) Lastly, I don't think the ZK backup/restore is a very compelling reason to add this feature because: # you cannot backup z-node one by one because that will lead to inconsistencies # to backup your ZK data, you simply need to copy the last ZK snapshot and transaction log # to restore from snaphot, you need to shutdown your ZK quorum, copy the snapshot and txn log to all participants and restart the quorum # after the restore, all z-nodes will have the correct ctime, mtime, etc. :) > Record ledger creation time > --- > > Key: BOOKKEEPER-879 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879 > Project: Bookkeeper > Issue Type: New Feature > Components: bookkeeper-client, bookkeeper-server >Affects Versions: 4.3.1 >Reporter: Enrico Olivelli > > I think that a creation timestamp would be very useful and it does not cost > very much. It would be an immutable value. > Nowadays (on 4.3.1) I must keep that info together with the id of the ledger, > for instance in zookeeper, but if that reference gets lost there is now way > to know how old a ledger is. > I think that this timestamp should be captured on client while calling > createLedger or asyncCreateLedger. > In addition to this very common field maybe it would be useful to add a > custom byte[] field named "custom client data" in order to let the client > 'describe' the ledger without the need of extra data on Zookeeper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[ https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027018#comment-15027018 ] ASF GitHub Bot commented on BOOKKEEPER-879: --- GitHub user eolivelli opened a pull request: https://github.com/apache/bookkeeper/pull/2 BOOKKEEPER-879: Record ledger creation time You can merge this pull request into a Git repository by running: $ git pull https://github.com/eolivelli/bookkeeper master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/bookkeeper/pull/2.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2 commit e33f2e01973f4d82dff8fad3a1d96e9037f4bdbb Author: eolivelli Date: 2015-11-25T16:07:13Z BOOKKEEPER-879: Record ledger creation time commit c43f0b4e0a400ac87acbc278055e582177812e2d Author: eolivelli Date: 2015-11-25T16:15:25Z BOOKKEEPER-879: Record ledger creation time > Record ledger creation time > --- > > Key: BOOKKEEPER-879 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879 > Project: Bookkeeper > Issue Type: New Feature > Components: bookkeeper-client, bookkeeper-server >Affects Versions: 4.3.1 >Reporter: Enrico Olivelli > > I think that a creation timestamp would be very useful and it does not cost > very much. It would be an immutable value. > Nowadays (on 4.3.1) I must keep that info together with the id of the ledger, > for instance in zookeeper, but if that reference gets lost there is now way > to know how old a ledger is. > I think that this timestamp should be captured on client while calling > createLedger or asyncCreateLedger. > In addition to this very common field maybe it would be useful to add a > custom byte[] field named "custom client data" in order to let the client > 'describe' the ledger without the need of extra data on Zookeeper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[ https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15026593#comment-15026593 ] Enrico Olivelli commented on BOOKKEEPER-879: I think that using z-node ctime in the future can break some kind of backup/restore status from zookeeper. I would like to save all the data of the ledger on the z-nde content > Record ledger creation time > --- > > Key: BOOKKEEPER-879 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879 > Project: Bookkeeper > Issue Type: New Feature > Components: bookkeeper-client, bookkeeper-server >Affects Versions: 4.3.1 >Reporter: Enrico Olivelli > > I think that a creation timestamp would be very useful and it does not cost > very much. It would be an immutable value. > Nowadays (on 4.3.1) I must keep that info together with the id of the ledger, > for instance in zookeeper, but if that reference gets lost there is now way > to know how old a ledger is. > I think that this timestamp should be captured on client while calling > createLedger or asyncCreateLedger. > In addition to this very common field maybe it would be useful to add a > custom byte[] field named "custom client data" in order to let the client > 'describe' the ledger without the need of extra data on Zookeeper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[ https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15026590#comment-15026590 ] Enrico Olivelli commented on BOOKKEEPER-879: Sorry. I was referring to org.apache.bookkeeper.client.LedgerMetadata#serialize which uses s.toString().getBytes() in branch-4.0. I checked out master and I found LedgerMetadataFormat > Record ledger creation time > --- > > Key: BOOKKEEPER-879 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879 > Project: Bookkeeper > Issue Type: New Feature > Components: bookkeeper-client, bookkeeper-server >Affects Versions: 4.3.1 >Reporter: Enrico Olivelli > > I think that a creation timestamp would be very useful and it does not cost > very much. It would be an immutable value. > Nowadays (on 4.3.1) I must keep that info together with the id of the ledger, > for instance in zookeeper, but if that reference gets lost there is now way > to know how old a ledger is. > I think that this timestamp should be captured on client while calling > createLedger or asyncCreateLedger. > In addition to this very common field maybe it would be useful to add a > custom byte[] field named "custom client data" in order to let the client > 'describe' the ledger without the need of extra data on Zookeeper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[ https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15025620#comment-15025620 ] Matteo Merli commented on BOOKKEEPER-879: - Ledger z-node already has a ctime that can be checked from outside and eventually exposed from API. > Record ledger creation time > --- > > Key: BOOKKEEPER-879 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879 > Project: Bookkeeper > Issue Type: New Feature > Components: bookkeeper-client, bookkeeper-server >Affects Versions: 4.3.1 >Reporter: Enrico Olivelli > > I think that a creation timestamp would be very useful and it does not cost > very much. It would be an immutable value. > Nowadays (on 4.3.1) I must keep that info together with the id of the ledger, > for instance in zookeeper, but if that reference gets lost there is now way > to know how old a ledger is. > I think that this timestamp should be captured on client while calling > createLedger or asyncCreateLedger. > In addition to this very common field maybe it would be useful to add a > custom byte[] field named "custom client data" in order to let the client > 'describe' the ledger without the need of extra data on Zookeeper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[ https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15024930#comment-15024930 ] Venkateswararao Jujjuri commented on BOOKKEEPER-879: +1 for ctime. :) > Record ledger creation time > --- > > Key: BOOKKEEPER-879 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879 > Project: Bookkeeper > Issue Type: New Feature > Components: bookkeeper-client, bookkeeper-server >Affects Versions: 4.3.1 >Reporter: Enrico Olivelli > > I think that a creation timestamp would be very useful and it does not cost > very much. It would be an immutable value. > Nowadays (on 4.3.1) I must keep that info together with the id of the ledger, > for instance in zookeeper, but if that reference gets lost there is now way > to know how old a ledger is. > I think that this timestamp should be captured on client while calling > createLedger or asyncCreateLedger. > In addition to this very common field maybe it would be useful to add a > custom byte[] field named "custom client data" in order to let the client > 'describe' the ledger without the need of extra data on Zookeeper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[ https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15024918#comment-15024918 ] Sijie Guo commented on BOOKKEEPER-879: -- [~eolivelli] that's cool. re 1): maybe follow the filesystem standard, calling it 'ctime'. re 2): you could extend metadata by adding field in LedgerMetadataFormat, which is protobuf based format. check bookkeeper-server/src/main/proto/DataFormats.proto re 3): the encoding is done by UTF_8, no? "new String(.., UTF_8)" and "getBytes(UTF_8)". Do you know which line doesn't do this? If no, we should fix it. feel free to create a ticket for that. > Record ledger creation time > --- > > Key: BOOKKEEPER-879 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879 > Project: Bookkeeper > Issue Type: New Feature > Components: bookkeeper-client, bookkeeper-server >Affects Versions: 4.3.1 >Reporter: Enrico Olivelli > > I think that a creation timestamp would be very useful and it does not cost > very much. It would be an immutable value. > Nowadays (on 4.3.1) I must keep that info together with the id of the ledger, > for instance in zookeeper, but if that reference gets lost there is now way > to know how old a ledger is. > I think that this timestamp should be captured on client while calling > createLedger or asyncCreateLedger. > In addition to this very common field maybe it would be useful to add a > custom byte[] field named "custom client data" in order to let the client > 'describe' the ledger without the need of extra data on Zookeeper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-879) Record ledger creation time
[ https://issues.apache.org/jira/browse/BOOKKEEPER-879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15024739#comment-15024739 ] Enrico Olivelli commented on BOOKKEEPER-879: I would like to start working on this issue, I have some question: 1) which is the best name for this new field ? maybe "creationTimestamp" 2) I see that LedgerMetadata serialized format is very difficult to extend, the ensemble list ends with a 'closed' line, should I create a new METADATA_VERSION = 2 ? or use some trick like adding a "timestamp:" line (using a prefix ?) ? I don't know how to preserve compatibility for rolling upgrades 3) I see that LedgerMetadata conversion from String to byte[] is done using the system encoding, sould I file another issue for this problem ? actually data does not contain locale sensitive info (usually addresses use only latin characters) > Record ledger creation time > --- > > Key: BOOKKEEPER-879 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-879 > Project: Bookkeeper > Issue Type: New Feature > Components: bookkeeper-client, bookkeeper-server >Affects Versions: 4.3.1 >Reporter: Enrico Olivelli > > I think that a creation timestamp would be very useful and it does not cost > very much. It would be an immutable value. > Nowadays (on 4.3.1) I must keep that info together with the id of the ledger, > for instance in zookeeper, but if that reference gets lost there is now way > to know how old a ledger is. > I think that this timestamp should be captured on client while calling > createLedger or asyncCreateLedger. > In addition to this very common field maybe it would be useful to add a > custom byte[] field named "custom client data" in order to let the client > 'describe' the ledger without the need of extra data on Zookeeper. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
