bvaradar commented on a change in pull request #600: Timeline Service with
Incremental View Syncing support
URL: https://github.com/apache/incubator-hudi/pull/600#discussion_r276430291
##########
File path:
hoodie-common/src/main/java/com/uber/hoodie/common/table/timeline/HoodieDefaultTimeline.java
##########
@@ -36,13 +41,27 @@
private static final transient Logger log =
LogManager.getLogger(HoodieDefaultTimeline.class);
- protected Function<HoodieInstant, Optional<byte[]>> details;
- protected List<HoodieInstant> instants;
+ protected transient Function<HoodieInstant, Optional<byte[]>> details;
+ private List<HoodieInstant> instants;
+ private String timelineHash;
public HoodieDefaultTimeline(Stream<HoodieInstant> instants,
Function<HoodieInstant, Optional<byte[]>> details) {
- this.instants = instants.collect(Collectors.toList());
this.details = details;
+ setInstants(instants.collect(Collectors.toList()));
+ }
+
+ public void setInstants(List<HoodieInstant> instants) {
+ this.instants = instants;
+ final MessageDigest md;
+ try {
+ md = MessageDigest.getInstance("SHA-256");
+ this.instants.stream().forEach(i -> md.update(
+ new String(i.getTimestamp() + "_" + i.getAction() + "_" +
i.getState()).getBytes()));
+ } catch (NoSuchAlgorithmException nse) {
+ throw new HoodieException(nse);
+ }
+ this.timelineHash = new String(Hex.encodeHex(md.digest()));
Review comment:
Yes, it includes pending compactions. A new or completed compaction instant
does affect file-system view and we would want the timeline hash to reflect
that.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
