[
https://issues.apache.org/jira/browse/GOBBLIN-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhixiong Chen updated GOBBLIN-1012:
-----------------------------------
Description:
- A `compactionWatermark` is a timestamp indicating the data we've seen up to
that time is compacted. If we explicitly checked that we've seen all the data
up to that time, the timestamp would be published also as
`completionAndCompactionWatermark`.
- `CompactionWithWatermarkSuite` reports compaction watermarks as part of the
compaction pipeline and publishes watermarks as hive table properties
was:`CompactionWithWatermarkSuite` will report compaction watermark as part
of the compaction pipeline.
> Implement CompactionWithWatermarkSuite
> --------------------------------------
>
> Key: GOBBLIN-1012
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1012
> Project: Apache Gobblin
> Issue Type: Task
> Reporter: Zhixiong Chen
> Assignee: Zhixiong Chen
> Priority: Major
>
> - A `compactionWatermark` is a timestamp indicating the data we've seen up to
> that time is compacted. If we explicitly checked that we've seen all the data
> up to that time, the timestamp would be published also as
> `completionAndCompactionWatermark`.
> - `CompactionWithWatermarkSuite` reports compaction watermarks as part of
> the compaction pipeline and publishes watermarks as hive table properties
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
