[
https://issues.apache.org/jira/browse/KAFKA-8315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16832577#comment-16832577
]
Andrew edited comment on KAFKA-8315 at 5/3/19 6:54 PM:
-------------------------------------------------------
[~vvcephei] Update I ran using grace = 20D and I see records from around 23D
prior to the latest data in the topics. So it seems that we definitely can join
the records, but that the grace period is the critical factor in doing this for
a historical ingest.
So, for a 2 year ingestion do I have to set my grace to 2 years? Seems a bit
strange, and maybe sounds a bit inefficient?
| |
| |
was (Author: the4thamigo_uk):
[~vvcephei] Update I ran using grace = 20D and I see records from around 23D
prior to the latest data in the topics. So it seems that we definitely can join
the records, but that the grace period is the critical factor in doing this for
a historical ingest.
So, for a 2 year ingestion do I have to set my grace to 2 years? Seems a bit
strange, and maybe sounds a bit inefficient?
-For reference, I have uploaded the rowkey and timestamps of the kind of
records we have in our streams : [https://github.com/the4thamigo-uk/join_data
]-[
|https://github.com/the4thamigo-uk/join_data]I think the data I put here is
incorrect let me fix.[
|https://github.com/the4thamigo-uk/join_data]
> Cannot pass Materialized into a join operation - hence cant set retention
> period independent of grace
> -----------------------------------------------------------------------------------------------------
>
> Key: KAFKA-8315
> URL: https://issues.apache.org/jira/browse/KAFKA-8315
> Project: Kafka
> Issue Type: Bug
> Reporter: Andrew
> Assignee: John Roesler
> Priority: Major
>
> The documentation says to use `Materialized` not `JoinWindows.until()`
> ([https://kafka.apache.org/22/javadoc/org/apache/kafka/streams/kstream/JoinWindows.html#until-long-]),
> but there is no where to pass a `Materialized` instance to the join
> operation, only to the group operation is supported it seems.
>
> Slack conversation here :
> [https://confluentcommunity.slack.com/archives/C48AHTCUQ/p1556799561287300]
> [Additional]
> From what I understand, the retention period should be independent of the
> grace period, so I think this is more than a documentation fix (see comments
> below)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
