bbejeck commented on code in PR #21739:
URL: https://github.com/apache/kafka/pull/21739#discussion_r2933739884
##########
streams/src/main/java/org/apache/kafka/streams/processor/internals/GlobalStateManagerImpl.java:
##########
@@ -169,41 +168,40 @@ public void setGlobalProcessorContext(final
InternalProcessorContext<?, ?> globa
@Override
public Set<String> initialize() {
- try {
- checkpointFileCache.putAll(checkpointFile.read());
- } catch (final IOException e) {
- throw new StreamsException("Failed to read checkpoints for global
state globalStores", e);
- }
-
droppedRecordsSensor = droppedRecordsSensor(
Thread.currentThread().getName(),
globalProcessorContext.taskId().toString(),
globalProcessorContext.metrics()
);
- final Set<String> changelogTopics = new HashSet<>();
+ final Map<TopicPartition, StateStore> wrappedStores = new HashMap<>();
for (final StateStore stateStore : topology.globalStateStores()) {
- final String sourceTopic =
storeToChangelogTopic.get(stateStore.name());
- changelogTopics.add(sourceTopic);
- stateStore.init(globalProcessorContext, stateStore);
- }
+ final List<TopicPartition> storePartitions =
topicPartitionsForStore(stateStore);
+ final StateStore maybeWrappedStore =
LegacyCheckpointingStateStore.maybeWrapStore(
+ stateStore, eosEnabled, new HashSet<>(storePartitions),
stateDirectory, null, logPrefix);
+ maybeWrappedStore.init(globalProcessorContext, maybeWrappedStore);
- // make sure each topic-partition from checkpointFileCache is
associated with a global state store
- checkpointFileCache.keySet().forEach(tp -> {
Review Comment:
We removed this check an it makes sense since we're moving away from
checkpoint files, with the new approach I'm thinking there's no way for this to
get out sync now?
##########
streams/src/test/java/org/apache/kafka/streams/processor/internals/GlobalStateManagerImplTest.java:
##########
@@ -175,83 +176,73 @@ public void before() {
@Test
public void shouldReadCheckpointOffsets() throws IOException {
- final Map<TopicPartition, Long> expected = writeCheckpoint();
+ writeCheckpoint();
Review Comment:
nit: Is there a test for full migration path? i.e. `legacy `.checkpoint`
exists → stores get migrated offsets → legacy file deleted` might be worth
adding
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
