mjsax commented on PR #20969:
URL: https://github.com/apache/kafka/pull/20969#issuecomment-3838790084
Thanks all.
From ChickenchickenLove:
Thanks for clarifying. For tickets with multiple propose solutions, it's
always best to first discuss. It's of course great if you have a concrete
reason why you would pick a solution over the others -- just best to clearly
communicate it upfront.
> but it would also cause a new processId on the next start (potentially
increasing task shuffling after restart)
That's a fair point I guess. Not sure how large the impact would be, but
it's for sure a risk.
From Bill:
> I like option 3, but I wonder if that could leave users not well versed on
why we store the process.id and cause issues down the road.
That's a good point Bill. Keeping the API simple might be best (if we can).
> I was thinking of possibly another option: we store the process.id in a
different location, the directory where the Kafka Streams application is
running. But that may raise more issues than it solves. WDYT?
Yeah, not sure. In the end, the idea to use directory structure
`/<state.dir>/<application.id>/` is to have everything in one place. We could
of course create a file inside `/<state.dir>/` with some encoding, but it seems
to not align to the overall design.
I also just checked the JavaDocs of `KafkaStreams#cleanUp()`:
```
Do a cleanup of the local {@link StateStore} directory ({@link
StreamsConfig#STATE_DIR_CONFIG}) by deleting all
data with regard to the {@link StreamsConfig#APPLICATION_ID_CONFIG
application ID}.
```
Technically it says "delete all data" with would include the process-id file
-- so strictly speaking it's a bug? -- But I do see the argument why we want to
keep the file and not delete it: this would be technically an API contract
change and might require a KIP (even if it might feel a little bit heavy).
So I am still torn a little bit, but overall I tend to think: let's do a
small KIP to document the change, and change the contract to _keep_ the
metadata file? This might be the cleanest solution?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
