Re: [PR] KAFKA-14412: Better Rocks column family management [kafka]

2023-12-05 Thread via GitHub
cadonna merged PR #14852: URL: https://github.com/apache/kafka/pull/14852 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

Re: [PR] KAFKA-14412: Better Rocks column family management [kafka]

2023-12-04 Thread via GitHub
nicktelford commented on PR #14852: URL: https://github.com/apache/kafka/pull/14852#issuecomment-1838948422 I've rebased against trunk so this should be good to go. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [PR] KAFKA-14412: Better Rocks column family management [kafka]

2023-12-04 Thread via GitHub
nicktelford commented on code in PR #14852: URL: https://github.com/apache/kafka/pull/14852#discussion_r1413700543 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBTimestampedStore.java: ## @@ -64,31 +62,14 @@ public RocksDBTimestampedStore(final String n

Re: [PR] KAFKA-14412: Better Rocks column family management [kafka]

2023-12-03 Thread via GitHub
cadonna commented on code in PR #14852: URL: https://github.com/apache/kafka/pull/14852#discussion_r1413468864 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBTimestampedStore.java: ## @@ -64,31 +62,14 @@ public RocksDBTimestampedStore(final String name,

Re: [PR] KAFKA-14412: Better Rocks column family management [kafka]

2023-12-03 Thread via GitHub
nicktelford commented on code in PR #14852: URL: https://github.com/apache/kafka/pull/14852#discussion_r1413187089 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBTimestampedStore.java: ## @@ -64,31 +62,14 @@ public RocksDBTimestampedStore(final String n

Re: [PR] KAFKA-14412: Better Rocks column family management [kafka]

2023-12-03 Thread via GitHub
nicktelford commented on code in PR #14852: URL: https://github.com/apache/kafka/pull/14852#discussion_r1413186918 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBTimestampedStore.java: ## @@ -64,31 +62,14 @@ public RocksDBTimestampedStore(final String n

Re: [PR] KAFKA-14412: Better Rocks column family management [kafka]

2023-12-01 Thread via GitHub
ableegoldman commented on code in PR #14852: URL: https://github.com/apache/kafka/pull/14852#discussion_r1412625873 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBTimestampedStore.java: ## @@ -64,31 +62,14 @@ public RocksDBTimestampedStore(final String

Re: [PR] KAFKA-14412: Better Rocks column family management [kafka]

2023-12-01 Thread via GitHub
nicktelford commented on PR #14852: URL: https://github.com/apache/kafka/pull/14852#issuecomment-1836167995 Sorry about the compile error. It should now build. Regarding tests: agreed. When KIP-892 lands, it adds an extra CF, but at that point RocksDBStoreTest and RocksDBTimestampedSt

Re: [PR] KAFKA-14412: Better Rocks column family management [kafka]

2023-12-01 Thread via GitHub
nicktelford commented on code in PR #14852: URL: https://github.com/apache/kafka/pull/14852#discussion_r1412137956 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBTimestampedStore.java: ## @@ -64,45 +62,27 @@ public RocksDBTimestampedStore(final String n

Re: [PR] KAFKA-14412: Better Rocks column family management [kafka]

2023-12-01 Thread via GitHub
cadonna commented on code in PR #14852: URL: https://github.com/apache/kafka/pull/14852#discussion_r1412123554 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBTimestampedStore.java: ## @@ -64,45 +62,27 @@ public RocksDBTimestampedStore(final String name,

Re: [PR] KAFKA-14412: Better Rocks column family management [kafka]

2023-12-01 Thread via GitHub
nicktelford commented on PR #14852: URL: https://github.com/apache/kafka/pull/14852#issuecomment-1836136670 @cadonna > Do we have unit tests in place that test all this logic? We don't have any tests dedicated to this, but `RocksDBStoreTest` and `RocksDBTimestampedStoreTest` impli

Re: [PR] KAFKA-14412: Better Rocks column family management [kafka]

2023-12-01 Thread via GitHub
cadonna commented on code in PR #14852: URL: https://github.com/apache/kafka/pull/14852#discussion_r1412057240 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java: ## @@ -278,18 +280,74 @@ private void addValueProvidersToMetricsRecorder() {

Re: [PR] KAFKA-14412: Better Rocks column family management [kafka]

2023-12-01 Thread via GitHub
nicktelford commented on PR #14852: URL: https://github.com/apache/kafka/pull/14852#issuecomment-1836027702 @mjsax @ableegoldman @lucasbru @wcarlson5 @bbejeck @vvcephei @guozhangwang I was recommended to tag you all in these Kafka Streams PRs. -- This is an automated message from the A

Re: [PR] KAFKA-14412: Better Rocks column family management [kafka]

2023-11-30 Thread via GitHub
nicktelford commented on code in PR #14852: URL: https://github.com/apache/kafka/pull/14852#discussion_r1410543947 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java: ## @@ -278,13 +280,55 @@ private void addValueProvidersToMetricsRecorder() {

Re: [PR] KAFKA-14412: Better Rocks column family management [kafka]

2023-11-30 Thread via GitHub
nicktelford commented on code in PR #14852: URL: https://github.com/apache/kafka/pull/14852#discussion_r1410509927 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java: ## @@ -278,13 +280,55 @@ private void addValueProvidersToMetricsRecorder() {

Re: [PR] KAFKA-14412: Better Rocks column family management [kafka]

2023-11-30 Thread via GitHub
cadonna commented on code in PR #14852: URL: https://github.com/apache/kafka/pull/14852#discussion_r1410391609 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java: ## @@ -278,13 +280,55 @@ private void addValueProvidersToMetricsRecorder() {

Re: [PR] KAFKA-14412: Better Rocks column family management [kafka]

2023-11-28 Thread via GitHub
nicktelford commented on PR #14852: URL: https://github.com/apache/kafka/pull/14852#issuecomment-1829817068 @cadonna -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] KAFKA-14412: Better Rocks column family management [kafka]

2023-11-28 Thread via GitHub
nicktelford commented on PR #14852: URL: https://github.com/apache/kafka/pull/14852#issuecomment-1829816811 This is part of KIP-892, and has been broken out into a separate PR to reduce the review burden on the main KIP-892 implementation, since it can be merged independently. Note:

[PR] KAFKA-14412: Better Rocks column family management [kafka]

2023-11-28 Thread via GitHub
nicktelford opened a new pull request, #14852: URL: https://github.com/apache/kafka/pull/14852 When opening RocksDB, we were checking for an error in `RocksDBTimestampedStore` to detect if the `keyValueWithTimestamp` CF is missing. The `openRocksDB` method now supports any number of