[GitHub] [kafka] satishd commented on pull request #11058: KAFKA-12802 Added a file based cache for consumed remote log metadata for each partition to avoid consuming again incase of broker restarts.
satishd commented on pull request #11058: URL: https://github.com/apache/kafka/pull/11058#issuecomment-913321617 @junrao gentle reminder to review the changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] satishd commented on pull request #11033: KAFKA-12988 Asynchronous API support for RemoteLogMetadataManager add/update methods.
satishd commented on pull request #11033: URL: https://github.com/apache/kafka/pull/11033#issuecomment-913321448 Thanks @junrao for your comment. Addressed the review comment with the latest commit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] satishd commented on a change in pull request #11033: KAFKA-12988 Asynchronous API support for RemoteLogMetadataManager add/update methods.
satishd commented on a change in pull request #11033:
URL: https://github.com/apache/kafka/pull/11033#discussion_r702548933
##
File path:
storage/api/src/main/java/org/apache/kafka/server/log/remote/storage/RemoteLogMetadataManager.java
##
@@ -62,16 +63,17 @@
* @param remoteLogSegmentMetadata metadata about the remote log segment.
* @throws RemoteStorageException if there are any storage related
errors occurred.
* @throws IllegalArgumentException if the given metadata instance does
not have the state as {@link RemoteLogSegmentState#COPY_SEGMENT_STARTED}
+ * @return a Future which will complete once this operation is finished.
*/
-void addRemoteLogSegmentMetadata(RemoteLogSegmentMetadata
remoteLogSegmentMetadata) throws RemoteStorageException;
+Future addRemoteLogSegmentMetadata(RemoteLogSegmentMetadata
remoteLogSegmentMetadata) throws RemoteStorageException;
Review comment:
Updated the PR with the suggested change of having `CompletableFuture`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] jasonyanwenl commented on pull request #11297: KAFKA-10038: Supports default client.id for ConsoleConsumer, ProducerPerformance, ConsumerPerformance
jasonyanwenl commented on pull request #11297: URL: https://github.com/apache/kafka/pull/11297#issuecomment-913314815 Hi @guozhangwang, I saw you are assigned as the reviewer of the ticket [KAFKA-10038](https://issues.apache.org/jira/browse/KAFKA-10038), please help take a look at this PR when you get a chance. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KAFKA-5666) Need feedback to user if consumption fails due to offsets.topic.replication.factor=3
[
https://issues.apache.org/jira/browse/KAFKA-5666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17410351#comment-17410351
]
Yanwen Lin commented on KAFKA-5666:
---
Hi [~yevabyzek], [~apurva], is this still an issue? I saw that for development,
we will start broker using the following command per this
[doc|https://kafka.apache.org/quickstart]:
{code:java}
# Start the Kafka broker service
$ bin/kafka-server-start.sh config/server.properties
{code}
And in *server.properties*, it already set the prop
offsets.topic.replication.factor as 1 (link:
[https://github.com/apache/kafka/blob/99b9b3e84f4e98c3f07714e1de6a139a004cbc5b/config/server.properties#L74])
so the issue described in this ticket should not be a problem? Anything I
misunderstood?
> Need feedback to user if consumption fails due to
> offsets.topic.replication.factor=3
>
>
> Key: KAFKA-5666
> URL: https://issues.apache.org/jira/browse/KAFKA-5666
> Project: Kafka
> Issue Type: Improvement
> Components: clients, consumer
>Affects Versions: 0.11.0.0
>Reporter: Yeva Byzek
>Priority: Major
> Labels: newbie, usability
>
> Introduced in 0.11: The offsets.topic.replication.factor broker config is now
> enforced upon auto topic creation. Internal auto topic creation will fail
> with a GROUP_COORDINATOR_NOT_AVAILABLE error until the cluster size meets
> this replication factor requirement.
> Issue: Default is setting offsets.topic.replication.factor=3, but in
> development and docker environments where there is only 1 broker, the offsets
> topic will fail to be created when a consumer tries to consume and no records
> will be returned. As a result, the user experience is bad. The user may
> have no idea about this setting change and enforcement, and they just see
> that `kafka-console-consumer` hangs with ZERO output. It is true that the
> broker log file will provide a message (e.g. {{ERROR [KafkaApi-1] Number of
> alive brokers '1' does not meet the required replication factor '3' for the
> offsets topic (configured via 'offsets.topic.replication.factor'). This error
> can be ignored if the cluster is starting up and not all brokers are up yet.
> (kafka.server.KafkaApis)}}) but many users do not have access to the log
> files or know how to get them.
> Suggestion: give feedback to the user/app if offsets topic cannot be created.
> For example, after some timeout.
> Workaround:
> Set offsets.topic.replication.factor=3
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [kafka] ijuma commented on a change in pull request #11296: KAFKA-13273: Add support for Java 17
ijuma commented on a change in pull request #11296:
URL: https://github.com/apache/kafka/pull/11296#discussion_r702529338
##
File path: raft/src/test/java/org/apache/kafka/raft/KafkaRaftClientTest.java
##
@@ -2799,4 +2779,11 @@ public void testObserverFetchWithNoLocalId() throws
Exception {
private static KafkaMetric getMetric(final Metrics metrics, final String
name) {
return metrics.metrics().get(metrics.metricName(name, "raft-metrics"));
}
+
+private static Optional return0IfBoundEqualsElectionTimeout(int
bound) {
Review comment:
Yes. I removed this method in the latest commit, but used `OptionalInt`
instead of `Optional` in `MockableRandom`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (KAFKA-9366) Upgrade log4j to log4j2
[ https://issues.apache.org/jira/browse/KAFKA-9366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17410283#comment-17410283 ] Dongjin Lee commented on KAFKA-9366: ALL // Sorry for being late. This KIP was originally passed for AK 3.0 but dropped from the release for the lack of review. I think it will be included in 3.1. If you need this feature urgently, please have a look at a custom build of 2.7.0 [here|http://home.apache.org/~dongjin/post/apache-kafka-log4j2-support/]. I could not complete it for 2.8.0 and 3.0 for my medical concerns but will resume the work now. If you are using log4j-appender, please have a look at [KIP-719|https://cwiki.apache.org/confluence/display/KAFKA/KIP-719%3A+Add+Log4J2+Appender]. This KIP proposes a log4j2 equivalent for log4j-appender. I am also working on a custom release of it. Thank you again for your interest in my workings. > Upgrade log4j to log4j2 > --- > > Key: KAFKA-9366 > URL: https://issues.apache.org/jira/browse/KAFKA-9366 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 2.2.0, 2.1.1, 2.3.0, 2.4.0 >Reporter: leibo >Assignee: Dongjin Lee >Priority: Critical > Labels: needs-kip > Fix For: 3.1.0 > > > h2. CVE-2019-17571 Detail > Included in Log4j 1.2 is a SocketServer class that is vulnerable to > deserialization of untrusted data which can be exploited to remotely execute > arbitrary code when combined with a deserialization gadget when listening to > untrusted network traffic for log data. This affects Log4j versions up to 1.2 > up to 1.2.17. > > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17571] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kafka] lbradstreet opened a new pull request #11298: MINOR: "partition" typos and method doc arg fix
lbradstreet opened a new pull request #11298: URL: https://github.com/apache/kafka/pull/11298 Some minor typos that have annoyed me along with a fix to a docstring. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] lbradstreet commented on a change in pull request #10914: [KAFKA-8522] Streamline tombstone and transaction marker removal
lbradstreet commented on a change in pull request #10914:
URL: https://github.com/apache/kafka/pull/10914#discussion_r702508503
##
File path: core/src/main/scala/kafka/log/LogCleanerManager.scala
##
@@ -198,8 +199,23 @@ private[log] class LogCleanerManager(val logDirs:
Seq[File],
val cleanableLogs = dirtyLogs.filter { ltc =>
(ltc.needCompactionNow && ltc.cleanableBytes > 0) ||
ltc.cleanableRatio > ltc.log.config.minCleanableRatio
}
+
if(cleanableLogs.isEmpty) {
-None
+val logsWithTombstonesExpired = dirtyLogs.filter {
+ case ltc =>
+// in this case, we are probably in a low throughput situation
+// therefore, we should take advantage of this fact and remove
tombstones if we can
+// under the condition that the log's latest delete horizon is
less than the current time
+// tracked
+ltc.log.latestDeleteHorizon != RecordBatch.NO_TIMESTAMP &&
ltc.log.latestDeleteHorizon <= time.milliseconds()
Review comment:
It seems like whether we track the delete horizon or the # of tombstones
we will need to checkpoint some state. Otherwise we will be forced to perform a
pass after every broker restart. Could we track the delete horizon upon each
log append, when we clean the log, and when we have to recover the log?
I'm not sure where a checkpoint should be stored given our current
checkpoint file formats and the need to support downgrades.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] lbradstreet commented on a change in pull request #10914: [KAFKA-8522] Streamline tombstone and transaction marker removal
lbradstreet commented on a change in pull request #10914:
URL: https://github.com/apache/kafka/pull/10914#discussion_r702503828
##
File path: core/src/main/scala/kafka/log/LogCleanerManager.scala
##
@@ -198,13 +204,28 @@ private[log] class LogCleanerManager(val logDirs:
Seq[File],
val cleanableLogs = dirtyLogs.filter { ltc =>
(ltc.needCompactionNow && ltc.cleanableBytes > 0) ||
ltc.cleanableRatio > ltc.log.config.minCleanableRatio
}
+
Review comment:
Preexisting nit issue: whitespace between `if` and `(` in
`if(cleanableLogs`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] lbradstreet commented on a change in pull request #10914: [KAFKA-8522] Streamline tombstone and transaction marker removal
lbradstreet commented on a change in pull request #10914:
URL: https://github.com/apache/kafka/pull/10914#discussion_r702503726
##
File path: core/src/main/scala/kafka/log/LogCleanerManager.scala
##
@@ -163,17 +168,18 @@ private[log] class LogCleanerManager(val logDirs:
Seq[File],
* Choose the log to clean next and add it to the in-progress set. We
recompute this
* each time from the full set of logs to allow logs to be dynamically
added to the pool of logs
* the log manager maintains.
+* Returns a tuple of an Option of the log selected to be cleaned and the
reason it was selected.
*/
- def grabFilthiestCompactedLog(time: Time, preCleanStats: PreCleanStats = new
PreCleanStats()): Option[LogToClean] = {
+ def grabFilthiestCompactedLog(time: Time, preCleanStats: PreCleanStats = new
PreCleanStats()): (Option[LogToClean], LogCleaningReason) = {
Review comment:
Could LogCleaningReason be included as a field in LogToClean?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (KAFKA-13261) KTable to KTable foreign key join loose events when using several partitions
[
https://issues.apache.org/jira/browse/KAFKA-13261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17410256#comment-17410256
]
Tomas Forsman commented on KAFKA-13261:
---
I've added a test that seem to verify a problem with the join. I need to look
at it more tomorrow. If anyone's curious it's here
https://github.com/apache/kafka/commit/2e6edf2951f482029aea80acd2f0960085a5f396
> KTable to KTable foreign key join loose events when using several partitions
>
>
> Key: KAFKA-13261
> URL: https://issues.apache.org/jira/browse/KAFKA-13261
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Affects Versions: 2.8.0, 2.7.1
>Reporter: Tomas Forsman
>Priority: Major
> Attachments: KafkaTest.java
>
>
> Two incoming streams A and B.
> Stream A uses a composite key [a, b]
> Stream B has key [b]
> Stream B has 4 partitions and steams A has 1 partition.
> What we try to do is repartition stream A to have 4 partitions too, then put
> both A and B into KTable and do a foreign key join on from A to B
> When doing this, all messages does not end up in the output topic.
> Repartitioning both to only use 1 partition each solve the problem so it seem
> like it has something to do with the foreign key join in combination with
> several partitions.
> One suspicion would be that it is not possible to define what partitioner to
> use for the join.
> Any insight or help is greatly appreciated.
> *Example code of the problem*
> {code:java}
> static Topology createTopoology(){
> var builder = new StreamsBuilder();
> KTable tableB = builder.table("B",
> stringMaterialized("table.b"));
> builder
> .stream("A", Consumed.with(Serde.of(KeyA.class),
> Serde.of(EventA.class)))
> .repartition(repartitionTopicA())
> .toTable(Named.as("table.a"), aMaterialized("table.a"))
> .join(tableB, EventA::getKeyB, topicAandBeJoiner(),
> Named.as("join.ab"), joinMaterialized("join.ab"))
> .toStream()
> .to("output", with(...));
> return builder.build();
> }
> private static Materialized aMaterialized(String name) {
> Materialized> table =
> Materialized.as(name);
> return
> table.withKeySerde(Serde.of(KeyA.class)).withValueSerde(Serde.of(EventA.class));
> }
> private static Repartitioned repartitionTopicA() {
> Repartitioned repartitioned =
> Repartitioned.as("driverperiod");
> return
> repartitioned.withKeySerde(Serde.of(KeyA.class)).withValueSerde(Serde.of(EventA.class))
> .withStreamPartitioner(topicAPartitioner())
> .withNumberOfPartitions(4);
> }
> private static StreamPartitioner
> topicAPartitioner() {
> return (topic, key, value, numPartitions) ->
> Math.abs(key.getKeyB().hashCode()) % numPartitions;
> }
> private static Materialized>
> joinMaterialized(String name) {
> Materialized>
> table = Materialized.as(name);
> return
> table.withKeySerde(Serde.of(KeyA.class)).withValueSerde(Serde.of(EventA.class));
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [kafka] gunnarmorling commented on a change in pull request #11296: KAFKA-13273: Add support for Java 17
gunnarmorling commented on a change in pull request #11296:
URL: https://github.com/apache/kafka/pull/11296#discussion_r702474291
##
File path: raft/src/test/java/org/apache/kafka/raft/KafkaRaftClientTest.java
##
@@ -2799,4 +2779,11 @@ public void testObserverFetchWithNoLocalId() throws
Exception {
private static KafkaMetric getMetric(final Metrics metrics, final String
name) {
return metrics.metrics().get(metrics.metricName(name, "raft-metrics"));
}
+
+private static Optional return0IfBoundEqualsElectionTimeout(int
bound) {
Review comment:
Tiny nit: could `OptionalInt` be returned instead?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] ijuma commented on pull request #11296: KAFKA-13273: Add support for Java 17
ijuma commented on pull request #11296: URL: https://github.com/apache/kafka/pull/11296#issuecomment-913214595 @chia7712 @omkreddy Any of you have the cycles to review this? Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] ijuma edited a comment on pull request #11296: KAFKA-13273: Add support for Java 17
ijuma edited a comment on pull request #11296: URL: https://github.com/apache/kafka/pull/11296#issuecomment-913214267 Build 6 (including Java 16) passed: https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-11296/6/ I will now remove Java 16 from Jenkinsfile. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [kafka] ijuma commented on pull request #11296: KAFKA-13273: Add support for Java 17
ijuma commented on pull request #11296: URL: https://github.com/apache/kafka/pull/11296#issuecomment-913214267 Build #6 passed: https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-11296/6/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (KAFKA-13275) Remove --add-opens from build.gradle (where possible)
[ https://issues.apache.org/jira/browse/KAFKA-13275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-13275: Description: We added several `--add-opens` to make a number of tests pass with Java 17 (via KAFKA-13273). It should be possible to remove many (most?) of them by adjusting the tests. We should merge the various PRs associated with KAFKA-7438's subtasks before attempting this, I suspect they will help. was: We added several `--add-opens` to make a number of tests pass with Java 17 (via KAFKA-13273). It should be possible to remove many (most?) of them by adjusting the tests. We should merge the various PRs associated with KAFKA-7438 before doing this, I suspect they will help. > Remove --add-opens from build.gradle (where possible) > - > > Key: KAFKA-13275 > URL: https://issues.apache.org/jira/browse/KAFKA-13275 > Project: Kafka > Issue Type: Improvement >Reporter: Ismael Juma >Priority: Major > > We added several `--add-opens` to make a number of tests pass with Java 17 > (via KAFKA-13273). > It should be possible to remove many (most?) of them by adjusting the tests. > We should merge the various PRs associated with KAFKA-7438's subtasks before > attempting this, I suspect they will help. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KAFKA-13275) Remove --add-opens from build.gradle (where possible)
[ https://issues.apache.org/jira/browse/KAFKA-13275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-13275: Description: We added several `--add-opens` to make a number of tests pass with Java 17 (via KAFKA-13273). It should be possible to remove many (most?) of them by adjusting the tests. We should merge the various PRs associated with KAFKA-7438 before doing this, I suspect they will help. was: We added several `--add-opens` to make a number of tests pass with Java 17 (via KAFKA-13273). It should be possible to remove many (most?) of them by adjusting the tests. > Remove --add-opens from build.gradle (where possible) > - > > Key: KAFKA-13275 > URL: https://issues.apache.org/jira/browse/KAFKA-13275 > Project: Kafka > Issue Type: Improvement >Reporter: Ismael Juma >Priority: Major > > We added several `--add-opens` to make a number of tests pass with Java 17 > (via KAFKA-13273). > It should be possible to remove many (most?) of them by adjusting the tests. > We should merge the various PRs associated with KAFKA-7438 before doing this, > I suspect they will help. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (KAFKA-13275) Remove --add-opens from build.gradle (where possible)
Ismael Juma created KAFKA-13275: --- Summary: Remove --add-opens from build.gradle (where possible) Key: KAFKA-13275 URL: https://issues.apache.org/jira/browse/KAFKA-13275 Project: Kafka Issue Type: Improvement Reporter: Ismael Juma We added several `--add-opens` to make a number of tests pass with Java 17 (via KAFKA-13273). It should be possible to remove many (most?) of them by adjusting the tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Issue Comment Deleted] (KAFKA-13273) Add support for Java 17
[ https://issues.apache.org/jira/browse/KAFKA-13273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-13273: Comment: was deleted (was: PR: https://github.com/apache/kafka/pull/11296) > Add support for Java 17 > --- > > Key: KAFKA-13273 > URL: https://issues.apache.org/jira/browse/KAFKA-13273 > Project: Kafka > Issue Type: Improvement >Reporter: Ismael Juma >Assignee: Ismael Juma >Priority: Major > Fix For: 3.1.0 > > > Java 17 is at release candidate stage and it will be a LTS release once > it's out (previous LTS release was Java 11). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (KAFKA-13274) Ensure system tests run successfully with Java 17
Ismael Juma created KAFKA-13274: --- Summary: Ensure system tests run successfully with Java 17 Key: KAFKA-13274 URL: https://issues.apache.org/jira/browse/KAFKA-13274 Project: Kafka Issue Type: Improvement Reporter: Ismael Juma -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kafka] ijuma commented on a change in pull request #11296: KAFKA-13273: Add support for Java 17
ijuma commented on a change in pull request #11296:
URL: https://github.com/apache/kafka/pull/11296#discussion_r702460031
##
File path: build.gradle
##
@@ -37,7 +37,7 @@ plugins {
id 'org.owasp.dependencycheck' version '6.1.6'
id 'org.nosphere.apache.rat' version "0.7.0"
- id "com.github.spotbugs" version '4.7.1' apply false
+ id "com.github.spotbugs" version '4.7.3' apply false
Review comment:
Will remove the spotBugs change from this PR and submit it separately.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] ijuma commented on a change in pull request #11296: KAFKA-13273: Add support for Java 17
ijuma commented on a change in pull request #11296:
URL: https://github.com/apache/kafka/pull/11296#discussion_r702450561
##
File path: build.gradle
##
@@ -37,7 +37,7 @@ plugins {
id 'org.owasp.dependencycheck' version '6.1.6'
id 'org.nosphere.apache.rat' version "0.7.0"
- id "com.github.spotbugs" version '4.7.1' apply false
+ id "com.github.spotbugs" version '4.7.3' apply false
Review comment:
The spotBugs version upgrade had some issues, need to investigate them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] ijuma commented on a change in pull request #11296: KAFKA-13273: Add support for Java 17
ijuma commented on a change in pull request #11296: URL: https://github.com/apache/kafka/pull/11296#discussion_r702450437 ## File path: gradle/wrapper/gradle-wrapper.properties ## @@ -1,6 +1,5 @@ distributionBase=GRADLE_USER_HOME distributionPath=wrapper/dists -distributionSha256Sum=9bb8bc05f562f2d42bdf1ba8db62f6b6fa1c3bf6c392228802cc7cb0578fe7e0 Review comment: I need to add back this line (the wrapper script doesn't generate it for some reason). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (KAFKA-13273) Add support for Java 17
[ https://issues.apache.org/jira/browse/KAFKA-13273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-13273: Description: Java 17 is at release candidate stage and it will be a LTS release once it's out (previous LTS release was Java 11). > Add support for Java 17 > --- > > Key: KAFKA-13273 > URL: https://issues.apache.org/jira/browse/KAFKA-13273 > Project: Kafka > Issue Type: Improvement >Reporter: Ismael Juma >Assignee: Ismael Juma >Priority: Major > Fix For: 3.1.0 > > > Java 17 is at release candidate stage and it will be a LTS release once > it's out (previous LTS release was Java 11). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (KAFKA-13273) Add support for Java 17
Ismael Juma created KAFKA-13273: --- Summary: Add support for Java 17 Key: KAFKA-13273 URL: https://issues.apache.org/jira/browse/KAFKA-13273 Project: Kafka Issue Type: Improvement Reporter: Ismael Juma Assignee: Ismael Juma Fix For: 3.1.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-13273) Add support for Java 17
[ https://issues.apache.org/jira/browse/KAFKA-13273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17410206#comment-17410206 ] Ismael Juma commented on KAFKA-13273: - PR: https://github.com/apache/kafka/pull/11296 > Add support for Java 17 > --- > > Key: KAFKA-13273 > URL: https://issues.apache.org/jira/browse/KAFKA-13273 > Project: Kafka > Issue Type: Improvement >Reporter: Ismael Juma >Assignee: Ismael Juma >Priority: Major > Fix For: 3.1.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [kafka] satishd commented on a change in pull request #11058: KAFKA-12802 Added a file based cache for consumed remote log metadata for each partition to avoid consuming again incase of broke
satishd commented on a change in pull request #11058:
URL: https://github.com/apache/kafka/pull/11058#discussion_r702422832
##
File path:
storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/CommittedOffsetsFile.java
##
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kafka.server.log.remote.metadata.storage;
+
+import org.apache.kafka.common.utils.Utils;
+
+import java.io.BufferedReader;
+import java.io.BufferedWriter;
+import java.io.File;
+import java.io.FileOutputStream;
+import java.io.IOException;
+import java.io.OutputStreamWriter;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.regex.Pattern;
+
+/**
+ * This class represents a file containing the committed offsets of remote log
metadata partitions.
+ */
+public class CommittedOffsetsFile {
Review comment:
This is updated with the latest commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Assigned] (KAFKA-9886) Validate segment range before reading in `Log.read`
[
https://issues.apache.org/jira/browse/KAFKA-9886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
HaiyuanZhao reassigned KAFKA-9886:
--
Assignee: HaiyuanZhao
> Validate segment range before reading in `Log.read`
> ---
>
> Key: KAFKA-9886
> URL: https://issues.apache.org/jira/browse/KAFKA-9886
> Project: Kafka
> Issue Type: Improvement
>Reporter: Jason Gustafson
>Assignee: HaiyuanZhao
>Priority: Major
>
> Log.read uses the following logic to set the upper limit on a segment read.
> {code}
> val maxPosition = {
>// Use the max offset position if it is on this segment; otherwise, the
> segment size is the limit.
> if (maxOffsetMetadata.segmentBaseOffset == segment.baseOffset) {
> maxOffsetMetadata.relativePositionInSegment
> } else {
> segment.size
> }
> }
> {code}
> In the else branch, the expectation is that
> `maxOffsetMetadata.segmentBaseOffset > segment.baseOffset`. In KAFKA-9838, we
> found a bug where this assumption failed which led to reads above the high
> watermark. We should validate the expectation explicitly so that we don't
> leave the door open for similar bugs in the future.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
