[jira] [Commented] (KAFKA-10859) add @Test annotation to FileStreamSourceTaskTest.testInvalidFile and reduce the loop count to speedup the test
[ https://issues.apache.org/jira/browse/KAFKA-10859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718338#comment-17718338 ] Manasvi Gupta commented on KAFKA-10859: --- This issue appears to have been resolved already in master branch. https://github.com/apache/kafka/blob/trunk/connect/file/src/test/java/org/apache/kafka/connect/file/FileStreamSourceTaskTest.java#L216 So, can we close this ticket ? > add @Test annotation to FileStreamSourceTaskTest.testInvalidFile and reduce > the loop count to speedup the test > -- > > Key: KAFKA-10859 > URL: https://issues.apache.org/jira/browse/KAFKA-10859 > Project: Kafka > Issue Type: Improvement >Reporter: Chia-Ping Tsai >Assignee: Tom Bentley >Priority: Major > Labels: newbie > > FileStreamSourceTaskTest.testInvalidFile miss a `@Test` annotation. Also, it > loops 100 times which spend about 2m to complete a unit test. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (KAFKA-13070) LogManager shutdown races with periodic work scheduled by the instance
[
https://issues.apache.org/jira/browse/KAFKA-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manasvi Gupta reassigned KAFKA-13070:
-
Assignee: Manasvi Gupta
> LogManager shutdown races with periodic work scheduled by the instance
> --
>
> Key: KAFKA-13070
> URL: https://issues.apache.org/jira/browse/KAFKA-13070
> Project: Kafka
> Issue Type: Bug
>Reporter: Kowshik Prakasam
>Assignee: Manasvi Gupta
>Priority: Major
>
> In the LogManager shutdown sequence (in LogManager.shutdown()), we don't
> cancel the periodic work scheduled by it prior to shutdown. As a result, the
> periodic work could race with the shutdown sequence causing some unwanted
> side effects. This is reproducible by a unit test in LogManagerTest.
>
> ```
> // set val maxLogAgeMs = 6 in the test
> @Test
> def testRetentionPeriodicWorkAfterShutdown(): Unit = {
> val log = logManager.getOrCreateLog(new TopicPartition(name, 0), topicId
> = None)
> val logFile = new File(logDir, name + "-0")
> assertTrue(logFile.exists)
> log.appendAsLeader(TestUtils.singletonRecords("test1".getBytes()),
> leaderEpoch = 0)
> log.updateHighWatermark(log.logEndOffset)
> logManager.shutdown()
> assertTrue(Files.exists(new File(logDir,
> LogLoader.CleanShutdownFile).toPath))
> time.sleep(maxLogAgeMs + logManager.InitialTaskDelayMs +
> logManager.retentionCheckMs + 1)
> logManager = null
> }
> ```
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (KAFKA-12767) Properly set Streams system test runtime classpath
[ https://issues.apache.org/jira/browse/KAFKA-12767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manasvi Gupta reassigned KAFKA-12767: - Assignee: Manasvi Gupta > Properly set Streams system test runtime classpath > -- > > Key: KAFKA-12767 > URL: https://issues.apache.org/jira/browse/KAFKA-12767 > Project: Kafka > Issue Type: Task > Components: streams, system tests >Affects Versions: 3.0.0 >Reporter: John Roesler >Assignee: Manasvi Gupta >Priority: Minor > Labels: newbie++ > > Some of the streams system tests started to fail recently when we stopped > exporting our transitive dependencies in the test jar. > [~lct45] was kind enough to submit > [https://github.com/apache/kafka/pull/10631] to get the system tests running > again, but that PR is only a stop-gap. > The real solution is to properly package the transitive dependencies and make > them available to the system test runtime. > Here is the reason: PR#10631 gets past the issue by removing runtime usages > on Hamcrest, but Hamcrest is still present in the compiletime classpath. > Tomorrow, a new PR could add a reference to Hamcrest, and all the unit and > integration tests would pass, but we would again see the mysterious system > test failures. Only after another round of costly investigation would we > realize the root cause, and we might again decide to just patch the test to > remove the reference. > It would be far better in the long run to fix the underlying condition: the > difference between the compiletime and runtime classpaths for the system > tests. > > To help get you started, I'll share some of the groundwork for this task, > which I put together while trying to understand the nature of the problem. > The first step is to actually copy the transitive dependencies. We had to > stop placing these dependencies in the `dependant-libs` build directory > because that results in us actually shipping those dependencies with our > releases. Copying a similar mechanism from the `:core` project, we can add a > new build directory (arbitrarily: `dependant-testlibs`), and again copy the > runtime dependencies there. Here is a commit in my fork that does just that: > [https://github.com/vvcephei/kafka/commit/8d4552dee05f2a963b8072b86aae756415ea2482] > The next step is to place those jars on the classpath of the system test > code. The mechanism for that is `kafka-run-class.sh`: > [https://github.com/apache/kafka/blob/trunk/bin/kafka-run-class.sh] > A specific example of this is the special logic for upgrade tests: > # If we are running upgrade tests, then we set the artifact directories to > the relevant version. Otherwise, we use the current build artifacts. > [https://github.com/apache/kafka/blob/fc405d792de12a50956195827eaf57bbf6c9/bin/kafka-run-class.sh#L77-L85] > # The, here's where we specifically pull in Hamcrest from those build > artifact directories: > [https://github.com/apache/kafka/blob/fc405d792de12a50956195827eaf57bbf6c9/bin/kafka-run-class.sh#L128-L136] > It seems to me that since Hamcrest is actually a more general dependency of > the tests, we might as well pack it up in `dependant-testlibs` and then pull > it into the classpath from there any time we're running tests. It looks like > we ought to set `streams_dependant_clients_lib_dir` to `dependant-testlibs` > any time `INCLUDE_TEST_JARS` is `true`. But if we do have > `UPGRADE_KAFKA_STREAMS_TEST_VERSION` set, then it should override the lib > dir, since those artifacts to copy over the transitive dependencies for those > older versions already. > > Although the proposed fix itself is pretty small, I think the testing will > take a few days. You might want to just put some `echo` statements in > kafka-run-class.sh to see what jars are included on the classpath while you > run different tests, both locally using Docker, and remotely using Jenkins. > I marked this ticket as `newbie++` because you don't need deep knowledge of > the codebase to tackle this ticket, just a high pain tolerance for digging > though gradle/docker/bash script debugging to make sure the right bits make > it into the system tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (KAFKA-9034) kafka-run-class.sh will fail if JAVA_HOME has space
[
https://issues.apache.org/jira/browse/KAFKA-9034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manasvi Gupta reassigned KAFKA-9034:
Assignee: Manasvi Gupta
> kafka-run-class.sh will fail if JAVA_HOME has space
> ---
>
> Key: KAFKA-9034
> URL: https://issues.apache.org/jira/browse/KAFKA-9034
> Project: Kafka
> Issue Type: Bug
> Components: tools
> Environment: macOS
>Reporter: Fenjin Wang
>Assignee: Manasvi Gupta
>Priority: Minor
> Labels: easyfix, newbie
>
> If set JAVA_HOME to IntelliJ's java, bin/zookeeper-server-start.sh can't work
> because the path has space in it.
>
> {quote}export JAVA_HOME="/Applications/IntelliJ
> IDEA.app/Contents/jbr/Contents/Home/"
> {quote}
>
> We can fix this by quote "$JAVA" in the shell script according to:
> [https://stackoverflow.com/a/7740746/1203241]
>
> I can send a PR if you like.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (KAFKA-9234) Consider using @Nullable and @Nonnull annotations
[ https://issues.apache.org/jira/browse/KAFKA-9234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manasvi Gupta reassigned KAFKA-9234: Assignee: Manasvi Gupta > Consider using @Nullable and @Nonnull annotations > - > > Key: KAFKA-9234 > URL: https://issues.apache.org/jira/browse/KAFKA-9234 > Project: Kafka > Issue Type: Improvement > Components: admin, clients, consumer, KafkaConnect, producer , > streams, streams-test-utils >Reporter: Matthias J. Sax >Assignee: Manasvi Gupta >Priority: Minor > Labels: beginner, newbie > > Java7 was dropped some time ago, and we might want to consider usein Java8 > `@Nullable` and `@Nonnull` annotations for all public facing APIs instead of > documenting it in JavaDocs only. > This tickets should be broken down in a series of smaller PRs to keep the > scope of each PR contained, allowing for more effective reviews. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-6629) SegmentedCacheFunctionTest does not cover session window serdes
[ https://issues.apache.org/jira/browse/KAFKA-6629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482819#comment-16482819 ] Manasvi Gupta commented on KAFKA-6629: -- Hi [~vpopov89], I am working on this issue. Thanks Manasvi > SegmentedCacheFunctionTest does not cover session window serdes > --- > > Key: KAFKA-6629 > URL: https://issues.apache.org/jira/browse/KAFKA-6629 > Project: Kafka > Issue Type: Improvement > Components: streams >Reporter: Guozhang Wang >Assignee: Manasvi Gupta >Priority: Major > Labels: newbie, unit-test > > The SegmentedCacheFunctionTest.java only covers time window serdes, but not > session window serdes. We should fill in this coverage gap. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-5706) log the name of the error instead of the error code in response objects
[ https://issues.apache.org/jira/browse/KAFKA-5706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16429742#comment-16429742 ] Manasvi Gupta commented on KAFKA-5706: -- I also don't see a toString() function implemented in this class - Errors.java. > log the name of the error instead of the error code in response objects > --- > > Key: KAFKA-5706 > URL: https://issues.apache.org/jira/browse/KAFKA-5706 > Project: Kafka > Issue Type: Improvement > Components: clients, core >Affects Versions: 0.11.0.0 >Reporter: Jun Rao >Assignee: Manasvi Gupta >Priority: Major > Labels: newbie > > Currently, when logging the error code in the response objects, we simply log > response.toString(), which contains the error code. It will be useful to log > the name of the corresponding exception for the error, which is more > meaningful than an error code. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-5706) log the name of the error instead of the error code in response objects
[ https://issues.apache.org/jira/browse/KAFKA-5706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16429739#comment-16429739 ] Manasvi Gupta commented on KAFKA-5706: -- Hi [~junrao], 1) Is there an example of current error message and desired one? 2) While digging through the code, I found enum *Errors.java* which provides message() function that returns error messages. Is this right class for this change? Thanks Manasvi > log the name of the error instead of the error code in response objects > --- > > Key: KAFKA-5706 > URL: https://issues.apache.org/jira/browse/KAFKA-5706 > Project: Kafka > Issue Type: Improvement > Components: clients, core >Affects Versions: 0.11.0.0 >Reporter: Jun Rao >Assignee: Manasvi Gupta >Priority: Major > Labels: newbie > > Currently, when logging the error code in the response objects, we simply log > response.toString(), which contains the error code. It will be useful to log > the name of the corresponding exception for the error, which is more > meaningful than an error code. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-3827) log.message.format.version should default to inter.broker.protocol.version
[ https://issues.apache.org/jira/browse/KAFKA-3827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16429491#comment-16429491 ] Manasvi Gupta commented on KAFKA-3827: -- Hi All, Just wanted to check if this is still a valid bug that needs to be fixed. Thanks Manasvi > log.message.format.version should default to inter.broker.protocol.version > -- > > Key: KAFKA-3827 > URL: https://issues.apache.org/jira/browse/KAFKA-3827 > Project: Kafka > Issue Type: Improvement >Affects Versions: 0.10.0.0 >Reporter: Jun Rao >Assignee: Manasvi Gupta >Priority: Major > Labels: newbie > > Currently, if one sets inter.broker.protocol.version to 0.9.0 and restarts > the broker, one will get the following exception since > log.message.format.version defaults to 0.10.0. It will be more intuitive if > log.message.format.version defaults to the value of > inter.broker.protocol.version. > java.lang.IllegalArgumentException: requirement failed: > log.message.format.version 0.10.0-IV1 cannot be used when > inter.broker.protocol.version is set to 0.9.0.1 > at scala.Predef$.require(Predef.scala:233) > at kafka.server.KafkaConfig.validateValues(KafkaConfig.scala:1023) > at kafka.server.KafkaConfig.(KafkaConfig.scala:994) > at kafka.server.KafkaConfig$.fromProps(KafkaConfig.scala:743) > at kafka.server.KafkaConfig$.fromProps(KafkaConfig.scala:740) > at > kafka.server.KafkaServerStartable$.fromProps(KafkaServerStartable.scala:28) > at kafka.Kafka$.main(Kafka.scala:58) > at kafka.Kafka.main(Kafka.scala) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KAFKA-6629) SegmentedCacheFunctionTest does not cover session window serdes
[ https://issues.apache.org/jira/browse/KAFKA-6629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manasvi Gupta reassigned KAFKA-6629: Assignee: Manasvi Gupta > SegmentedCacheFunctionTest does not cover session window serdes > --- > > Key: KAFKA-6629 > URL: https://issues.apache.org/jira/browse/KAFKA-6629 > Project: Kafka > Issue Type: Improvement > Components: streams >Reporter: Guozhang Wang >Assignee: Manasvi Gupta >Priority: Major > Labels: newbie, unit-test > > The SegmentedCacheFunctionTest.java only covers time window serdes, but not > session window serdes. We should fill in this coverage gap. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-5706) log the name of the error instead of the error code in response objects
[ https://issues.apache.org/jira/browse/KAFKA-5706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247103#comment-16247103 ] Manasvi Gupta commented on KAFKA-5706: -- Yeah. If i am unable to follow-up in next few days, i will assign to you (assuming you want it). Thanks > log the name of the error instead of the error code in response objects > --- > > Key: KAFKA-5706 > URL: https://issues.apache.org/jira/browse/KAFKA-5706 > Project: Kafka > Issue Type: Improvement > Components: clients, core >Affects Versions: 0.11.0.0 >Reporter: Jun Rao >Assignee: Manasvi Gupta > Labels: newbie > > Currently, when logging the error code in the response objects, we simply log > response.toString(), which contains the error code. It will be useful to log > the name of the corresponding exception for the error, which is more > meaningful than an error code. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KAFKA-5706) log the name of the error instead of the error code in response objects
[ https://issues.apache.org/jira/browse/KAFKA-5706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16143340#comment-16143340 ] Manasvi Gupta commented on KAFKA-5706: -- Hello [~holdenk], Yeah, I plan to work on this issue this week. (BTW, big fan of your work on Apache Spark). Thanks Manasvi > log the name of the error instead of the error code in response objects > --- > > Key: KAFKA-5706 > URL: https://issues.apache.org/jira/browse/KAFKA-5706 > Project: Kafka > Issue Type: Improvement > Components: clients, core >Affects Versions: 0.11.0.0 >Reporter: Jun Rao >Assignee: Manasvi Gupta > Labels: newbie > > Currently, when logging the error code in the response objects, we simply log > response.toString(), which contains the error code. It will be useful to log > the name of the corresponding exception for the error, which is more > meaningful than an error code. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (KAFKA-5706) log the name of the error instead of the error code in response objects
[ https://issues.apache.org/jira/browse/KAFKA-5706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manasvi Gupta reassigned KAFKA-5706: Assignee: Manasvi Gupta > log the name of the error instead of the error code in response objects > --- > > Key: KAFKA-5706 > URL: https://issues.apache.org/jira/browse/KAFKA-5706 > Project: Kafka > Issue Type: Improvement > Components: clients, core >Affects Versions: 0.11.0.0 >Reporter: Jun Rao >Assignee: Manasvi Gupta > Labels: newbie > > Currently, when logging the error code in the response objects, we simply log > response.toString(), which contains the error code. It will be useful to log > the name of the corresponding exception for the error, which is more > meaningful than an error code. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
