[jira] [Commented] (KAFKA-7491) Kafka streams and Kafka streams test utils have split packages
[ https://issues.apache.org/jira/browse/KAFKA-7491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17519420#comment-17519420 ] Robin Van Praet commented on KAFKA-7491: [~k.schnit...@sap.com] we include the kafka-streams-testability-patch.jar as a dependency (which will be used during testing). But in our deliverable we exclude this and include the original kafka-streams in its place. We have like a bom-pom for the deliverable, we exclude the testability patch as dependency there and explicitly include the normal one. I know, it is not the best solution, but it worked for us in until this would be fixed. But as you can see, we logged it more then 3 years ago and it is still not fixed. Also, in our project we are no longer using kafka-streams so we did not invest time in finding another solution. > Kafka streams and Kafka streams test utils have split packages > -- > > Key: KAFKA-7491 > URL: https://issues.apache.org/jira/browse/KAFKA-7491 > Project: Kafka > Issue Type: Bug > Components: streams >Reporter: Robin Van Praet >Priority: Major > > When trying to test a Kafka Streams application using JDK 9+, using the > module path, (test) compilation errors occur. > The TopologyTestDriver cannot be located in kafka-streams-test-utils because > package 'org.apache.kafka.streams' is already provided by module > kafka-streams. > Please make sure that packages are not re-used between production libraries > and test util libraries. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (KAFKA-7491) Kafka streams and Kafka streams test utils have split packages
[ https://issues.apache.org/jira/browse/KAFKA-7491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17519412#comment-17519412 ] Karsten Schnitter commented on KAFKA-7491: -- [~robin.vanpr...@aca-it.be] I do not quite understand, how that would help. In production I would not want to run with the shaded jar but use the originally provided one. The test-util code is not necessary for production and using the shaded jar would make me an owner of the entire kafka-streams code base. In the end this will create an automatic module out of the kafka-streams.jar. In the unit tests, I still cannot use the shaded-jar, since it conflicts with the original. Did I understand your proposal correctly to always use the kafka-streams-testability-patch.jar instead of the original kafa-streams.jar? > Kafka streams and Kafka streams test utils have split packages > -- > > Key: KAFKA-7491 > URL: https://issues.apache.org/jira/browse/KAFKA-7491 > Project: Kafka > Issue Type: Bug > Components: streams >Reporter: Robin Van Praet >Priority: Major > > When trying to test a Kafka Streams application using JDK 9+, using the > module path, (test) compilation errors occur. > The TopologyTestDriver cannot be located in kafka-streams-test-utils because > package 'org.apache.kafka.streams' is already provided by module > kafka-streams. > Please make sure that packages are not re-used between production libraries > and test util libraries. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (KAFKA-7491) Kafka streams and Kafka streams test utils have split packages
[
https://issues.apache.org/jira/browse/KAFKA-7491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17519371#comment-17519371
]
Robin Van Praet commented on KAFKA-7491:
What we did was using the shade-plugin to combine kafka-streams and
kafka-streams-test-utils into a new artifact (we called it
kafka-streams-testability-patch). And we use that one for our builds. You just
need to make sure that you include the original one in your deliverable.
{code:java}
http://maven.apache.org/POM/4.0.0";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
...
kafka-streams-testability-patch
org.apache.kafka
kafka-streams
2.6.2
org.apache.kafka
connect-json
org.apache.kafka
kafka-streams-test-utils
2.6.2
maven-shade-plugin
3.2.0
package
shade
org.apache.kafka:kafka-streams*
true
kafka.streams
{code}
> Kafka streams and Kafka streams test utils have split packages
> --
>
> Key: KAFKA-7491
> URL: https://issues.apache.org/jira/browse/KAFKA-7491
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Reporter: Robin Van Praet
>Priority: Major
>
> When trying to test a Kafka Streams application using JDK 9+, using the
> module path, (test) compilation errors occur.
> The TopologyTestDriver cannot be located in kafka-streams-test-utils because
> package 'org.apache.kafka.streams' is already provided by module
> kafka-streams.
> Please make sure that packages are not re-used between production libraries
> and test util libraries.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (KAFKA-7491) Kafka streams and Kafka streams test utils have split packages
[
https://issues.apache.org/jira/browse/KAFKA-7491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17519124#comment-17519124
]
Karsten Schnitter commented on KAFKA-7491:
--
Hi, this issue has not seen much action in quite some time. I am experiencing
the same issue. I tried patch-module in the pom.xml but to no avail:
{code:xml}
org.apache.maven.plugins
maven-compiler-plugin
test-compile
process-test-sources
testCompile
--patch-module
kafka.streams=${settings.localRepository}/org/apache/kafka/kafka-streams-test-utils/3.0.1/kafka-streams-test-utils-3.0.1.jar
{code}
That way, I got around the "missing symbol" issue, that is caused by the
test-util sitting in the same package on the classpath, where it is abandoned
due to the automatic kafka-streams module. But compilation still errors with an
incompatible type error for a TestRecord. The offending code in my project is:
{code:java}
private TestRecord readOutput() {
return outputTopic.readRecord();
}
{code}
The error message is:
{noformat}
incompatible types:
org.apache.kafka.streams.test.TestRecord
cannot be converted to
org.apache.kafka.streams.test.TestRecord
{noformat}
I cannot say for sure, if my own {{QuotaStatisticsMessage}} may cause the
problem, but I doubt it, since there are no failures at other places. I think,
the {{TestRecord}} might be incompatible, maybe due to the {{--patch-module}}.
Is there any idea, how to proceed. One way would be to release a second
test-util with different package names.
> Kafka streams and Kafka streams test utils have split packages
> --
>
> Key: KAFKA-7491
> URL: https://issues.apache.org/jira/browse/KAFKA-7491
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Reporter: Robin Van Praet
>Priority: Major
>
> When trying to test a Kafka Streams application using JDK 9+, using the
> module path, (test) compilation errors occur.
> The TopologyTestDriver cannot be located in kafka-streams-test-utils because
> package 'org.apache.kafka.streams' is already provided by module
> kafka-streams.
> Please make sure that packages are not re-used between production libraries
> and test util libraries.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (KAFKA-7491) Kafka streams and Kafka streams test utils have split packages
[ https://issues.apache.org/jira/browse/KAFKA-7491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16644182#comment-16644182 ] Ismael Juma commented on KAFKA-7491: [~vvcephei], we only test with the classpath. We would need to use modules to see this. > Kafka streams and Kafka streams test utils have split packages > -- > > Key: KAFKA-7491 > URL: https://issues.apache.org/jira/browse/KAFKA-7491 > Project: Kafka > Issue Type: Bug > Components: streams >Reporter: Robin Van Praet >Priority: Major > > When trying to test a Kafka Streams application using JDK 9+, using the > module path, (test) compilation errors occur. > The TopologyTestDriver cannot be located in kafka-streams-test-utils because > package 'org.apache.kafka.streams' is already provided by module > kafka-streams. > Please make sure that packages are not re-used between production libraries > and test util libraries. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-7491) Kafka streams and Kafka streams test utils have split packages
[ https://issues.apache.org/jira/browse/KAFKA-7491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16642902#comment-16642902 ] Robin Van Praet commented on KAFKA-7491: Thank you for your response. I have created a small example project that demonstrates the problem: https://github.com/robinvanpraet/kafka-streams-jdk9--failure/tree/develop > Kafka streams and Kafka streams test utils have split packages > -- > > Key: KAFKA-7491 > URL: https://issues.apache.org/jira/browse/KAFKA-7491 > Project: Kafka > Issue Type: Bug > Components: streams >Reporter: Robin Van Praet >Priority: Major > > When trying to test a Kafka Streams application using JDK 9+, using the > module path, (test) compilation errors occur. > The TopologyTestDriver cannot be located in kafka-streams-test-utils because > package 'org.apache.kafka.streams' is already provided by module > kafka-streams. > Please make sure that packages are not re-used between production libraries > and test util libraries. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-7491) Kafka streams and Kafka streams test utils have split packages
[ https://issues.apache.org/jira/browse/KAFKA-7491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16642194#comment-16642194 ] John Roesler commented on KAFKA-7491: - Interesting... That's a pain. Can you provide some steps so we can reproduce this problem? We build and test Kafka (including Streams) on Java 10, and I don't think we encounter this issue. It sounds like we may need to break compatibility to fix this, but I'd like to have a repro in hand to explore our options. Thanks (and sorry), -John > Kafka streams and Kafka streams test utils have split packages > -- > > Key: KAFKA-7491 > URL: https://issues.apache.org/jira/browse/KAFKA-7491 > Project: Kafka > Issue Type: Improvement >Reporter: Robin Van Praet >Priority: Major > > When trying to test a Kafka Streams application using JDK 9+, using the > module path, (test) compilation errors occur. > The TopologyTestDriver cannot be located in kafka-streams-test-utils because > package 'org.apache.kafka.streams' is already provided by module > kafka-streams. > Please make sure that packages are not re-used between production libraries > and test util libraries. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
