[jira] [Commented] (KAFKA-9011) Add KStream#flatTransform and KStream#flatTransformValues to Scala API
[ https://issues.apache.org/jira/browse/KAFKA-9011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17004684#comment-17004684 ] Ismael Juma commented on KAFKA-9011: Yeah, public APIs can only be added in feature releases to ensure forward compatibility within a release series. > Add KStream#flatTransform and KStream#flatTransformValues to Scala API > -- > > Key: KAFKA-9011 > URL: https://issues.apache.org/jira/browse/KAFKA-9011 > Project: Kafka > Issue Type: Improvement > Components: streams >Affects Versions: 2.3.0 >Reporter: Alex Kokachev >Assignee: Alex Kokachev >Priority: Major > Labels: scala, streams > Fix For: 2.5.0, 2.4.1 > > > Part of KIP-313: > [https://cwiki.apache.org/confluence/display/KAFKA/KIP-313%3A+Add+KStream.flatTransform+and+KStream.flatTransformValues] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-9011) Add KStream#flatTransform and KStream#flatTransformValues to Scala API
[ https://issues.apache.org/jira/browse/KAFKA-9011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17004611#comment-17004611 ] Matthias J. Sax commented on KAFKA-9011: I think it's actually borderline to cherry-pick public API changes to bug-fix releases. Even if there was a KIP for 2.3.0, it seems that we should "finish" a KIP in subsequent minor releases only. \cc [~guozhang] [~ijuma] WDYT? > Add KStream#flatTransform and KStream#flatTransformValues to Scala API > -- > > Key: KAFKA-9011 > URL: https://issues.apache.org/jira/browse/KAFKA-9011 > Project: Kafka > Issue Type: Improvement > Components: streams >Affects Versions: 2.3.0 >Reporter: Alex Kokachev >Assignee: Alex Kokachev >Priority: Major > Labels: scala, streams > Fix For: 2.5.0, 2.4.1 > > > Part of KIP-313: > [https://cwiki.apache.org/confluence/display/KAFKA/KIP-313%3A+Add+KStream.flatTransform+and+KStream.flatTransformValues] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-9011) Add KStream#flatTransform and KStream#flatTransformValues to Scala API
[
https://issues.apache.org/jira/browse/KAFKA-9011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17001799#comment-17001799
]
John Roesler commented on KAFKA-9011:
-
Hey [~akokachev], I cherry-picked them to 2.4, but on 2.3 we get some build
failures in the scala test suite.
If you really want them on 2.3, can you send a PR against the 2.3 branch,
porting the PRs and fixing the build?
Thanks,
-John
Failures:
{noformat}
> Task :streams:streams-scala:compileTestScala FAILED
Pruning sources from previous analysis, due to incompatible CompileSetup.
/home/confluent/kafka/streams/streams-scala/src/test/scala/org/apache/kafka/streams/scala/kstream/KStreamTest.scala:193:
not found: value Instant
val now = Instant.now()
^
/home/confluent/kafka/streams/streams-scala/src/test/scala/org/apache/kafka/streams/scala/kstream/KStreamTest.scala:195:
value createInput is not a member of
org.apache.kafka.streams.TopologyTestDriver
val testInput = testDriver.createInput[String, String](sourceTopic)
^
/home/confluent/kafka/streams/streams-scala/src/test/scala/org/apache/kafka/streams/scala/kstream/KStreamTest.scala:196:
value createOutput is not a member of
org.apache.kafka.streams.TopologyTestDriver
val testOutput = testDriver.createOutput[String, String](sinkTopic)
^
/home/confluent/kafka/streams/streams-scala/src/test/scala/org/apache/kafka/streams/scala/kstream/KStreamTest.scala:228:
not found: value Instant
val now = Instant.now()
^
/home/confluent/kafka/streams/streams-scala/src/test/scala/org/apache/kafka/streams/scala/kstream/KStreamTest.scala:230:
value createInput is not a member of
org.apache.kafka.streams.TopologyTestDriver
val testInput = testDriver.createInput[String, String](sourceTopic)
^
/home/confluent/kafka/streams/streams-scala/src/test/scala/org/apache/kafka/streams/scala/kstream/KStreamTest.scala:231:
value createOutput is not a member of
org.apache.kafka.streams.TopologyTestDriver
val testOutput = testDriver.createOutput[String, String](sinkTopic)
^
/home/confluent/kafka/streams/streams-scala/src/test/scala/org/apache/kafka/streams/scala/kstream/KStreamTest.scala:263:
not found: value Instant
val now = Instant.now()
^
/home/confluent/kafka/streams/streams-scala/src/test/scala/org/apache/kafka/streams/scala/kstream/KStreamTest.scala:265:
value createInput is not a member of
org.apache.kafka.streams.TopologyTestDriver
val testInput = testDriver.createInput[String, String](sourceTopic)
^
/home/confluent/kafka/streams/streams-scala/src/test/scala/org/apache/kafka/streams/scala/kstream/KStreamTest.scala:266:
value createOutput is not a member of
org.apache.kafka.streams.TopologyTestDriver
val testOutput = testDriver.createOutput[String, String](sinkTopic)
^
/home/confluent/kafka/streams/streams-scala/src/test/scala/org/apache/kafka/streams/scala/kstream/KStreamTest.scala:296:
not found: value Instant
val now = Instant.now()
^
/home/confluent/kafka/streams/streams-scala/src/test/scala/org/apache/kafka/streams/scala/kstream/KStreamTest.scala:298:
value createInput is not a member of
org.apache.kafka.streams.TopologyTestDriver
val testInput = testDriver.createInput[String, String](sourceTopic)
^
/home/confluent/kafka/streams/streams-scala/src/test/scala/org/apache/kafka/streams/scala/kstream/KStreamTest.scala:299:
value createOutput is not a member of
org.apache.kafka.streams.TopologyTestDriver
val testOutput = testDriver.createOutput[String, String](sinkTopic)
^
12 errors found
{noformat}
> Add KStream#flatTransform and KStream#flatTransformValues to Scala API
> --
>
> Key: KAFKA-9011
> URL: https://issues.apache.org/jira/browse/KAFKA-9011
> Project: Kafka
> Issue Type: Improvement
> Components: streams
>Affects Versions: 2.3.0
>Reporter: Alex Kokachev
>Assignee: Alex Kokachev
>Priority: Major
> Labels: scala, streams
> Fix For: 2.5.0
>
>
> Part of KIP-313:
> [https://cwiki.apache.org/confluence/display/KAFKA/KIP-313%3A+Add+KStream.flatTransform+and+KStream.flatTransformValues]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-9011) Add KStream#flatTransform and KStream#flatTransformValues to Scala API
[ https://issues.apache.org/jira/browse/KAFKA-9011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16999649#comment-16999649 ] Alex Kokachev commented on KAFKA-9011: -- [~bbejeck], [~vvcephei], could you guys please cherry-pick commits from PRs [https://github.com/apache/kafka/pull/7685] and [https://github.com/apache/kafka/pull/7520] into 2.3 and 2.4 branches? Thanks. > Add KStream#flatTransform and KStream#flatTransformValues to Scala API > -- > > Key: KAFKA-9011 > URL: https://issues.apache.org/jira/browse/KAFKA-9011 > Project: Kafka > Issue Type: Improvement > Components: streams >Affects Versions: 2.3.0 >Reporter: Alex Kokachev >Assignee: Alex Kokachev >Priority: Major > Labels: scala, streams > Fix For: 2.5.0 > > > Part of KIP-313: > [https://cwiki.apache.org/confluence/display/KAFKA/KIP-313%3A+Add+KStream.flatTransform+and+KStream.flatTransformValues] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-9011) Add KStream#flatTransform and KStream#flatTransformValues to Scala API
[ https://issues.apache.org/jira/browse/KAFKA-9011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974604#comment-16974604 ] ASF GitHub Bot commented on KAFKA-9011: --- bbejeck commented on pull request #7685: KAFKA-9011: Removed multiple calls to supplier.get() URL: https://github.com/apache/kafka/pull/7685 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add KStream#flatTransform and KStream#flatTransformValues to Scala API > -- > > Key: KAFKA-9011 > URL: https://issues.apache.org/jira/browse/KAFKA-9011 > Project: Kafka > Issue Type: Improvement > Components: streams >Affects Versions: 2.3.0 >Reporter: Alex Kokachev >Assignee: Alex Kokachev >Priority: Major > Labels: scala, streams > Fix For: 2.5.0 > > > Part of KIP-313: > [https://cwiki.apache.org/confluence/display/KAFKA/KIP-313%3A+Add+KStream.flatTransform+and+KStream.flatTransformValues] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-9011) Add KStream#flatTransform and KStream#flatTransformValues to Scala API
[ https://issues.apache.org/jira/browse/KAFKA-9011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972957#comment-16972957 ] ASF GitHub Bot commented on KAFKA-9011: --- kokachev commented on pull request #7685: KAFKA-9011: Removed multiple calls to supplier.get() URL: https://github.com/apache/kafka/pull/7685 This is a followup PR to #7520 to address issue of multiple calls to get() as it was pointed out by @bbejeck in https://github.com/apache/kafka/pull/7520#discussion_r345374820 @vvcephei *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of testing strategy (including rationale) for the feature or bug fix. Unit and/or integration tests are expected for any behaviour change and system tests should be considered for larger changes.* ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add KStream#flatTransform and KStream#flatTransformValues to Scala API > -- > > Key: KAFKA-9011 > URL: https://issues.apache.org/jira/browse/KAFKA-9011 > Project: Kafka > Issue Type: Improvement > Components: streams >Affects Versions: 2.3.0 >Reporter: Alex Kokachev >Assignee: Alex Kokachev >Priority: Major > Labels: scala, streams > Fix For: 2.5.0 > > > Part of KIP-313: > [https://cwiki.apache.org/confluence/display/KAFKA/KIP-313%3A+Add+KStream.flatTransform+and+KStream.flatTransformValues] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-9011) Add KStream#flatTransform and KStream#flatTransformValues to Scala API
[ https://issues.apache.org/jira/browse/KAFKA-9011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972568#comment-16972568 ] frid commented on KAFKA-9011: - bill Bejeck could take a look here please we are blocked https://stackoverflow.com/questions/58815493/bug-cannot-change-retention-bytes-of-changelog-kafka-topic > Add KStream#flatTransform and KStream#flatTransformValues to Scala API > -- > > Key: KAFKA-9011 > URL: https://issues.apache.org/jira/browse/KAFKA-9011 > Project: Kafka > Issue Type: Improvement > Components: streams >Affects Versions: 2.3.0 >Reporter: Alex Kokachev >Assignee: Alex Kokachev >Priority: Major > Labels: scala, streams > Fix For: 2.5.0 > > > Part of KIP-313: > [https://cwiki.apache.org/confluence/display/KAFKA/KIP-313%3A+Add+KStream.flatTransform+and+KStream.flatTransformValues] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-9011) Add KStream#flatTransform and KStream#flatTransformValues to Scala API
[ https://issues.apache.org/jira/browse/KAFKA-9011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972521#comment-16972521 ] ASF GitHub Bot commented on KAFKA-9011: --- bbejeck commented on pull request #7520: KAFKA-9011: Scala bindings for flatTransform and flatTransformValues in KStream URL: https://github.com/apache/kafka/pull/7520 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add KStream#flatTransform and KStream#flatTransformValues to Scala API > -- > > Key: KAFKA-9011 > URL: https://issues.apache.org/jira/browse/KAFKA-9011 > Project: Kafka > Issue Type: Improvement > Components: streams >Affects Versions: 2.3.0 >Reporter: Alex Kokachev >Assignee: Alex Kokachev >Priority: Major > Labels: scala, streams > > Part of KIP-313: > [https://cwiki.apache.org/confluence/display/KAFKA/KIP-313%3A+Add+KStream.flatTransform+and+KStream.flatTransformValues] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-9011) Add KStream#flatTransform and KStream#flatTransformValues to Scala API
[ https://issues.apache.org/jira/browse/KAFKA-9011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16954798#comment-16954798 ] Matthias J. Sax commented on KAFKA-9011: Thanks for the PR [~akokachev] – I added you to the list of contributors and assigned the ticket to you. You can now also self-assign tickets. > Add KStream#flatTransform and KStream#flatTransformValues to Scala API > -- > > Key: KAFKA-9011 > URL: https://issues.apache.org/jira/browse/KAFKA-9011 > Project: Kafka > Issue Type: Improvement > Components: streams >Affects Versions: 2.3.0 >Reporter: Alex Kokachev >Assignee: Alex Kokachev >Priority: Major > Labels: scala, streams > > Part of KIP-313: > [https://cwiki.apache.org/confluence/display/KAFKA/KIP-313%3A+Add+KStream.flatTransform+and+KStream.flatTransformValues] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-9011) Add KStream#flatTransform and KStream#flatTransformValues to Scala API
[ https://issues.apache.org/jira/browse/KAFKA-9011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16953583#comment-16953583 ] Alex Kokachev commented on KAFKA-9011: -- [~cadonna], raised PR [https://github.com/apache/kafka/pull/7520] > Add KStream#flatTransform and KStream#flatTransformValues to Scala API > -- > > Key: KAFKA-9011 > URL: https://issues.apache.org/jira/browse/KAFKA-9011 > Project: Kafka > Issue Type: Improvement > Components: streams >Affects Versions: 2.3.0 >Reporter: Alex Kokachev >Priority: Major > Labels: scala, streams > > Part of KIP-313: > [https://cwiki.apache.org/confluence/display/KAFKA/KIP-313%3A+Add+KStream.flatTransform+and+KStream.flatTransformValues] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-9011) Add KStream#flatTransform and KStream#flatTransformValues to Scala API
[ https://issues.apache.org/jira/browse/KAFKA-9011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951662#comment-16951662 ] ASF GitHub Bot commented on KAFKA-9011: --- kokachev commented on pull request #7520: KAFKA-9011: Scala bindings for flatTransform and flatTransformValues in KStream URL: https://github.com/apache/kafka/pull/7520 @cadonna *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of testing strategy (including rationale) for the feature or bug fix. Unit and/or integration tests are expected for any behaviour change and system tests should be considered for larger changes.* ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add KStream#flatTransform and KStream#flatTransformValues to Scala API > -- > > Key: KAFKA-9011 > URL: https://issues.apache.org/jira/browse/KAFKA-9011 > Project: Kafka > Issue Type: Improvement > Components: streams >Affects Versions: 2.3.0 >Reporter: Alex Kokachev >Priority: Major > Labels: scala, streams > > Part of KIP-313: > [https://cwiki.apache.org/confluence/display/KAFKA/KIP-313%3A+Add+KStream.flatTransform+and+KStream.flatTransformValues] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-9011) Add KStream#flatTransform and KStream#flatTransformValues to Scala API
[ https://issues.apache.org/jira/browse/KAFKA-9011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16949661#comment-16949661 ] Bruno Cadonna commented on KAFKA-9011: -- [~akokachev] Yes, please feel free to open a PR. I will be glad to review it. > Add KStream#flatTransform and KStream#flatTransformValues to Scala API > -- > > Key: KAFKA-9011 > URL: https://issues.apache.org/jira/browse/KAFKA-9011 > Project: Kafka > Issue Type: Improvement > Components: streams >Affects Versions: 2.3.0 >Reporter: Alex Kokachev >Priority: Major > Labels: scala, streams > > Part of KIP-313: > [https://cwiki.apache.org/confluence/display/KAFKA/KIP-313%3A+Add+KStream.flatTransform+and+KStream.flatTransformValues] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-9011) Add KStream#flatTransform and KStream#flatTransformValues to Scala API
[
https://issues.apache.org/jira/browse/KAFKA-9011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16949002#comment-16949002
]
Alex Kokachev commented on KAFKA-9011:
--
I came up with the code below. If it's acceptable, happy to raise a PR. I'm
fairly new to Scala, so open to any feedback.
KStream.scala:
{noformat}
def flatTransform[K1, V1](transformerSupplier: TransformerSupplier[K, V,
Iterable[KeyValue[K1, V1]]],
stateStoreNames: String*): KStream[K1, V1] =
inner.flatTransform(transformerSupplier.asIterable, stateStoreNames:
_*){noformat}
FucntionsCompatConversions.scala:
{code:java}
implicit class IterableTransformerSupplier[K, V, VO](val supplier :
TransformerSupplier[K, V, Iterable[VO]] )
extends AnyVal {
def asIterable: TransformerSupplier[K, V, JIterable[VO]] = new
TransformerSupplier[K, V, JIterable[VO]] {
override def get(): Transformer[K, V, JIterable[VO]] = {
new Transformer[K, V, JIterable[VO]] {
override def transform(key: K, value: V): JIterable[VO] =
supplier.get().transform(key, value).asJava
override def init(context: ProcessorContext): Unit =
supplier.get().init(context)
override def close(): Unit = supplier.get().close()
}
}
}
}
{code}
> Add KStream#flatTransform and KStream#flatTransformValues to Scala API
> --
>
> Key: KAFKA-9011
> URL: https://issues.apache.org/jira/browse/KAFKA-9011
> Project: Kafka
> Issue Type: Improvement
> Components: streams
>Affects Versions: 2.3.0
>Reporter: Alex Kokachev
>Priority: Major
> Labels: scala, streams
>
> Part of KIP-313:
> [https://cwiki.apache.org/confluence/display/KAFKA/KIP-313%3A+Add+KStream.flatTransform+and+KStream.flatTransformValues]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
