[jira] [Commented] (KAFKA-3121) KStream DSL API Improvement
[ https://issues.apache.org/jira/browse/KAFKA-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15128915#comment-15128915 ] ASF GitHub Bot commented on KAFKA-3121: --- Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/839 > KStream DSL API Improvement > --- > > Key: KAFKA-3121 > URL: https://issues.apache.org/jira/browse/KAFKA-3121 > Project: Kafka > Issue Type: Sub-task >Reporter: Guozhang Wang >Assignee: Guozhang Wang > Fix For: 0.9.1.0 > > > From some collected feedbacks, here is a list of potential improvements that > we want to make: > 1. Remove AggregateSupplier for aggregate, and provide more built-in > aggregations. > 2. Join to return KeyValue<>. > 3. "Windows" class syntax-sugers. > 4. Add print() to KTable / KStream. > 5. flatMap / flatMapValues to return arrays in addition to Iterable. > 6. make the API function names aligned with Java 8+, e.g. filterOut -> > filterNot > 7. collapse process() and transform() in KStream. > 8. validate Streaming configs and allow passing properties to KafkaStreaming. > [DONE] > 9. Rename KafkaStreaming to Streams. [DONE] > Also move some of the state package into internals, and create a new > top-level common folder with KeyValue / etc in it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-3121) KStream DSL API Improvement
[ https://issues.apache.org/jira/browse/KAFKA-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15124460#comment-15124460 ] ASF GitHub Bot commented on KAFKA-3121: --- GitHub user guozhangwang opened a pull request: https://github.com/apache/kafka/pull/839 KAFKA-3121: Refactor KStream Aggregate to be Lambda-able. You can merge this pull request into a Git repository by running: $ git pull https://github.com/guozhangwang/kafka K3121s2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/839.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #839 commit bf4c4cb3dbb5b4066d9c3e0ada5b7ffd98eb129a Author: Guozhang Wang Date: 2016-01-14T20:27:58Z add internal source topic for tracking commit 1485dff08a76c6ff685b0fe72226ce3b629b1d3c Author: Guozhang Wang Date: 2016-01-14T22:32:08Z minor fix for this.interSourceTopics commit 60cafd0885c41f93e408f8d89880187ddec789a1 Author: Guozhang Wang Date: 2016-01-15T01:09:00Z add KStream windowed aggregation commit 983a626008d987828deabe45d75e26e909032843 Author: Guozhang Wang Date: 2016-01-15T01:34:56Z merge from apache trunk commit 57051720de4238feb4dc3c505053096042a87d9c Author: Guozhang Wang Date: 2016-01-15T21:38:53Z v1 commit 4a49205fcab3a05ed1fd05a34c7a9a92794b992d Author: Guozhang Wang Date: 2016-01-15T22:07:17Z minor fix on HoppingWindows commit 9b4127e91c3a551fb655155d9b8e0df50132d0b7 Author: Guozhang Wang Date: 2016-01-15T22:43:14Z fix HoppingWindows commit 9649fe5c8a9b2e900e7746ae7b8745bb65694583 Author: Guozhang Wang Date: 2016-01-16T19:00:54Z add retainDuplicate option in RocksDBWindowStore commit 8a9ea02ac3f9962416defa79d16069431063eac0 Author: Guozhang Wang Date: 2016-01-16T19:06:12Z minor fixes commit 4123528cf4695b05235789ebfca3a63e8a832ffa Author: Guozhang Wang Date: 2016-01-18T17:55:02Z Merge branch 'trunk' of https://git-wip-us.apache.org/repos/asf/kafka into K3104 commit 46e8c8d285c0afae6da9ec7437d082060599f3f1 Author: Guozhang Wang Date: 2016-01-18T19:15:47Z add wordcount and pipe jobs commit 582d3ac24bfe08edb1c567461971cd35c1f75a00 Author: Guozhang Wang Date: 2016-01-18T21:53:21Z merge from trunk commit 5a002fadfcf760627274ddaa016deeaed5a3199f Author: Guozhang Wang Date: 2016-01-19T00:06:34Z 1. WallClockTimestampExtractor as default; 2. remove windowMS config; 3. override state dir with jobId prefix; commit 7425673e523c42806b29a364564a747443712a53 Author: Guozhang Wang Date: 2016-01-19T01:26:11Z Add PageViewJob commit ca04ba8d18674c521ad67872562a7671cb0e2c0d Author: Guozhang Wang Date: 2016-01-19T06:23:05Z minor changes on topic names commit 563cc546b3a0dd16d586d2df33c37d2c5a5bfb18 Author: Guozhang Wang Date: 2016-01-19T21:30:11Z change config importance levels commit 4218904505363e61bb4c6b60dc5b13badfd39697 Author: Guozhang Wang Date: 2016-01-21T00:11:34Z Merge branch 'trunk' of https://git-wip-us.apache.org/repos/asf/kafka into K3066 commit 26fb5f3f5a8c9b304c5b1e61778c6bc1d9d5fccb Author: Guozhang Wang Date: 2016-01-21T06:43:04Z demo examples v1 commit 6d92a55d770e058183daabb7aaef7675335fbbad Author: Guozhang Wang Date: 2016-01-22T00:41:12Z Merge branch 'trunk' of https://git-wip-us.apache.org/repos/asf/kafka into K3066 commit 929e405058eb61d38510120f3f3ed50cd0cfaf47 Author: Guozhang Wang Date: 2016-01-22T01:02:04Z add RollingRocksDBStore commit 324eb584b97ed3c228347d108d697d2f5133ea99 Author: Guozhang Wang Date: 2016-01-22T01:23:32Z modify MeteredWindowStore commit 7ba2d90fe1de1ca776cea23ff1c2e8f8b3a6c3f2 Author: Guozhang Wang Date: 2016-01-22T01:35:10Z remove getter commit a4d78bac9d84dfd1c7dab4ae465b9115ddc451b3 Author: Guozhang Wang Date: 2016-01-22T01:36:51Z remove RollingRocksDB commit d0e8198ac6a25315d7ab8d21894acf0077f88fde Author: Guozhang Wang Date: 2016-01-22T17:24:32Z adding cache layer on RocksDB commit 257b53d3b6df967f8a015a06c9e178d4219d0f8c Author: Guozhang Wang Date: 2016-01-22T23:15:08Z dummy commit 25fd73107c577ac2e4b32300d4fe132ad7ff7312 Author: Guozhang Wang Date: 2016-01-22T23:21:29Z merge from trunk commit e57abf3c117d4ea7398c6157983d75443194ce9f Author: Guozhang Wang Date: 2016-01-22T23:29:59Z merge from demo example changes again commit a0caee14d4a7b77045d9813a1072143db0fc8fa1 Author: Guozhang Wang Date: 2016-01-22T23:50:19Z resolve conflicts commit 5501e6fef16fdef55cb2136d8c3f8035d394d79f Author: Guozhang Wang Date: 2016-01-26T17:12:12Z Merge branch 'trunk' of https://git-wip-us.apache.org/repos/asf/kafka into K3060 commit d73b594147d1b4e25a61497550f498b9f14822a0 Author: Guozhang Wang Date: 2
[jira] [Commented] (KAFKA-3121) KStream DSL API Improvement
[ https://issues.apache.org/jira/browse/KAFKA-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15109790#comment-15109790 ] ASF GitHub Bot commented on KAFKA-3121: --- Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/795 > KStream DSL API Improvement > --- > > Key: KAFKA-3121 > URL: https://issues.apache.org/jira/browse/KAFKA-3121 > Project: Kafka > Issue Type: Sub-task >Reporter: Guozhang Wang >Assignee: Guozhang Wang > > From some collected feedbacks, here is a list of potential improvements that > we want to make: > 1. Remove AggregateSupplier for aggregate, and provide more built-in > aggregations. > 2. Join to return KeyValue<>. > 3. "Windows" class syntax-sugers. > 4. Add print() to KTable / KStream. > 5. flatMap / flatMapValues to return arrays in addition to Iterable. > 6. make the API function names aligned with Java 8+, e.g. filterOut -> > filterNot > 7. collapse process() and transform() in KStream. > 8. validate Streaming configs and allow passing properties to KafkaStreaming. > 9. Rename KafkaStreaming to Streams. > Also move some of the state package into internals, and create a new > top-level common folder with KeyValue / etc in it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-3121) KStream DSL API Improvement
[ https://issues.apache.org/jira/browse/KAFKA-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15109381#comment-15109381 ] ASF GitHub Bot commented on KAFKA-3121: --- GitHub user guozhangwang opened a pull request: https://github.com/apache/kafka/pull/795 KAFKA-3121: Remove aggregatorSupplier and add Reduce functions You can merge this pull request into a Git repository by running: $ git pull https://github.com/guozhangwang/kafka K3121s1 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/795.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #795 commit 10837b24c7a8af0e2843a8ae2102610dc28aff3c Author: Guozhang Wang Date: 2016-01-20T02:48:07Z refactor aggregate and add reduce commit c8fe67f78809276b2c9aa3ca4f36d64b79483c1a Author: Guozhang Wang Date: 2016-01-20T06:48:19Z add new files commit ad90555eee8ce7de799a258c175522a95cafb372 Author: Guozhang Wang Date: 2016-01-20T20:52:24Z KAFKA-3121: Remove AggregateSupplier for aggregate and add reduce functions > KStream DSL API Improvement > --- > > Key: KAFKA-3121 > URL: https://issues.apache.org/jira/browse/KAFKA-3121 > Project: Kafka > Issue Type: Sub-task >Reporter: Guozhang Wang >Assignee: Guozhang Wang > > From some collected feedbacks, here is a list of potential improvements that > we want to make: > 1. Remove AggregateSupplier for aggregate, and provide more built-in > aggregations. > 2. Join to return KeyValue<>. > 3. "Windows" class syntax-sugers. > 4. Add print() to KTable / KStream. > 5. flatMap / flatMapValues to return arrays in addition to Iterable. > 6. make the API function names aligned with Java 8+, e.g. filterOut -> > filterNot > 7. collapse process() and transform() in KStream. > 8. validate Streaming configs and allow passing properties to KafkaStreaming. > 9. Rename KafkaStreaming to Streams. > Also move some of the state package into internals, and create a new > top-level common folder with KeyValue / etc in it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)