GitHub user mbalassi opened a pull request:
https://github.com/apache/incubator-flink/pull/72
Streaming addon prototype
Support for low latency streaming jobs.
The API provided is fairly similar to the batch one, for example see our
[WordCount](https://github.com/mbalassi/incubator-flink/blob/streaming-ready/flink-addons/flink-streaming/flink-streaming-examples/src/main/java/org/apache/flink/streaming/examples/wordcount/WordCountLocal.java).
Legal issues were extensively discussed on the mailing list.
Further work is needed for fault-tolerance, state management and cluster
performance optimization.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mbalassi/incubator-flink streaming-ready
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-flink/pull/72.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #72
----
commit 64580abe83c284bc82407f8079d18ad2caf9040e
Author: Yingjun Wu <[email protected]>
Date: 2014-07-14T14:29:11Z
[streaming] minor bug fixed
commit ed35783978f327667708c61984479e164ca67511
Author: gyfora <[email protected]>
Date: 2014-07-14T14:29:11Z
[streaming] basic RabbitMQ tolopolgy add to test future support for RabbitMQ
commit 9d6f69505d8bc67e9fe4a8601cc0653068424778
Author: gaborhermann <[email protected]>
Date: 2014-07-14T14:29:11Z
[streaming] FaultTolerance set via JobGraphBuilder config
commit 37adc7c7ed1e08f919dad077ca206e22817ee5d7
Author: gaborhermann <[email protected]>
Date: 2014-07-14T14:29:11Z
[streaming] Merged RMQTopology
commit 27dcac7a859e5037222c45ea18ecc7f7e8204a4b
Author: gyfora <[email protected]>
Date: 2014-07-14T14:29:11Z
[streaming] added support for udf objects in jobgraphbuilder
commit 05ef21e6e64b589e452a7b8e82bbca72bc7f809a
Author: gyfora <[email protected]>
Date: 2014-07-14T14:29:11Z
[streaming] JavaDoc and api update
commit f5c1eb0cc535da69f29544478859c5c878f6db20
Author: ghermann <[email protected]>
Date: 2014-07-14T14:29:11Z
[streaming] Fixed double logging, added FieldTypeMismatchException
commit dbc3be1971f1c53e3da2cc9c8030a32fbea4b849
Author: gyfora <[email protected]>
Date: 2014-07-14T14:29:12Z
[streaming] api refactor
commit 7c668966b7e4d793992a3ac7e43f6ee4584aa977
Author: Yingjun Wu <[email protected]>
Date: 2014-07-14T14:29:12Z
[streaming] add clear function for StreamRecord, add framework of
checkpointer.
commit e795794b9390d865d12cc944771c0670903405cc
Author: Márton Balassi <[email protected]>
Date: 2014-07-14T14:29:12Z
[streaming] License fix
commit 01b8e3ebd5a2c73b6d98b3c0c17d1fee4ee6abff
Author: Márton Balassi <[email protected]>
Date: 2014-07-14T14:29:12Z
[streaming] Version set to 0.5
commit 17f6e9ae40762a963e4b7675722b7a6835c8f45f
Author: gyfora <[email protected]>
Date: 2014-07-14T14:29:12Z
[streaming] api cleanup
commit 39f4e64ffa94e35e239dde7586258872ba9a31b4
Author: Yingjun Wu <[email protected]>
Date: 2014-07-14T14:29:12Z
[streaming] add frameworks for iterative computation.
commit f224993173bbd8974ae2b7b35f367fe168e22429
Author: Yingjun Wu <[email protected]>
Date: 2014-07-14T14:29:12Z
[streaming] add stream join and stream window join example, refactor window
state
commit 8fb83534d14a3c1de29b2fd6a6ac0ce851d4eb0a
Author: Yingjun Wu <[email protected]>
Date: 2014-07-14T14:29:12Z
[streaming] add license & implement iterative processing
commit c67db12c2aa83745fc29e4a71f6b66b38b322719
Author: Márton Balassi <[email protected]>
Date: 2014-07-14T14:29:12Z
[streaming] Version set to 0.5
commit d8271fa429686b019352059604ba92d325a5dcf2
Author: Yingjun Wu <[email protected]>
Date: 2014-07-14T14:29:12Z
[streaming] add kafka topology
commit 693e338949beeee6ebb462e83c6bcf549c1cf924
Author: ghermann <[email protected]>
Date: 2014-07-14T14:29:12Z
[streaming] Replaced List<RecordReader> with UnionRecordReader
commit c8f2e151b4f874d88d619ab76a9d8a6313340c3d
Author: ghermann <[email protected]>
Date: 2014-07-14T14:29:13Z
[streaming] Refactored StreamComponentHelper and StreamComponentTest
commit 697a747f9c832b395b94c2b044a0ec4b4607eaf9
Author: ghermann <[email protected]>
Date: 2014-07-14T14:29:13Z
[streaming] Refactored logging to avoid expensive String concatenation
commit d0943c1e5f208cf6a980efc472ba2355979391a8
Author: gyfora <[email protected]>
Date: 2014-07-14T14:29:13Z
[streaming] branch added for new api
commit f9b3c12357c3b76488e39cced160b06fd6b20193
Author: gyfora <[email protected]>
Date: 2014-07-14T14:29:13Z
[streaming] StreamCollector added
commit baca005ee950f919c5ee379167f2bcc482209b7f
Author: gyfora <[email protected]>
Date: 2014-07-14T14:29:13Z
[streaming] streamrecord reworked
commit eb35e02e15d515ca8505423cee62a562d4aa4e36
Author: ghermann <[email protected]>
Date: 2014-07-14T14:29:13Z
[streaming] datastream added
commit 292ca5f3f1cc024f72b7e194e8c69634a3071628
Author: Márton Balassi <[email protected]>
Date: 2014-07-14T14:29:13Z
[streaming] Project version set to 0.1
commit 3050b6f1d0aa4fbd1c77dcaea98952831112139b
Author: Márton Balassi <[email protected]>
Date: 2014-07-14T14:29:13Z
[streaming] Project version set to 0.2-SNAPSHOT
commit 019ff793f949fbf9fcafa52e0e3b6885374d3e27
Author: gyfora <[email protected]>
Date: 2014-07-14T14:29:13Z
[streaming] array based streamrecord added
commit cea6d66511046f3d1d2f4947d1c73ede5a33719b
Author: gyfora <[email protected]>
Date: 2014-07-14T14:29:13Z
[streaming] license added
commit 5e510e3dca98d2f426cb9c1409f9ce8f08851037
Author: gyfora <[email protected]>
Date: 2014-07-14T14:29:13Z
[streaming] Standard and Union StreamRecordReader added for improved
serialization, not yet added to StreamTask and Sink
commit 27a8f6ae7a78a1d57c8a14e74fcf0f2293bfe3c3
Author: gyfora <[email protected]>
Date: 2014-07-14T14:29:14Z
[streaming] created abstract streamrecord class + added StreamRecordReader
and UnionStreamRecordReader to streamcomponenthelper
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
