GitHub user gyfora opened a pull request:
https://github.com/apache/incubator-flink/pull/275
Streaming Scala API
This PR contains the commits for the Scala api for Flink Streaming. Most
functionality is already implemented and I think it is ready to merge after
maybe some slight refactoring.
I would appreciate if someone with more scala experience could look it
through, that code works but it might not be very pretty everywhere.
Also If anyone can please run a scala formatter on the code since it seems
impossible to get line wrapping settings in eclipse so I had to break the lines
manually to make travis pass.
What's missing:
- groupBy("field") operator for case classes. The streaming api currently
doesn't use the comparators but instead it extracts the keys from elements. I
need to figure this out for case classes.
- Connected data stream functionality, I will add this soon. This is
trivial.
- Proper scala code-style formatting, see above
- I am not sure about the package structure, any comments on how I did it?
Feel free to play around and test the features, I am sure that there will
be bugs to fix :)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mbalassi/incubator-flink scala-api
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-flink/pull/275.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #275
----
commit 4c65fcb45a3c23b6b67af82522c6fc43b27b9c87
Author: Gyula Fora <[email protected]>
Date: 2014-12-11T14:22:03Z
[scala] [streaming] Base functionality added for streaming scala api
commit 11a08ca6660ee426931f3bcfcc38f4ce1258c6b4
Author: Gyula Fora <[email protected]>
Date: 2014-12-11T23:12:49Z
[scala] [streaming] Extended scala data stream functionality to include
simple operators
commit cf63e578d33732f417309f32e161bf33b393e145
Author: Gyula Fora <[email protected]>
Date: 2014-12-13T00:08:08Z
[scala] [streaming] Finished scala StreamExecutionEnvrionment functionality
+ DataStream sinks + docs
commit f09a6d9754f90fac27c954c9ce7281b31e0c69c2
Author: mbalassi <[email protected]>
Date: 2014-12-15T13:15:35Z
[scala] [streaming] Added SplitDataStream functionality
Conflicts:
flink-scala/src/main/scala/org/apache/flink/api/scala/streaming/DataStream.scala
commit 40ec285ffac7659f6d2a6be4fe7868ee48899f1b
Author: Gyula Fora <[email protected]>
Date: 2014-12-15T15:21:00Z
[scala] [streaming] Windowing functionality added to scala api
commit fc8118dae8b6eebdebb7bb4125c04fb76ab5e282
Author: Gyula Fora <[email protected]>
Date: 2014-12-18T21:46:00Z
[scala] [streaming] Added support for iterative streams for scala api
commit 14ef43140391c0d2c467d4aa81c4e79d7c08b4dc
Author: Gyula Fora <[email protected]>
Date: 2014-12-19T20:05:01Z
[streaming] Temporal join and cross rework for consistence and extended
features
commit 9448335378341b8ddc940bdf15075f20ede1e386
Author: Gyula Fora <[email protected]>
Date: 2014-12-20T17:12:19Z
[scala] [streaming] Temporal join operator added
commit fe3f9a97e1665decee5f5029e90adafe61aa7c27
Author: Gyula Fora <[email protected]>
Date: 2014-12-20T18:34:55Z
[scala] [streaming] Temporal cross operator added
commit 4dfd4a1c5efa20e75f166390c76fddf73cbba42b
Author: Gyula Fora <[email protected]>
Date: 2014-12-20T22:46:35Z
[scala] [streaming] Modified aggregations to work on scala tuples
commit 91fe7d8630676a71b0d1d9dd12d96fa9075362e3
Author: Gyula Fora <[email protected]>
Date: 2014-12-21T00:05:40Z
[scala] [streaming] Fixed scala formatting
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
