[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2016-04-06 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-206367839 Thanks @nikste for implementing! @StefanPapp We would be delighted if you tried out the new shell! --- If your project is set up for it, you can reply to this email

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2016-04-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/1412 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2016-04-06 Thread nikste
Github user nikste commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-206352670 Adjusted the welcome message now --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2016-04-06 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-206351430 Merging this with an adjustment of the welcome message. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If yo

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2016-04-06 Thread StefanPapp
Github user StefanPapp commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-206351050 I also would like to have this feature. Flink is more and more perceived as streaming framework that can do batch too if necessary. So having a batch execution enviro

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2016-04-06 Thread chiwanpark
Github user chiwanpark commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-206350336 Ah, right @rmetzger. We can go ahead. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2016-04-06 Thread rmetzger
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-206349631 @chiwanpark: I think you can register multiple email addresses with GitHub, so that they can associate all your commits (from different addresses) to you. --- If your

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2016-04-06 Thread chiwanpark
Github user chiwanpark commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-206345431 I just found the author email is not same as @nikste's github account. It is trivial but fixing email address would be better. --- If your project is set up for it,

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2016-04-06 Thread chiwanpark
Github user chiwanpark commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-206344388 Looks good to me. +1 I tested a simple streaming word count example with socket stream in following configuration: * Flink on Hadoop YARN 2.7.2 (both Scal

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2016-04-06 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-206343969 +1 to merge although I still think `senv` and `benv` will confuse people. One more thing before merging, we need to change the initial welcome message which explain

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2016-04-06 Thread rmetzger
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-206339995 I think the changes look good to merge. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2016-04-04 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-205285378 As you wish, I don't mind. I only felt it was hard to read and to keep apart. --- If your project is set up for it, you can reply to this email and have your reply appear o

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2016-04-04 Thread chiwanpark
Github user chiwanpark commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-205274226 I prefer `senv` and `benv` because it is shorter than `streamEnv` and `batchEnv`. In shell, shorter variables would be better even though there is a auto-complete sup

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2016-04-04 Thread nikste
Github user nikste commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-205271252 I was following @StephanEwen 's suggestion above here: > The Scala community seems to like brevity (senv vs. streamEnv). --- If your project is set up for it, you can

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2016-04-04 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-205267364 How about renaming `benv` to `batchEnv` and `senv` to `streamEnv`? Makes the two more distinct from each other and the names more self-explanatory. --- If your project is s

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2016-04-04 Thread nikste
Github user nikste commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-205234801 @mxm rebased to master and added documentation --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2016-04-04 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-205204935 @nikste Could you rebase to the latest master? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2016-04-04 Thread mxm
Github user mxm commented on a diff in the pull request: https://github.com/apache/flink/pull/1412#discussion_r58346629 --- Diff: flink-scala-shell/src/main/scala/org/apache/flink/api/scala/FlinkILoop.scala --- @@ -124,7 +127,9 @@ class FlinkILoop( "org.apache.flink.api.ja

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2016-03-31 Thread nikste
Github user nikste commented on a diff in the pull request: https://github.com/apache/flink/pull/1412#discussion_r58094009 --- Diff: flink-scala-shell/src/main/scala/org/apache/flink/api/scala/FlinkILoop.scala --- @@ -124,7 +127,9 @@ class FlinkILoop( "org.apache.flink.api

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2016-03-31 Thread nikste
Github user nikste commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-204038110 So since a lot has changed in the master I basically rewrote the whole thing. Seems to work now. What default imports should be included for the streaming API?

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2016-03-31 Thread rawkintrevo
Github user rawkintrevo commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-203953994 @nikste awesome work, any updates? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project do

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2016-02-15 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-184194014 @nikste Cool! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enab

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2016-02-15 Thread nikste
Github user nikste commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-184177781 @mxm @aljoscha i was on vacation for the last month, I'll continue working on this now.. --- If your project is set up for it, you can reply to this email and have your

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2016-01-29 Thread aljoscha
Github user aljoscha commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-176847731 @nikste Any updates on this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2016-01-18 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-172494058 By the way, sorry for the late response. I was traveling. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2016-01-18 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-172493786 The context environment is set in the `ScalaShellRemoteEnvironment` to disable creation of new environments by the user. This is done when you first create the batch environ

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2015-12-30 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-168099353 This is usually the message you get when using the {{Client}} to run a program. The context environment sets the flag that prohibits explicit local and remote enviro

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2015-12-21 Thread nikste
Github user nikste commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-166394233 Hey @mxm, So I've tried to initialize the scala shell with one and two different miniclusters and initialize the streaming and batch environment simultaneously. Howev

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2015-12-08 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-162886600 Great. Thank you @nikste! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2015-12-08 Thread nikste
Github user nikste commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-162843154 Ok, preferences here seem to be clear, I'll change it to make use of two environments simultaneously in the shell then. --- If your project is set up for it, you can rep

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2015-12-07 Thread chiwanpark
Github user chiwanpark commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-162804155 @nikste, Yes that is what I mean. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2015-12-07 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-162556835 I am a bit biased towards having two environments, like @chiwanpark and @mxm suggest. We could also keep calling the batch environment `env` and the stream

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2015-12-07 Thread chiwanpark
Github user chiwanpark commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-162529724 I also prefer multiple environments during startup of the shell. I'm adding YARN session support of the shell. If there is another startup mode, implementation could

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2015-12-07 Thread nikste
Github user nikste commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-162534231 Hey @mxm, my survey among two people was inconclusive. I can see both versions working, I'm more tending towards the version that is implemented now, I might be a bit bia

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2015-12-07 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-162528807 Hi @nikste. Pinging again :) What do you think about my suggestion? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2015-12-02 Thread mxm
Github user mxm commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-161318473 Does it have to be an extra startup mode? How about you instantiating both `ExecutionEnvironment` and `StreamExecutionEnvironment` during startup of the shell? We could bind

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2015-11-30 Thread nikste
Github user nikste commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-160659980 @rmetzger, thanks for the comments. Changed the code accordingly, also I've extended the documentation for the scala shell. --- If your project is set up for it, yo

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2015-11-30 Thread nikste
Github user nikste commented on a diff in the pull request: https://github.com/apache/flink/pull/1412#discussion_r46155833 --- Diff: flink-staging/flink-scala-shell/src/main/scala/org/apache/flink/api/scala/FlinkILoop.scala --- @@ -95,24 +140,49 @@ class FlinkILoop( new F

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2015-11-30 Thread nikste
Github user nikste commented on a diff in the pull request: https://github.com/apache/flink/pull/1412#discussion_r46155052 --- Diff: flink-staging/flink-scala-shell/src/main/java/org/apache/flink/api/java/ScalaShellRemoteStreamEnvironment.java --- @@ -0,0 +1,155 @@ +/* + *

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2015-11-30 Thread nikste
Github user nikste commented on a diff in the pull request: https://github.com/apache/flink/pull/1412#discussion_r46154962 --- Diff: flink-staging/flink-scala-shell/pom.xml --- @@ -84,6 +108,11 @@ under the License. test +

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2015-11-30 Thread rmetzger
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/1412#issuecomment-160648316 I had some questions for the pull request. Also, can you update the documentation according to the changes ( https://ci.apache.org/projects/flink/flink-docs-mas

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2015-11-30 Thread rmetzger
Github user rmetzger commented on a diff in the pull request: https://github.com/apache/flink/pull/1412#discussion_r46149535 --- Diff: flink-staging/flink-scala-shell/src/main/scala/org/apache/flink/api/scala/FlinkILoop.scala --- @@ -95,24 +140,49 @@ class FlinkILoop( new

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2015-11-30 Thread rmetzger
Github user rmetzger commented on a diff in the pull request: https://github.com/apache/flink/pull/1412#discussion_r46149442 --- Diff: flink-staging/flink-scala-shell/src/main/java/org/apache/flink/api/java/ScalaShellRemoteStreamEnvironment.java --- @@ -0,0 +1,155 @@ +/* +

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2015-11-30 Thread rmetzger
Github user rmetzger commented on a diff in the pull request: https://github.com/apache/flink/pull/1412#discussion_r46149305 --- Diff: flink-staging/flink-scala-shell/pom.xml --- @@ -84,6 +108,11 @@ under the License. test +

[GitHub] flink pull request: [FLINK-2522] Streaming support for Flink-Scala...

2015-11-30 Thread rmetzger
Github user rmetzger commented on a diff in the pull request: https://github.com/apache/flink/pull/1412#discussion_r46149356 --- Diff: flink-staging/flink-scala-shell/pom.xml --- @@ -84,6 +108,11 @@ under the License. test +