Re: Adding the streaming project to the main repository

2014-07-13 Thread Henry Saputra
Thanks for the update Robert. This needs some review so let's wait merging to master or any branch On Sunday, July 13, 2014, Robert Metzger wrote: > Regarding the dependencies, I found that they require "jblas", with this > license: https://github.com/mikiobraun/jblas/blob/master/COPYING > It se

Re: Adding the streaming project to the main repository

2014-07-13 Thread Robert Metzger
Regarding the dependencies, I found that they require "jblas", with this license: https://github.com/mikiobraun/jblas/blob/master/COPYING It seems to be a BSD license, which is compatible with ASF projects [1]. The connectors package depends on RabbitMQ, which is MPL Licensed: http://www.rabbitmq.

Re: Adding the streaming project to the main repository

2014-07-13 Thread Márton Balassi
Thanks for the effort. Sorry for the mess, I'll clean it up as soon as possible. Cheers, Marton On Sun, Jul 13, 2014 at 5:25 PM, Stephan Ewen wrote: > Hi everyone! > > I have found a way to add the code into the main repository in a different > branch, preserving all history. > All code is re

Re: Adding the streaming project to the main repository

2014-07-13 Thread Stephan Ewen
Hi everyone! I have found a way to add the code into the main repository in a different branch, preserving all history. All code is rewritten (with history) to be in "flink-addons/flink-streaming" and the commits are prefixed with [streaming]. https://github.com/StephanEwen/incubator-flink/commits

Re: Adding the streaming project to the main repository

2014-07-13 Thread Stephan Ewen
Good point! I will ping Marton and Gyula for that. On Sun, Jul 13, 2014 at 4:22 PM, Robert Metzger wrote: > Lets see if the variant with rewriting the history using git filter-branch > works better. > > > One other thing regarding the merge: > I'm not sure if we have to do any legal checks prio

Re: Adding the streaming project to the main repository

2014-07-13 Thread Robert Metzger
Lets see if the variant with rewriting the history using git filter-branch works better. One other thing regarding the merge: I'm not sure if we have to do any legal checks prior to merging the changes into our project. Maybe we even need a SGA or CCLA if the code has been written as part of an e

Re: Adding the streaming project to the main repository

2014-07-13 Thread Stephan Ewen
Okay, here is a try: https://github.com/StephanEwen/incubator-flink/tree/streaming/flink-addons/flink-streaming It attributes all files to my commit, but it preseves all authors in git blame. It is a bit strange, the history is broken, but some author information is preserved. Not ideal. Hope we

Re: Adding the streaming project to the main repository

2014-07-13 Thread Márton Balassi
Let us know if we can assist the merge in any way. On Sun, Jul 13, 2014 at 3:50 PM, Stephan Ewen wrote: > Okay. How do we do this, because it is cross-repository merge? I'll look > into Robert's referene... >

Re: Adding the streaming project to the main repository

2014-07-13 Thread Stephan Ewen
Okay, subtree merge looks promising: http://stackoverflow.com/questions/1425892/how-do-you-merge-two-git-repositories I'll give it a try...

Re: Adding the streaming project to the main repository

2014-07-13 Thread Stephan Ewen
Okay. How do we do this, because it is cross-repository merge? I'll look into Robert's referene...

Re: Code renamed and adjusted to Apache Incubator guidelines

2014-07-13 Thread Márton Balassi
Double-checked Stratosphere, you're correct. I also agree on legacy names, it is fine if they slowly but surely get eliminated - thus noone has the inconvenient task to deal with them once. :) On Sun, Jul 13, 2014 at 3:43 PM, Robert Metzger wrote: > I think the only "Stratosphere" references l

Re: Adding the streaming project to the main repository

2014-07-13 Thread Márton Balassi
Thanks, Stefan & Robert. We'd definitely vote for merging with history as we've invested 4 months of work to reach the current stage. It is also benefitial for Flink as the merge will add 7 contributors to the project then. On Sun, Jul 13, 2014 at 3:38 PM, Robert Metzger wrote: > I think it is

Re: Code renamed and adjusted to Apache Incubator guidelines

2014-07-13 Thread Robert Metzger
I think the only "Stratosphere" references left in the code are URLs to the old Github Account or the documentation for the 0.4 release. The quickstart.sh script also still points to the 0.5.1 release, because it always points to the last stable release. If you have found references other than that

Re: Adding the streaming project to the main repository

2014-07-13 Thread Robert Metzger
I think it is also possible to merge the streaming project keeping its history: http://git-scm.com/book/en/Git-Tools-Subtree-Merging. I saw this recently in Optiq's JIRA. They are doing something like: git subtree add --prefix=example-csv https://github.com/julianhyde/optiq-csv.git master On S

Re: Adding the streaming project to the main repository

2014-07-13 Thread Stephan Ewen
Hi folks! I have made a version that added the code to the flink repository. The thing is: all code is attributed to me (as the one who added the files). If you do not mind, I can commit it like that. If you want to code to be attributed to you, you need to make a pull request that puts the cont

Re: [2/3] git commit: [FLINK-1018] Add tests to verify correct placement of pipeline breakers with broadcast variables

2014-07-13 Thread Stephan Ewen
Okay, I'll wait with merging the streaming code until the rat is updated.

Re: [2/3] git commit: [FLINK-1018] Add tests to verify correct placement of pipeline breakers with broadcast variables

2014-07-13 Thread Robert Metzger
I'm currently preparing a commit with some more license fixes and a strict rat configuration. On Sun, Jul 13, 2014 at 2:58 PM, Stephan Ewen wrote: > Ah, good point. They com from old pull requests. > > Let me know if you manage to configure rat stricter. >

Re: [2/3] git commit: [FLINK-1018] Add tests to verify correct placement of pipeline breakers with broadcast variables

2014-07-13 Thread Stephan Ewen
Ah, good point. They com from old pull requests. Let me know if you manage to configure rat stricter.

Re: Code renamed and adjusted to Apache Incubator guidelines

2014-07-13 Thread Stephan Ewen
Hi Marton! I agree, cleaning up legacy names is desirable. I tend to do that on the fly, in classes that I work on. A dedicated swipe would take quite a lot of time, so we have not gotten around to that so far. Greetings, Stephan

Re: [2/3] git commit: [FLINK-1018] Add tests to verify correct placement of pipeline breakers with broadcast variables

2014-07-13 Thread Robert Metzger
The two files you committed here "PipelineBreakerTest.java" and "SelectOneReducer.java" both contain the old license header. I'll try and see if I can make the license checker more strict. On Sat, Jul 12, 2014 at 7:32 PM, wrote: > [FLINK-1018] Add tests to verify correct placement of pipeline b

Re: Code renamed and adjusted to Apache Incubator guidelines

2014-07-13 Thread Márton Balassi
A file search of "Stratosphere" on "*java" files in Eclipse still produces 36 hits. (They might be duplicated by the way as both flink-parent and flink- were included...) Another related issue that might be worth to consider: I've noticed that you've eliminated "nephele" from the package naming co