Release 0.9.0-milestone-1 preview

2015-04-03 Thread Robert Metzger
Hi All, As discussed on this list, we've decided to create a release outside the regular 3 monthly release schedule for the ApacheCon announcement and for giving our users a convenient way of trying out the great new features. This thread is not an official release vote. It is meant for testing t

Re: Release 0.9.0-milestone-1 preview

2015-04-03 Thread Fabian Hueske
Thanks Robert for pushing this forward. I'd like to have the following issues fixed in the release: - FLINK[1656] by PR #525 - FLINK[1776] by PR #532 - FLINK[1664] by PR #541 - FLINK[1817] by PR #565 - Failed tests on Windows by PR #491 Especially the first two fixes crucial. They address semanti

Re: Rework of the window-join semantics

2015-04-03 Thread Márton Balassi
That would be really neat, the problem I see there, that we do not distinguish between dataStream.window() and dataStream.window().every() currently, they both return WindowedDataStreams and TriggerPolicies of the every call do not make much sense in this setting (in fact practically the trigger is

Re: Rework of the window-join semantics

2015-04-03 Thread Gyula Fóra
I think it should be possible to make this compatible with the .window().every() calls. Maybe if there is some trigger set in "every" we would not join that stream 1 by 1 but every so many elements. The problem here is that the window and every in this case are very-very different than the normal w

NullPointerException in DeltaIteration when no ForwardedFileds annotation

2015-04-03 Thread Vasiliki Kalavri
Hello to my squirrels, I've been getting a NullPointerException for a DeltaIteration program I'm trying to implement and I could really use your help :-) It seems that some of the input Tuples of the Join operator that I'm using to create the next workset / solution set delta are null. It also see

Re: NullPointerException in DeltaIteration when no ForwardedFileds annotation

2015-04-03 Thread Fabian Hueske
That looks pretty much like a bug. As you said, fwd fields annotations are optional and may improve the performance of a program, but never change its semantics (if set correctly). I'll have a look at it later. Would be great if you could provide some data to reproduce the bug. On Apr 3, 2015 12:

Re: Test sources in wrong folder

2015-04-03 Thread Márton Balassi
Dear Flavio, 'mvn clean install -DskipTests' should do the trick. On Fri, Apr 3, 2015 at 12:11 AM, Flavio Pompermaier wrote: > Hi to all, > > I was trying to compile Flink 0.9 skipping test compilation > (-Dmaven.test.skip=true) but this is not possible because there are > projects like flink-

Re: NullPointerException in DeltaIteration when no ForwardedFileds annotation

2015-04-03 Thread Vasiliki Kalavri
Hi Fabian, I am using the dblp co-authorship dataset from SNAP: http://snap.stanford.edu/data/com-DBLP.html I also pushed my slightly modified version of ConnectedComponents, here: https://github.com/vasia/flink/tree/cc-test. It basically generates the vertex dataset from the edges, so that you do

[jira] [Created] (FLINK-1821) simplify coding

2015-04-03 Thread Liu Jiongzhou (JIRA)
Liu Jiongzhou created FLINK-1821: Summary: simplify coding Key: FLINK-1821 URL: https://issues.apache.org/jira/browse/FLINK-1821 Project: Flink Issue Type: Improvement Components:

Re: Test sources in wrong folder

2015-04-03 Thread Flavio Pompermaier
This is my problem maybe..I'm quite strict about maven usage :) SkipTests compiles the tests so you find the classes only because you compiled them. This is somehow wrong because that are test-scoped classes, I should not be forced to compile them if I'm not interested in testing at all. Don't you

[jira] [Created] (FLINK-1822) Update "Community" page on website with a "Getting Help" section

2015-04-03 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-1822: - Summary: Update "Community" page on website with a "Getting Help" section Key: FLINK-1822 URL: https://issues.apache.org/jira/browse/FLINK-1822 Project: Flink

[jira] [Created] (FLINK-1823) Rename linq.md to something like table.md to avoid potential trademark issue

2015-04-03 Thread Henry Saputra (JIRA)
Henry Saputra created FLINK-1823: Summary: Rename linq.md to something like table.md to avoid potential trademark issue Key: FLINK-1823 URL: https://issues.apache.org/jira/browse/FLINK-1823 Project: F

[jira] [Created] (FLINK-1824) Allow missing types for DataStreams

2015-04-03 Thread Gyula Fora (JIRA)
Gyula Fora created FLINK-1824: - Summary: Allow missing types for DataStreams Key: FLINK-1824 URL: https://issues.apache.org/jira/browse/FLINK-1824 Project: Flink Issue Type: Improvement

Re: Release 0.9.0-milestone-1 preview

2015-04-03 Thread Stephan Ewen
I think this thread if for the milestone release. Let's track the issues that must be included in 0.9 in a separate track. You can also mark them as release blockers in JIRA. On Fri, Apr 3, 2015 at 11:16 AM, Fabian Hueske wrote: > Thanks Robert for pushing this forward. > > I'd like to have the

Re: Release 0.9.0-milestone-1 preview

2015-04-03 Thread Fabian Hueske
All issues I listed are relevant for the milestone release, IMO. On Apr 3, 2015 6:27 PM, "Stephan Ewen" wrote: > I think this thread if for the milestone release. > > Let's track the issues that must be included in 0.9 in a separate track. > You can also mark them as release blockers in JIRA. > >

Re: Storm compatibility layer for Flink (first beta available)

2015-04-03 Thread Fabian Hueske
Right now, your commits in your working branch are mixed with commits which are already in pushed to the master branch. Merging this branch to the master branch in order to push it to our master might turn out into a complex merging process. Merging becomes far easier for us, if all commits that yo

Re: Storm compatibility layer for Flink (first beta available)

2015-04-03 Thread Robert Metzger
As far as I understood git rebase [1], cherry-picking all changes in order to the current master is exactly equal to "git rebase flink/master". The problem is that you have to resolve all conflicts again. But in this case the changes to existing code are pretty small, so that might actually work co

Re: NullPointerException in DeltaIteration when no ForwardedFileds annotation

2015-04-03 Thread Fabian Hueske
Thanks for the nice setup! I could easily reproduce the exception you are facing. But that's the only good news so far :-( I checked the plans and both are valid and should compute the correct result for the program. The split-of solution set delta is required because the it needs to be repartitio

[jira] [Created] (FLINK-1825) Fix the constructor comments with correct parameter in DefaultMemoryManager

2015-04-03 Thread Sibao Hong (JIRA)
Sibao Hong created FLINK-1825: - Summary: Fix the constructor comments with correct parameter in DefaultMemoryManager Key: FLINK-1825 URL: https://issues.apache.org/jira/browse/FLINK-1825 Project: Flink

[jira] [Created] (FLINK-1826) Remove the code never be executed in function getNumPages

2015-04-03 Thread Sibao Hong (JIRA)
Sibao Hong created FLINK-1826: - Summary: Remove the code never be executed in function getNumPages Key: FLINK-1826 URL: https://issues.apache.org/jira/browse/FLINK-1826 Project: Flink Issue Type:

Re: HBase TableOutputFormat fix (Flink 0.8.1)

2015-04-03 Thread Flavio Pompermaier
Any fix for this? On Apr 3, 2015 7:43 AM, "Flavio Pompermaier" wrote: