Re: Building Flink for Hadoop 2

2014-12-16 Thread Henry Saputra
Ah yes, I vague remember that patch. Thanks for the reminder. - Henry On Tue, Dec 16, 2014 at 10:24 PM, Chiwan Park wrote: > Hi. > > After 0.8-incubating, the default profile of hadoop is 2. > (https://issues.apache.org/jira/browse/FLINK-1234 >

Re: Building Flink for Hadoop 2

2014-12-16 Thread Chiwan Park
Hi. After 0.8-incubating, the default profile of hadoop is 2. (https://issues.apache.org/jira/browse/FLINK-1234 ) So, you don’t need to set hadoop 2 profile to build Flink for hadoop 2. I tested with `mvn clean package -DskipTests` and the build

Building Flink for Hadoop 2

2014-12-16 Thread Henry Saputra
HI All, I just realized that to build Flink for Hadoop 2 I could no longer run this: mvn clean package -DskipTests -Dhadoop.profile=2 it barfed with error: [INFO] Scanning for projects... [ERROR] The build could not read 1 project -> [Help 1] [ERROR] [ERROR] The project org.apache.flink:fli

Re: Master bumped to 0.9

2014-12-16 Thread Márton Balassi
Robert's Kryo PR is the last outstanding feature patch. I'd like to stick to bugfixing that pops up during the tests. Speaking of which today we've had a look at the streaming code with Gyula for the release and produced a couple of commits. We will manually rebase anything else that is needed for

Re: Master bumped to 0.9

2014-12-16 Thread Henry Saputra
Nice! So do we have official list of JIRA and PRs will be in 0.8? I do not see the conclusions in the other thread. Does this mean no more patches for 0.8 release or you and Marton will manually rebase them? - Henry On Tue, Dec 16, 2014 at 1:32 PM, Stephan Ewen wrote: > Hi all! > > As per discu

Master bumped to 0.9

2014-12-16 Thread Stephan Ewen
Hi all! As per discussion in a separate thread, I forked the release branch (release-0.8 - for testing and release candidates) and bumped the master to "0.9-incubating-SNAPSHOT". Greetings, Stephan

Re: Forking off the 0.8 release branch.

2014-12-16 Thread Stephan Ewen
Hi! I forked a branch "release-0.8" and bumped the master version to "0.9-incubating-SNAPSHOT". The closure cleaner is merged. Robert has opened a PR for Kryo - I would suggest to merge that to "master", cherry-pick to "release-0.8" and then: Merry releasing! Greetings, Stephan On Tue, Dec 16

[jira] [Created] (FLINK-1333) Getter/Setter recognition for POJO fields with generics is not working

2014-12-16 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-1333: - Summary: Getter/Setter recognition for POJO fields with generics is not working Key: FLINK-1333 URL: https://issues.apache.org/jira/browse/FLINK-1333 Project: Flink

Re: Forking off the 0.8 release branch.

2014-12-16 Thread Márton Balassi
@Timo&Stephan: The streaming code uses this since last week [1], Timo starting working on the issue prior way before that - so it also has to do with the streaming team pushing last minute features to be honest... Then are we waiting for the Java Clousure cleaner and the Kyro support, I guess. Alj

Re: Forking off the 0.8 release branch.

2014-12-16 Thread Timo Walther
Yes, I also think so. Stability is more important. > Am 16.12.2014 um 17:46 schrieb Stephan Ewen : > > Yes, the fact that we both did not realize that until now sort of reveals > that there are more implications than anticipated. That is never good for > "last minute" patches. > > That's why I

Re: Forking off the 0.8 release branch.

2014-12-16 Thread Stephan Ewen
Yes, the fact that we both did not realize that until now sort of reveals that there are more implications than anticipated. That is never good for "last minute" patches. That's why I would like to merge it into 0.8.1 and 0.9, rather than 0.8.0 That gives the code a bit more testing/use and us a b

Re: Forking off the 0.8 release branch.

2014-12-16 Thread Timo Walther
Sorry I totally forgot to check the streaming API. Do they use the getXXXReturnTypes() methods of the TypeExtractor? > Am 16.12.2014 um 17:23 schrieb Robert Metzger : > > I've worked on adding Kryo support to Flink today. I would really like to > include this into the release. Many users were co

Re: Forking off the 0.8 release branch.

2014-12-16 Thread Robert Metzger
I've worked on adding Kryo support to Flink today. I would really like to include this into the release. Many users were complaining about this ( https://issues.apache.org/jira/browse/FLINK-610). I've based my work on top of Timo's pull request. If we're not going to merge it, I will have to rebas

Re: Forking off the 0.8 release branch.

2014-12-16 Thread Stephan Ewen
Yes, one thing I just realized is that the streaming API is not aware of MissingTypeInfos and it may cause weird behavior. On Tue, Dec 16, 2014 at 4:56 PM, Fabian Hueske wrote: > > +1 for stability first > > 2014-12-16 16:50 GMT+01:00 Stephan Ewen : > > > > I am currently working through the type

Re: Forking off the 0.8 release branch.

2014-12-16 Thread Fabian Hueske
+1 for stability first 2014-12-16 16:50 GMT+01:00 Stephan Ewen : > > I am currently working through the type hints pull request... > > Right now, I actually think that we should postpone this for the next > bugfix release. > > This pull request changes actually some tricky mechanisms in the Java A

Re: Forking off the 0.8 release branch.

2014-12-16 Thread Aljoscha Krettek
+1 on postponing if it introduces behaviour we don't fully understand On Tue, Dec 16, 2014 at 4:50 PM, Stephan Ewen wrote: > I am currently working through the type hints pull request... > > Right now, I actually think that we should postpone this for the next > bugfix release. > > This pull requ

Re: Forking off the 0.8 release branch.

2014-12-16 Thread Stephan Ewen
I am currently working through the type hints pull request... Right now, I actually think that we should postpone this for the next bugfix release. This pull request changes actually some tricky mechanisms in the Java API (it delays the type information), which may have implications that me and T

[jira] [Created] (FLINK-1332) Possible deadlock with two broadcast sets

2014-12-16 Thread Sebastian Schelter (JIRA)
Sebastian Schelter created FLINK-1332: - Summary: Possible deadlock with two broadcast sets Key: FLINK-1332 URL: https://issues.apache.org/jira/browse/FLINK-1332 Project: Flink Issue Type:

Re: Forking off the 0.8 release branch.

2014-12-16 Thread Stephan Ewen
+1 for adding the closure cleaner... Am 16.12.2014 15:23 schrieb "Aljoscha Krettek" : > Do we want the Java Closure Cleaner in 0.8 as well? > > On Tue, Dec 16, 2014 at 10:30 AM, Márton Balassi > wrote: > > Cool, thanks. > > > > On Tue, Dec 16, 2014 at 10:29 AM, Stephan Ewen wrote: > >> > >> Hi!

Re: Forking off the 0.8 release branch.

2014-12-16 Thread Aljoscha Krettek
Do we want the Java Closure Cleaner in 0.8 as well? On Tue, Dec 16, 2014 at 10:30 AM, Márton Balassi wrote: > Cool, thanks. > > On Tue, Dec 16, 2014 at 10:29 AM, Stephan Ewen wrote: >> >> Hi! >> >> The type hints PR is ready now, Timo made some final adjustments yesterday >> evening. I'll review

Re: Forking off the 0.8 release branch.

2014-12-16 Thread Márton Balassi
Cool, thanks. On Tue, Dec 16, 2014 at 10:29 AM, Stephan Ewen wrote: > > Hi! > > The type hints PR is ready now, Timo made some final adjustments yesterday > evening. I'll review and merge it now, then we can fork the branch. > > Cheers, > Stephan > > > On Mon, Dec 15, 2014 at 10:08 PM, Márton Bal

Re: Forking off the 0.8 release branch.

2014-12-16 Thread Stephan Ewen
Hi! The type hints PR is ready now, Timo made some final adjustments yesterday evening. I'll review and merge it now, then we can fork the branch. Cheers, Stephan On Mon, Dec 15, 2014 at 10:08 PM, Márton Balassi wrote: > > Hey, > > Checking the PRs listed by Stephan it seems every import one h