Re: [DISCUSS] @Public libraries

I think in general annotating library methods/classes is a good idea. The question is just which APIs are going to be marked stable. In the past we've seen that we might have marked some of Flink's APIs stable too early. As a consequence we have to carry them along for quite some time (at the

Re: [DISCUSS] Hold copies in HeapStateBackend

Yes, you're right. This is not a principled solution but rather a work-around for a specific use case. The ReduceFunction must be used in the right way and it is easy to get wrong. (OTOH, there is currently no way to get object reusage right. So think the change would not worsen the current

[jira] [Created] (FLINK-5147) StreamingOperatorsITCase.testGroupedFoldOperation failed on Travis

Chesnay Schepler created FLINK-5147: --- Summary: StreamingOperatorsITCase.testGroupedFoldOperation failed on Travis Key: FLINK-5147 URL: https://issues.apache.org/jira/browse/FLINK-5147 Project:

Re: [DISCUSS] @Public libraries

I would be for also annotating library methods/classes. Maybe Robert has a stronger opinion on this because he introduced these annotations. On Tue, 22 Nov 2016 at 18:56 Greg Hogan wrote: > Hi all, > > Should stable APIs in Flink's CEP, ML, and Gelly libraries be annotated >

Re: [DISCUSS] Hold copies in HeapStateBackend

You can go ahead and do the change. I just think that this is quite fragile. For example, this depends on the reduce function returning the right object for reuse. If we hand in the copied object as the first input and the ReduceFunction reuses the second input then we again have a reference to

[jira] [Created] (FLINK-5146) Improved resource cleanup in RocksDB keyed state backend

Stefan Richter created FLINK-5146: - Summary: Improved resource cleanup in RocksDB keyed state backend Key: FLINK-5146 URL: https://issues.apache.org/jira/browse/FLINK-5146 Project: Flink

[jira] [Created] (FLINK-5145) WebInterface to aggressive in pulling metrics

Chesnay Schepler created FLINK-5145: --- Summary: WebInterface to aggressive in pulling metrics Key: FLINK-5145 URL: https://issues.apache.org/jira/browse/FLINK-5145 Project: Flink Issue

Re: [DISCUSS] Hold copies in HeapStateBackend

Hi Aljoscha, sure, there many issues with holding the state as objects on the heap. However, I think we don't have to solve all problems related to that in order to add a small fix that solves one specific issue. I would not explicitly expose the fix to users but it would be nice if we could

[jira] [Created] (FLINK-5144) Error while applying rule AggregateJoinTransposeRule

Timo Walther created FLINK-5144: --- Summary: Error while applying rule AggregateJoinTransposeRule Key: FLINK-5144 URL: https://issues.apache.org/jira/browse/FLINK-5144 Project: Flink Issue Type:

Re: [DISCUSS] Hold copies in HeapStateBackend

hi，Fabian Hueske, Sorry for mistake for the whole PR #2792 > On Nov 23, 2016, at 17:10, Fabian Hueske wrote: > > Hi, > > Why do you think that this means "much code changes"? > I think it would actually be a pretty lightweight change in > HeapReducingState. > > The proposal

Re: [DISCUSS] deprecated function need more detail

+1 This should always be the norm, especially for user-facing code. While we are at it, perhaps when someone deprecates functionality the new alternative should also be replaced right away. E.g. Checkpointed is deprecated but all state management tests are actually using this alternative.

Re: [DISCUSS] deprecated function need more detail

+1 and we should apply the same to all deprecated interfaces/abstract classes. > On Nov 23, 2016, at 11:13 AM, Aljoscha Krettek wrote: > > +1 That sounds excellent. > > On Wed, 23 Nov 2016 at 11:04 Till Rohrmann wrote: > >> +1 for your proposal. >>

Re: [DISCUSS] deprecated function need more detail

+1 That sounds excellent. On Wed, 23 Nov 2016 at 11:04 Till Rohrmann wrote: > +1 for your proposal. > > Cheers, > Till > > On Wed, Nov 23, 2016 at 9:33 AM, Fabian Hueske wrote: > > > I agree on this one. > > Whenever we deprecate a method or a feature

Re: [DISCUSS] deprecated function need more detail

+1 for your proposal. Cheers, Till On Wed, Nov 23, 2016 at 9:33 AM, Fabian Hueske wrote: > I agree on this one. > Whenever we deprecate a method or a feature we should add a comment that > explains the new API or why the feature was removed without replacement. > > Enforcing

Re: [DISCUSS] Hold copies in HeapStateBackend

Hi, Why do you think that this means "much code changes"? I think it would actually be a pretty lightweight change in HeapReducingState. The proposal is to copy the *first* value that goes into a ReducingState. The copy would be done by a TypeSerializer and hence be a deep copy. This will allow

[jira] [Created] (FLINK-5141) Implement MiniClusterStreamEnvironment to run new mini cluster in flip-6 branch

Biao Liu created FLINK-5141: --- Summary: Implement MiniClusterStreamEnvironment to run new mini cluster in flip-6 branch Key: FLINK-5141 URL: https://issues.apache.org/jira/browse/FLINK-5141 Project: Flink