Re:refactor StreamConfig

Hi Aljoscha: I sum up my thoughts now. 1. rename StreamConfig to StreamTaskConfig. 2. OperatorConig can be changed to be serialisable. If StreamTaskConfig is also serialisable, it cannot be deserialized when it is passed to the jobManager, which do not depend on

Re: refactor StreamConfig

Hi Aljoscha: Ye, I agree with you that an operator should not see output edges and serialisers. The call getChainIndex() is used only in OperatorConfig.toString(), it can be removed. However, isChainStart() and isChainEnd() is used in AbstractStreamOperator.setup(...). But I think what

Re: [DISCUSS] FLIP-22: Eager State Declaration

Hi Tzu-Li, Thanks for the proposal. The changes are great. I have several questions about some details. First, do you have any plan to provide a method to remove states? Now states can only be created (either lazily or eagerly), but cannot be removed. We cannot remove those states not registered

Re: [DISCUSS] FLIP-22: Eager State Declaration

Hi, all, we have jobs which create state according to type of the key and a dynamic configuration: eg: key_type_1's aggregation function is average, while key_type_2's is sum we need to create state dynamically because the aggregation function may change in runtime and different aggregation

[jira] [Created] (FLINK-7104) AccessDeniedException get when FileUtilsTest#testDeleteDirectoryConcurrently call

mingleizhang created FLINK-7104: --- Summary: AccessDeniedException get when FileUtilsTest#testDeleteDirectoryConcurrently call Key: FLINK-7104 URL: https://issues.apache.org/jira/browse/FLINK-7104

Re: [DISCUSS] FLIP-22: Eager State Declaration

Could you add an example to the FLIP for how a user can register a state with the methods in the RichFunction interface? Currently it only contains an example for the annotation option. These methods look like they are called by the user, but that doesn't really make sense to me as after all

[DISCUSS] FLIP-22: Eager State Declaration

Hi Flink devs! I would like to propose the following FLIP - Eager State Declaration for Flink managed state:  https://cwiki.apache.org/confluence/display/FLINK/FLIP-22%3A+Eager+State+Declaration. The proposal is a result of some offline discussions with Aljoscha Krettek, Stephan Ewen, and

Re: [DISCUSS] FLIP proposal for Model Serving over Flink

Hi all, yes, we did too. We - from Radicalbit - have submitted a talk focused on the recently released flink-jpmml library about model serving. Lately, it became part of the FlinkML project. Cheers, Andrea 2017-07-04 16:14 GMT+02:00 Boris Lublinsky : > Yes, > I

Re: connect data stream with parameter stream

Hi Lei, I’m afraid there is currently no API for doing this in one operation. I see two options right now: 1. Built a custom operator that implements windowing and also has a second input for the parameter stream. This would be a subclass of TwoInputStreamOperator. As an example, you can look

Re: Streaming Graph processing

I cannot answer that for sure since graph streams are still a research topic. It depends on the demand and how fast graph stream representations and operations will become adopted. If there is high demand on Flink we can definitely start a FLIP at some point but for now it makes sense to see

[jira] [Created] (FLINK-7103) Implement dispatcher component

Till Rohrmann created FLINK-7103: Summary: Implement dispatcher component Key: FLINK-7103 URL: https://issues.apache.org/jira/browse/FLINK-7103 Project: Flink Issue Type: Sub-task

Re:Re: refactor StreamConfig

Hi Aljoscha: Thanks a lot for your advice. I think I have not need to separate steps, because what I do is only that introducing OperatorConfig and moving the fields. StreamConfig still relys on an underlying Configuration which flows from client to the jobmanager and then to the

[jira] [Created] (FLINK-7102) improve ClassLoaderITCase

Nico Kruber created FLINK-7102: -- Summary: improve ClassLoaderITCase Key: FLINK-7102 URL: https://issues.apache.org/jira/browse/FLINK-7102 Project: Flink Issue Type: Bug Components:

Re: refactor StreamConfig （Appending a picture）

The picture didn't go thru. Please use third party site. On Tue, Jul 4, 2017 at 7:09 AM, xu wrote: > I All: > I am sorry about working with StreamConfig(https://github. > com/apache/flink/pull/4241) which may conflicts with others' work before > discussing. > >

Re: refactor StreamConfig

I think the proposed changed are good, I just wanted to make sure that they don’t interfere with what other people are doing. I also proposed these steps on the Github PR: Also, for actually doing the changes I suggest separate steps, i.e. separate commits. With possibly separate PRs to make

Re:refactor StreamConfig （Appending a picture）

I All: I am sorry about working with StreamConfig(https://github.com/apache/flink/pull/4241) which may conflicts with others' work before discussing. Motivation: A Task contains one or more operators with chainning, however configs of operator and task are all put in

refactor StreamConfig

HI All: I am sorry about working with StreamConfig(https://github.com/apache/flink/pull/4241) which may conflicts with others' work before discussing. Motivation: A Task contains one or more operators with chainning, however configs of operator and task are all put in

[jira] [Created] (FLINK-7101) Fix Non-windowed group-aggregate error when using `minIdleStateRetentionTime` config and retract agg

sunjincheng created FLINK-7101: -- Summary: Fix Non-windowed group-aggregate error when using `minIdleStateRetentionTime` config and retract agg Key: FLINK-7101 URL: https://issues.apache.org/jira/browse/FLINK-7101

[jira] [Created] (FLINK-7100) TaskManager metrics are registered twice

Chesnay Schepler created FLINK-7100: --- Summary: TaskManager metrics are registered twice Key: FLINK-7100 URL: https://issues.apache.org/jira/browse/FLINK-7100 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-7099) Replace usages of deprecated JOB_MANAGER_IPC_PORT_KEY

Chesnay Schepler created FLINK-7099: --- Summary: Replace usages of deprecated JOB_MANAGER_IPC_PORT_KEY Key: FLINK-7099 URL: https://issues.apache.org/jira/browse/FLINK-7099 Project: Flink

[jira] [Created] (FLINK-7096) Adapt start-up scripts to start Flip-6 cluster

Till Rohrmann created FLINK-7096: Summary: Adapt start-up scripts to start Flip-6 cluster Key: FLINK-7096 URL: https://issues.apache.org/jira/browse/FLINK-7096 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-7095) Add proper command line parsing tool to TaskManagerRunner.main

Till Rohrmann created FLINK-7095: Summary: Add proper command line parsing tool to TaskManagerRunner.main Key: FLINK-7095 URL: https://issues.apache.org/jira/browse/FLINK-7095 Project: Flink

Re: Re: Tips to fix IDEA strange problem after updating master code

After deactivating scala-2.10 profile in IntelliJ, this issue is gone. Thank you all. On 7/4/17, 17:11, "Piotr Nowojski" wrote: Maybe try $ mvn clean Before reimporting and restarting/invalidating caches in IntelliJ? Did you deactivate

Re: Tips to fix IDEA strange problem after updating master code

You could also try the nuclear option of actually re-importing the project into intellij (as if you just downloaded it), that worked for me. On 04.07.2017 11:11, Piotr Nowojski wrote: Maybe try $ mvn clean Before reimporting and restarting/invalidating caches in IntelliJ? Did you deactivate

Re: Tips to fix IDEA strange problem after updating master code

Maybe try $ mvn clean Before reimporting and restarting/invalidating caches in IntelliJ? Did you deactivate scala-2.10 profile in the IntelliJ? Piotrek > On Jul 4, 2017, at 11:05 AM, 郭健 wrote: > > I have done all these but still got some issue in IDEA, especially in

Re: Re: Tips to fix IDEA strange problem after updating master code

I have done all these but still got some issue in IDEA, especially in the flink-connector project. On 7/4/17, 15:57, "Piotr Nowojski" wrote: Besides deactivating “scala-2.10” profile in the Intellij it might be necessary to: - reimport maven project:

Re: Problem to deploy of flink program in cluster

I only see pom.xml in the attachment.  Consider using pastebin for the other files.  Original message From: simone sguazza Date: 7/4/17 12:34 AM (GMT-08:00) To: dev@flink.apache.org Subject: Problem to deploy of flink program in cluster Hi,

Re: Tips to fix IDEA strange problem after updating master code

Besides deactivating “scala-2.10” profile in the Intellij it might be necessary to: - reimport maven project: 1. Right click on root module: “flink-parent” 2. Maven 3. reimport - invalidate caches and restart: File -> Invalidate caches and restart -> invalidate /restart -

Re: Problem to deploy of flink program in cluster

Hi Simone, Is the job actually running? Or did it fail with some exception? If it failed, there should be some error message in the JM logs. Could you take a look at that and see if it hints anything? Cheers, Gordon On 4 July 2017 at 3:35:07 PM, simone sguazza (simone-sgua...@hotmail.com)

[jira] [Created] (FLINK-7094) Update outdated doc release 1.2 to latest stable release 1.3

mingleizhang created FLINK-7094: --- Summary: Update outdated doc release 1.2 to latest stable release 1.3 Key: FLINK-7094 URL: https://issues.apache.org/jira/browse/FLINK-7094 Project: Flink

Re: [外部邮件] Tips to fix IDEA strange problem after updating master code

+1 Meet the same problem. I have tried reimporting the maven project but have no effect. From: Jark Wu Reply-To: "dev@flink.apache.org" Date: Tuesday, July 4, 2017 at 11:56 To: "dev@flink.apache.org" Subject: [外部邮件] Tips to fix

Re: Tips to fix IDEA strange problem after updating master code

Thanks for the hint! > On 4. Jul 2017, at 06:03, Ted Yu wrote: > > Looks like the picture didn't go thru. > > Mind using third party site ? > > Thanks > > On Mon, Jul 3, 2017 at 8:56 PM, Jark Wu wrote: > >> Hi devs, >> >> Yesterday, I updated the

Problem to deploy of flink program in cluster

Hi, below you can find my mail and my problem before the my subscribe to mailing list. simone Da: simone sguazza Inviato: lunedì 3 luglio 2017 11:57 A: dev@flink.apache.org Oggetto: Problem to deploy of flink program in cluster Hi,

[jira] [Created] (FLINK-7092) Shutdown ResourceManager components properly

Till Rohrmann created FLINK-7092: Summary: Shutdown ResourceManager components properly Key: FLINK-7092 URL: https://issues.apache.org/jira/browse/FLINK-7092 Project: Flink Issue Type:

[jira] [Created] (FLINK-7090) Rework ConnectionMonitor to not depend on Akka

Till Rohrmann created FLINK-7090: Summary: Rework ConnectionMonitor to not depend on Akka Key: FLINK-7090 URL: https://issues.apache.org/jira/browse/FLINK-7090 Project: Flink Issue Type:

[jira] [Created] (FLINK-7089) Rework LaunchCoordinator to not depend on Akka

Till Rohrmann created FLINK-7089: Summary: Rework LaunchCoordinator to not depend on Akka Key: FLINK-7089 URL: https://issues.apache.org/jira/browse/FLINK-7089 Project: Flink Issue Type:

[jira] [Created] (FLINK-7088) Rework TaskMonitor to not depend on Akka

Till Rohrmann created FLINK-7088: Summary: Rework TaskMonitor to not depend on Akka Key: FLINK-7088 URL: https://issues.apache.org/jira/browse/FLINK-7088 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-7087) Implement Flip-6 container entry point

Till Rohrmann created FLINK-7087: Summary: Implement Flip-6 container entry point Key: FLINK-7087 URL: https://issues.apache.org/jira/browse/FLINK-7087 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-7086) Create entry point for standalone session mode

Till Rohrmann created FLINK-7086: Summary: Create entry point for standalone session mode Key: FLINK-7086 URL: https://issues.apache.org/jira/browse/FLINK-7086 Project: Flink Issue Type:

[jira] [Created] (FLINK-7084) Flip-6 testing

Till Rohrmann created FLINK-7084: Summary: Flip-6 testing Key: FLINK-7084 URL: https://issues.apache.org/jira/browse/FLINK-7084 Project: Flink Issue Type: Test Components: Tests