[jira] [Commented] (FLINK-19128) Remove the runtime execution configuration in sql-client-defaults.yaml

2020-09-02 Thread Jark Wu (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-19128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189845#comment-17189845
 ] 

Jark Wu commented on FLINK-19128:
-

Not remove the totall file, but only the config entries, such as 
{{parallelism}}.

> Remove the runtime execution configuration in sql-client-defaults.yaml
> --
>
> Key: FLINK-19128
> URL: https://issues.apache.org/jira/browse/FLINK-19128
> Project: Flink
>  Issue Type: New Feature
>  Components: Table SQL / Client
>Reporter: Jark Wu
>Priority: Major
>
> Currently, {{sql-client-defaults.yaml}} sets some runtime execution 
> configuration by defualt, e.g. parallelism, restart strategy. This is very 
> confusing. Because users are confused why paralleslim is changed in 
> {{flink-conf.yaml}} but it doesn't work. 
> I think the community had an agreement that SQL CLI shouldn't provide a new 
> set of runtime configuration. As a first step, I suggest to remove these 
> configurations defined in {{sql-client-defaults.yaml}}, to make the 
> configurations in {{flink-conf.yaml}} work by default. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Issue Comment Deleted] (FLINK-19067) FileNotFoundException when run flink examples on standby JobManager

2020-09-02 Thread JieFang.He (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-19067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

JieFang.He updated FLINK-19067:
---
Comment: was deleted

(was: [~rmetzger]  This situation does not always happen. My cluster no longer 
appears after reinstallation. I am now trying to reproduce the problem.)

> FileNotFoundException when run flink examples on standby JobManager
> ---
>
> Key: FLINK-19067
> URL: https://issues.apache.org/jira/browse/FLINK-19067
> Project: Flink
>  Issue Type: Bug
>  Components: Runtime / Coordination
>Affects Versions: 1.11.1
>Reporter: JieFang.He
>Priority: Major
>
> 1、When run examples/batch/WordCount.jar on standby JobManager,it will fail 
> with the exception:
> Caused by: java.io.FileNotFoundException: 
> /data2/storageDir/default/blob/job_d29414828f614d5466e239be4d3889ac/blob_p-a2ebe1c5aa160595f214b4bd0f39d80e42ee2e93-f458f1c12dc023e78d25f191de1d7c4b
>  (No such file or directory)
>  at java.io.FileInputStream.open0(Native Method)
>  at java.io.FileInputStream.open(FileInputStream.java:195)
>  at java.io.FileInputStream.(FileInputStream.java:138)
>  at 
> org.apache.flink.core.fs.local.LocalDataInputStream.(LocalDataInputStream.java:50)
>  at 
> org.apache.flink.core.fs.local.LocalFileSystem.open(LocalFileSystem.java:143)
>  at 
> org.apache.flink.runtime.blob.FileSystemBlobStore.get(FileSystemBlobStore.java:105)
>  at 
> org.apache.flink.runtime.blob.FileSystemBlobStore.get(FileSystemBlobStore.java:87)
>  at 
> org.apache.flink.runtime.blob.BlobServer.getFileInternal(BlobServer.java:501)
>  at 
> org.apache.flink.runtime.blob.BlobServerConnection.get(BlobServerConnection.java:231)
>  at 
> org.apache.flink.runtime.blob.BlobServerConnection.run(BlobServerConnection.java:117)
>  
> 2、Run examples success on other nodes
> 3、After run success on the other node, it can run success on the Standby 
> JobManager. But run again will fail
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [flink] flinkbot edited a comment on pull request #13230: [FLINK-18950][python][docs] Add documentation for Operations in Python DataStream API.

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13230:
URL: https://github.com/apache/flink/pull/13230#issuecomment-679164841


   
   ## CI report:
   
   * 7a79a9dcb38787f5be0f7be9903fb81bb1408dce Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6128)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #13279: [FLINK-19090][docs-zh] Translate "Local Cluster" of "Clusters & Depolyment" page into Chinese

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13279:
URL: https://github.com/apache/flink/pull/13279#issuecomment-683391097


   
   ## CI report:
   
   * 55564089efdd5b17f61b92c51213a6b0d019bdda Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6129)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] klion26 commented on a change in pull request #13225: [FLINK-18974][docs-zh]Translate the 'User-Defined Functions' page of "Application Development's DataStream API" into Chinese

2020-09-02 Thread GitBox


klion26 commented on a change in pull request #13225:
URL: https://github.com/apache/flink/pull/13225#discussion_r482706564



##
File path: docs/dev/user_defined_functions.zh.md
##
@@ -147,95 +153,78 @@ data.map (new RichMapFunction[String, Int] {
 
 
 
-Rich functions provide, in addition to the user-defined function (map,
-reduce, etc), four methods: `open`, `close`, `getRuntimeContext`, and
-`setRuntimeContext`. These are useful for parameterizing the function
-(see [Passing Parameters to Functions]({{ site.baseurl 
}}/dev/batch/index.html#passing-parameters-to-functions)),
-creating and finalizing local state, accessing broadcast variables (see
-[Broadcast Variables]({{ site.baseurl 
}}/dev/batch/index.html#broadcast-variables)), and for accessing runtime
-information such as accumulators and counters (see
-[Accumulators and Counters](#accumulators--counters)), and information
-on iterations (see [Iterations]({{ site.baseurl }}/dev/batch/iterations.html)).
+除了用户自定义的功能(map,reduce 等),Rich functions 
还提供了四个方法:`open`、`close`、`getRuntimeContext` 和
+`setRuntimeContext`。这些方法对于参数化函数
+(参阅 [给函数传递参数]({{ site.baseurl }}{% link 
zh/dev/batch/index.html#passing-parameters-to-functions %})),

Review comment:
   现在的链接还是有问题的,如果在本地执行 `./docs/docker/run.sh` 然后执行 `./build_docs.sh -p` 
是可以看到报错的
   正确的应该是 `{%link dev/batch/index.zh.md %}#passing-parameters-to-functions`





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Comment Edited] (FLINK-19128) Remove the runtime execution configuration in sql-client-defaults.yaml

2020-09-02 Thread Nicholas Jiang (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-19128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189839#comment-17189839
 ] 

Nicholas Jiang edited comment on FLINK-19128 at 9/3/20, 5:04 AM:
-

Hi, [~jark], do you mean that the first step you mentioned should remove this 
sql-client-defaults.yaml configuration file, and use flink-conf.yaml as the 
default configuration file (DEFAULT_ENV_FILE) for LocalExecutor?


was (Author: nicholasjiang):
Hi, [~jark], do you mean that the first step you mentioned should remove this 
sql-client-defaults.yaml configuration file, and use flink-conf.yaml as the 
default configuration file? 
And the DEFAULT_ENV_FILE of LocalExecutor could be flink-conf.yaml.

> Remove the runtime execution configuration in sql-client-defaults.yaml
> --
>
> Key: FLINK-19128
> URL: https://issues.apache.org/jira/browse/FLINK-19128
> Project: Flink
>  Issue Type: New Feature
>  Components: Table SQL / Client
>Reporter: Jark Wu
>Priority: Major
>
> Currently, {{sql-client-defaults.yaml}} sets some runtime execution 
> configuration by defualt, e.g. parallelism, restart strategy. This is very 
> confusing. Because users are confused why paralleslim is changed in 
> {{flink-conf.yaml}} but it doesn't work. 
> I think the community had an agreement that SQL CLI shouldn't provide a new 
> set of runtime configuration. As a first step, I suggest to remove these 
> configurations defined in {{sql-client-defaults.yaml}}, to make the 
> configurations in {{flink-conf.yaml}} work by default. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (FLINK-19128) Remove the runtime execution configuration in sql-client-defaults.yaml

2020-09-02 Thread Nicholas Jiang (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-19128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189839#comment-17189839
 ] 

Nicholas Jiang commented on FLINK-19128:


Hi, [~jark], do you mean that the first step you mentioned should remove this 
sql-client-defaults.yaml configuration file, and use flink-conf.yaml as the 
default configuration file? 
And the DEFAULT_ENV_FILE of LocalExecutor could be flink-conf.yaml.

> Remove the runtime execution configuration in sql-client-defaults.yaml
> --
>
> Key: FLINK-19128
> URL: https://issues.apache.org/jira/browse/FLINK-19128
> Project: Flink
>  Issue Type: New Feature
>  Components: Table SQL / Client
>Reporter: Jark Wu
>Priority: Major
>
> Currently, {{sql-client-defaults.yaml}} sets some runtime execution 
> configuration by defualt, e.g. parallelism, restart strategy. This is very 
> confusing. Because users are confused why paralleslim is changed in 
> {{flink-conf.yaml}} but it doesn't work. 
> I think the community had an agreement that SQL CLI shouldn't provide a new 
> set of runtime configuration. As a first step, I suggest to remove these 
> configurations defined in {{sql-client-defaults.yaml}}, to make the 
> configurations in {{flink-conf.yaml}} work by default. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (FLINK-19130) Expose backpressure metrics / logs for function dispatcher operator

2020-09-02 Thread Tzu-Li (Gordon) Tai (Jira)
Tzu-Li (Gordon) Tai created FLINK-19130:
---

 Summary: Expose backpressure metrics / logs for function 
dispatcher operator
 Key: FLINK-19130
 URL: https://issues.apache.org/jira/browse/FLINK-19130
 Project: Flink
  Issue Type: Task
  Components: Stateful Functions
Reporter: Tzu-Li (Gordon) Tai
Assignee: Tzu-Li (Gordon) Tai


As of now, there is no visibility on why or how backpressure is applied in 
Stateful Functions.
This JIRA attemps to add two metrics as an initial effort of providing more 
visibility:
- Total number of addresses that have asked to be blocked
- Total number of inflight pending async operations



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [flink] flinkbot edited a comment on pull request #13284: [FLINK-17016][runtime] Change blink planner batch jobs to run with pipelined region scheduling

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13284:
URL: https://github.com/apache/flink/pull/13284#issuecomment-683683000


   
   ## CI report:
   
   * e5a5f7d8e5a8cb5ce7e18bbf54ed2671f31e75a4 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6098)
 
   * a541f5410ff96d39708288ae7aa672ed2ebb05a8 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6135)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #13232: [FLINK-18951][python][docs] Add documentation for Configurations in Python DataStream API.

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13232:
URL: https://github.com/apache/flink/pull/13232#issuecomment-679225070


   
   ## CI report:
   
   * 492cf7e87bfb8ac6f3fafc73617345b1de1eefe5 Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6127)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #13284: [FLINK-17016][runtime] Change blink planner batch jobs to run with pipelined region scheduling

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13284:
URL: https://github.com/apache/flink/pull/13284#issuecomment-683683000


   
   ## CI report:
   
   * e5a5f7d8e5a8cb5ce7e18bbf54ed2671f31e75a4 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6098)
 
   * a541f5410ff96d39708288ae7aa672ed2ebb05a8 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #13295: [FLINK-18598][python][docs] Add instructions for asynchronous execute in PyFlink doc.

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13295:
URL: https://github.com/apache/flink/pull/13295#issuecomment-684586420


   
   ## CI report:
   
   * f67e466f192eaecfc167467334d844a571123afe Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6125)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #13306: [FLINK-17779][Connectors/ORC]Orc file format support filter push down

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13306:
URL: https://github.com/apache/flink/pull/13306#issuecomment-685717187


   
   ## CI report:
   
   * 46f53e213dcc4c40866e9001da2f785c31f2c9ad Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6101)
 
   * a7f4abab05ad875bf6acef6510fcec6df7d2dc6c UNKNOWN
   * 8048b8f9ded32203cb0ca0761414ef7186a68e5f Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6132)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #13297: [FLINK-19105][docs]update demo error

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13297:
URL: https://github.com/apache/flink/pull/13297#issuecomment-684720563


   
   ## CI report:
   
   * 859933a6d90db9df03ca1d01841bef274ec20f7f Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6054)
 
   * b82b6f91fa2258a040e3e208f551c5f1c22ef1c5 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6134)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #13311: [FLINK-18721] Migrate YarnResourceManager to the new YarnResourceManagerDriver

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13311:
URL: https://github.com/apache/flink/pull/13311#issuecomment-686196434


   
   ## CI report:
   
   * 6d73ba60af6a8aa17399a144e2414f197f40ed21 Azure: 
[CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6126)
 
   * b7660009b48d48261b767855d8973c63741ab493 UNKNOWN
   * a994636565707811a1f905e14b2b839ad2a0e408 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6133)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #13306: [FLINK-17779][Connectors/ORC]Orc file format support filter push down

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13306:
URL: https://github.com/apache/flink/pull/13306#issuecomment-685717187


   
   ## CI report:
   
   * 46f53e213dcc4c40866e9001da2f785c31f2c9ad Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6101)
 
   * a7f4abab05ad875bf6acef6510fcec6df7d2dc6c UNKNOWN
   * 8048b8f9ded32203cb0ca0761414ef7186a68e5f UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #13297: [FLINK-19105][docs]update demo error

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13297:
URL: https://github.com/apache/flink/pull/13297#issuecomment-684720563


   
   ## CI report:
   
   * 859933a6d90db9df03ca1d01841bef274ec20f7f Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6054)
 
   * b82b6f91fa2258a040e3e208f551c5f1c22ef1c5 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] YongjinZhou commented on a change in pull request #13297: [FLINK-19105][docs]update demo error

2020-09-02 Thread GitBox


YongjinZhou commented on a change in pull request #13297:
URL: https://github.com/apache/flink/pull/13297#discussion_r482687021



##
File path: docs/dev/table/common.md
##
@@ -743,9 +743,9 @@ TableEnvironment tableEnv = ...; // see "Create a 
TableEnvironment" section
 final Schema schema = new Schema()
 .field("a", DataTypes.INT())
 .field("b", DataTypes.STRING())
-.field("c", DataTypes.LONG());
+.field("c", DataTypes.BIGINT());
 
-tableEnv.connect(new FileSystem("/path/to/file"))
+tableEnv.connect(new FileSystem().path("/path/to/file"))

Review comment:
   ok, It has been updated.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #13306: [FLINK-17779][Connectors/ORC]Orc file format support filter push down

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13306:
URL: https://github.com/apache/flink/pull/13306#issuecomment-685717187


   
   ## CI report:
   
   * 46f53e213dcc4c40866e9001da2f785c31f2c9ad Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6101)
 
   * a7f4abab05ad875bf6acef6510fcec6df7d2dc6c UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Commented] (FLINK-18934) Idle stream does not advance watermark in connected stream

2020-09-02 Thread Zhu Zhu (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-18934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189816#comment-17189816
 ] 

Zhu Zhu commented on FLINK-18934:
-

[~knaufk] [~aljoscha] Given that this issue was proposed as a blocker for 
releasing 1.11.2, I have a couple of questions to understand the state of it. 
Appreciate if you can help with them.
 - Should it be a strict blocker of releasing 1.11.2?
 - Is there a concise on how we can fix it? 
 - Does next Monday as the deadline to cut RC1 work for the fix?

> Idle stream does not advance watermark in connected stream
> --
>
> Key: FLINK-18934
> URL: https://issues.apache.org/jira/browse/FLINK-18934
> Project: Flink
>  Issue Type: Bug
>  Components: API / DataStream
>Affects Versions: 1.11.1
>Reporter: Truong Duc Kien
>Priority: Critical
> Fix For: 1.12.0, 1.11.2
>
>
> Per Flink documents, when a stream is idle, it will allow watermarks of 
> downstream operator to advance. However, when I connect an active data stream 
> with an idle data stream, the output watermark of the CoProcessOperator does 
> not increase.
> Here's a small test that reproduces the problem.
> https://github.com/kien-truong/flink-idleness-testing



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [flink] flinkbot edited a comment on pull request #13304: [FLINK-19118][python] Support Expression in the operations of Python Table API

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13304:
URL: https://github.com/apache/flink/pull/13304#issuecomment-685633990


   
   ## CI report:
   
   * d308fe842311e794be70b21c1f9335c93fb29b5b Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6096)
 
   * 8ad8edefaaa28a0459b40b3587013d9ca7c14933 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6131)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #13292: [FLINK-19097][python] Support add_jars() for Python StreamExecutionEnvironment.

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13292:
URL: https://github.com/apache/flink/pull/13292#issuecomment-684516974


   
   ## CI report:
   
   * 44b87544d58616f56eeb8105764a37011c07ea9d Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6075)
 
   * a7b36aa890a9edc52d4bc6d173be798f94f23917 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6130)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #13230: [FLINK-18950][python][docs] Add documentation for Operations in Python DataStream API.

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13230:
URL: https://github.com/apache/flink/pull/13230#issuecomment-679164841


   
   ## CI report:
   
   * 86c48415d9aace2a986d704154f1f26af3fe2cc1 Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6079)
 
   * 7a79a9dcb38787f5be0f7be9903fb81bb1408dce Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6128)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Commented] (FLINK-19126) Failed to run job in yarn-cluster mode due to No Executor found.

2020-09-02 Thread Yang Wang (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-19126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189815#comment-17189815
 ] 

Yang Wang commented on FLINK-19126:
---

Yes, {{flink-shaded-hadoop}} has been deprecated from flink-shaded-11 and the 
Flink community suggests to use {{HADOOP_CLASSPATH}} instead. However, you 
could still download it in the maven repository[1].

 

[1]. 
[https://mvnrepository.com/artifact/org.apache.flink/flink-shaded-hadoop-2-uber]

> Failed to run job in yarn-cluster mode due to No Executor found.
> 
>
> Key: FLINK-19126
> URL: https://issues.apache.org/jira/browse/FLINK-19126
> Project: Flink
>  Issue Type: Bug
>  Components: Deployment / YARN
>Affects Versions: 1.11.1
>Reporter: Tang Yan
>Priority: Major
>
> I've build the flink package successfully, but when I run the below command, 
> it failed to submit the jobs.
> [yanta@flink-1.11]$ bin/flink run -m yarn-cluster -p 2 -c 
> org.apache.flink.examples.java.wordcount.WordCount 
> examples/batch/WordCount.jar  --input hdfs:///user/yanta/aa.txt --output 
> hdfs:///user/yanta/result.txt
> Setting HADOOP_CONF_DIR=/etc/hadoop/conf because no HADOOP_CONF_DIR or 
> HADOOP_CLASSPATH was set.
>  The program 
> finished with the following exception:
> java.lang.IllegalStateException: No Executor found. Please make sure to 
> export the HADOOP_CLASSPATH environment variable or have hadoop in your 
> classpath. For more information refer to the "Deployment & Operations" 
> section of the official Apache Flink documentation. at 
> org.apache.flink.yarn.cli.FallbackYarnSessionCli.isActive(FallbackYarnSessionCli.java:59)
>  at 
> org.apache.flink.client.cli.CliFrontend.validateAndGetActiveCommandLine(CliFrontend.java:1090)
>  at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:218) at 
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:916) 
> at 
> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:992) 
> at 
> org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
>  at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:992)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [flink] flinkbot edited a comment on pull request #13279: [FLINK-19090][docs-zh] Translate "Local Cluster" of "Clusters & Depolyment" page into Chinese

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13279:
URL: https://github.com/apache/flink/pull/13279#issuecomment-683391097


   
   ## CI report:
   
   * aa1a04a8c48aba5f3a8a4a2c222d589f045ec81f Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5992)
 
   * 55564089efdd5b17f61b92c51213a6b0d019bdda Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6129)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Updated] (FLINK-19109) Split Reader eats chained periodic watermarks

2020-09-02 Thread Zhu Zhu (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-19109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhu Zhu updated FLINK-19109:

Priority: Blocker  (was: Critical)

> Split Reader eats chained periodic watermarks
> -
>
> Key: FLINK-19109
> URL: https://issues.apache.org/jira/browse/FLINK-19109
> Project: Flink
>  Issue Type: Bug
>  Components: Runtime / Task
>Affects Versions: 1.10.0, 1.10.1, 1.11.0, 1.10.2, 1.11.1
>Reporter: David Anderson
>Assignee: Roman Khachatryan
>Priority: Blocker
>  Labels: pull-request-available
> Fix For: 1.12.0, 1.11.2, 1.10.3
>
>
> Attempting to generate watermarks chained to the Split Reader / 
> ContinuousFileReaderOperator, as in
> {code:java}
> SingleOutputStreamOperator results = env
>   .readTextFile(...)
>   .map(...)
>   .assignTimestampsAndWatermarks(bounded)
>   .keyBy(...)
>   .process(...);{code}
> leads to the Watermarks failing to be produced. Breaking the chain, via 
> {{disableOperatorChaining()}} or a {{rebalance}}, works around the bug. Using 
> punctuated watermarks also avoids the issue.
> Looking at this in the debugger reveals that timer service is being 
> prematurely quiesced.
> In many respects this is FLINK-7666 brought back to life.
> The problem is not present in 1.9.3.
> There's a minimal reproducible example in 
> [https://github.com/alpinegizmo/flink-question-001/tree/bug].



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (FLINK-18641) "Failure to finalize checkpoint" error in MasterTriggerRestoreHook

2020-09-02 Thread Zhu Zhu (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-18641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhu Zhu updated FLINK-18641:

Priority: Blocker  (was: Critical)

> "Failure to finalize checkpoint" error in MasterTriggerRestoreHook
> --
>
> Key: FLINK-18641
> URL: https://issues.apache.org/jira/browse/FLINK-18641
> Project: Flink
>  Issue Type: Bug
>  Components: Runtime / Checkpointing
>Affects Versions: 1.11.0
>Reporter: Brian Zhou
>Assignee: Jiangjie Qin
>Priority: Blocker
>  Labels: pull-request-available
> Fix For: 1.12.0, 1.11.2
>
>
> https://github.com/pravega/flink-connectors is a Pravega connector for Flink. 
> The ReaderCheckpointHook[1] class uses the Flink `MasterTriggerRestoreHook` 
> interface to trigger the Pravega checkpoint during Flink checkpoints to make 
> sure the data recovery. The checkpoint recovery tests are running fine in 
> Flink 1.10, but it has below issues in Flink 1.11 causing the tests time out. 
> Suspect it is related to the checkpoint coordinator thread model changes in 
> Flink 1.11
> Error stacktrace:
> {code}
> 2020-07-09 15:39:39,999 30945 [jobmanager-future-thread-5] WARN  
> o.a.f.runtime.jobmaster.JobMaster - Error while processing checkpoint 
> acknowledgement message
> org.apache.flink.runtime.checkpoint.CheckpointException: Could not finalize 
> the pending checkpoint 3. Failure reason: Failure to finalize checkpoint.
>  at 
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator.completePendingCheckpoint(CheckpointCoordinator.java:1033)
>  at 
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator.receiveAcknowledgeMessage(CheckpointCoordinator.java:948)
>  at 
> org.apache.flink.runtime.scheduler.SchedulerBase.lambda$acknowledgeCheckpoint$4(SchedulerBase.java:802)
>  at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>  at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>  at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>  at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>  at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>  at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.flink.util.SerializedThrowable: Pending checkpoint has 
> not been fully acknowledged yet
>  at 
> org.apache.flink.util.Preconditions.checkState(Preconditions.java:195)
>  at 
> org.apache.flink.runtime.checkpoint.PendingCheckpoint.finalizeCheckpoint(PendingCheckpoint.java:298)
>  at 
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator.completePendingCheckpoint(CheckpointCoordinator.java:1021)
>  ... 9 common frames omitted
> {code}
> More detail in this mailing thread: 
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Pravega-connector-cannot-recover-from-the-checkpoint-due-to-quot-Failure-to-finalize-checkpoint-quot-td36652.html
> Also in https://github.com/pravega/flink-connectors/issues/387



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (FLINK-18641) "Failure to finalize checkpoint" error in MasterTriggerRestoreHook

2020-09-02 Thread Zhu Zhu (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-18641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhu Zhu updated FLINK-18641:

Priority: Critical  (was: Major)

> "Failure to finalize checkpoint" error in MasterTriggerRestoreHook
> --
>
> Key: FLINK-18641
> URL: https://issues.apache.org/jira/browse/FLINK-18641
> Project: Flink
>  Issue Type: Bug
>  Components: Runtime / Checkpointing
>Affects Versions: 1.11.0
>Reporter: Brian Zhou
>Assignee: Jiangjie Qin
>Priority: Critical
>  Labels: pull-request-available
> Fix For: 1.12.0, 1.11.2
>
>
> https://github.com/pravega/flink-connectors is a Pravega connector for Flink. 
> The ReaderCheckpointHook[1] class uses the Flink `MasterTriggerRestoreHook` 
> interface to trigger the Pravega checkpoint during Flink checkpoints to make 
> sure the data recovery. The checkpoint recovery tests are running fine in 
> Flink 1.10, but it has below issues in Flink 1.11 causing the tests time out. 
> Suspect it is related to the checkpoint coordinator thread model changes in 
> Flink 1.11
> Error stacktrace:
> {code}
> 2020-07-09 15:39:39,999 30945 [jobmanager-future-thread-5] WARN  
> o.a.f.runtime.jobmaster.JobMaster - Error while processing checkpoint 
> acknowledgement message
> org.apache.flink.runtime.checkpoint.CheckpointException: Could not finalize 
> the pending checkpoint 3. Failure reason: Failure to finalize checkpoint.
>  at 
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator.completePendingCheckpoint(CheckpointCoordinator.java:1033)
>  at 
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator.receiveAcknowledgeMessage(CheckpointCoordinator.java:948)
>  at 
> org.apache.flink.runtime.scheduler.SchedulerBase.lambda$acknowledgeCheckpoint$4(SchedulerBase.java:802)
>  at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>  at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>  at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>  at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>  at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>  at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.flink.util.SerializedThrowable: Pending checkpoint has 
> not been fully acknowledged yet
>  at 
> org.apache.flink.util.Preconditions.checkState(Preconditions.java:195)
>  at 
> org.apache.flink.runtime.checkpoint.PendingCheckpoint.finalizeCheckpoint(PendingCheckpoint.java:298)
>  at 
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator.completePendingCheckpoint(CheckpointCoordinator.java:1021)
>  ... 9 common frames omitted
> {code}
> More detail in this mailing thread: 
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Pravega-connector-cannot-recover-from-the-checkpoint-due-to-quot-Failure-to-finalize-checkpoint-quot-td36652.html
> Also in https://github.com/pravega/flink-connectors/issues/387



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (FLINK-18641) "Failure to finalize checkpoint" error in MasterTriggerRestoreHook

2020-09-02 Thread Zhu Zhu (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-18641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhu Zhu updated FLINK-18641:

Fix Version/s: 1.11.2
   1.12.0

> "Failure to finalize checkpoint" error in MasterTriggerRestoreHook
> --
>
> Key: FLINK-18641
> URL: https://issues.apache.org/jira/browse/FLINK-18641
> Project: Flink
>  Issue Type: Bug
>  Components: Runtime / Checkpointing
>Affects Versions: 1.11.0
>Reporter: Brian Zhou
>Assignee: Jiangjie Qin
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.12.0, 1.11.2
>
>
> https://github.com/pravega/flink-connectors is a Pravega connector for Flink. 
> The ReaderCheckpointHook[1] class uses the Flink `MasterTriggerRestoreHook` 
> interface to trigger the Pravega checkpoint during Flink checkpoints to make 
> sure the data recovery. The checkpoint recovery tests are running fine in 
> Flink 1.10, but it has below issues in Flink 1.11 causing the tests time out. 
> Suspect it is related to the checkpoint coordinator thread model changes in 
> Flink 1.11
> Error stacktrace:
> {code}
> 2020-07-09 15:39:39,999 30945 [jobmanager-future-thread-5] WARN  
> o.a.f.runtime.jobmaster.JobMaster - Error while processing checkpoint 
> acknowledgement message
> org.apache.flink.runtime.checkpoint.CheckpointException: Could not finalize 
> the pending checkpoint 3. Failure reason: Failure to finalize checkpoint.
>  at 
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator.completePendingCheckpoint(CheckpointCoordinator.java:1033)
>  at 
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator.receiveAcknowledgeMessage(CheckpointCoordinator.java:948)
>  at 
> org.apache.flink.runtime.scheduler.SchedulerBase.lambda$acknowledgeCheckpoint$4(SchedulerBase.java:802)
>  at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>  at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>  at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>  at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>  at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>  at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.flink.util.SerializedThrowable: Pending checkpoint has 
> not been fully acknowledged yet
>  at 
> org.apache.flink.util.Preconditions.checkState(Preconditions.java:195)
>  at 
> org.apache.flink.runtime.checkpoint.PendingCheckpoint.finalizeCheckpoint(PendingCheckpoint.java:298)
>  at 
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator.completePendingCheckpoint(CheckpointCoordinator.java:1021)
>  ... 9 common frames omitted
> {code}
> More detail in this mailing thread: 
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Pravega-connector-cannot-recover-from-the-checkpoint-due-to-quot-Failure-to-finalize-checkpoint-quot-td36652.html
> Also in https://github.com/pravega/flink-connectors/issues/387



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (FLINK-19126) Failed to run job in yarn-cluster mode due to No Executor found.

2020-09-02 Thread Tang Yan (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-19126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189812#comment-17189812
 ] 

Tang Yan commented on FLINK-19126:
--

[~fly_in_gis] I used Flink 1.11.1, and I see it's using shade-11 in pom.xml, 
and when I build flink-shaded-11, I did not find flink-shaded-hadoop-xxx.jar in 
the build target. I remember it had flink-shaded-hadoop-xxx.jar when I built 
flink-shaded-9 last time. Any change in the new release?

 

> Failed to run job in yarn-cluster mode due to No Executor found.
> 
>
> Key: FLINK-19126
> URL: https://issues.apache.org/jira/browse/FLINK-19126
> Project: Flink
>  Issue Type: Bug
>  Components: Deployment / YARN
>Affects Versions: 1.11.1
>Reporter: Tang Yan
>Priority: Major
>
> I've build the flink package successfully, but when I run the below command, 
> it failed to submit the jobs.
> [yanta@flink-1.11]$ bin/flink run -m yarn-cluster -p 2 -c 
> org.apache.flink.examples.java.wordcount.WordCount 
> examples/batch/WordCount.jar  --input hdfs:///user/yanta/aa.txt --output 
> hdfs:///user/yanta/result.txt
> Setting HADOOP_CONF_DIR=/etc/hadoop/conf because no HADOOP_CONF_DIR or 
> HADOOP_CLASSPATH was set.
>  The program 
> finished with the following exception:
> java.lang.IllegalStateException: No Executor found. Please make sure to 
> export the HADOOP_CLASSPATH environment variable or have hadoop in your 
> classpath. For more information refer to the "Deployment & Operations" 
> section of the official Apache Flink documentation. at 
> org.apache.flink.yarn.cli.FallbackYarnSessionCli.isActive(FallbackYarnSessionCli.java:59)
>  at 
> org.apache.flink.client.cli.CliFrontend.validateAndGetActiveCommandLine(CliFrontend.java:1090)
>  at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:218) at 
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:916) 
> at 
> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:992) 
> at 
> org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
>  at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:992)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [flink] flinkbot edited a comment on pull request #13304: [FLINK-19118][python] Support Expression in the operations of Python Table API

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13304:
URL: https://github.com/apache/flink/pull/13304#issuecomment-685633990


   
   ## CI report:
   
   * d308fe842311e794be70b21c1f9335c93fb29b5b Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6096)
 
   * 8ad8edefaaa28a0459b40b3587013d9ca7c14933 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #13311: [FLINK-18721] Migrate YarnResourceManager to the new YarnResourceManagerDriver

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13311:
URL: https://github.com/apache/flink/pull/13311#issuecomment-686196434


   
   ## CI report:
   
   * 6d73ba60af6a8aa17399a144e2414f197f40ed21 Azure: 
[CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6126)
 
   * b7660009b48d48261b767855d8973c63741ab493 UNKNOWN
   * a994636565707811a1f905e14b2b839ad2a0e408 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #13292: [FLINK-19097][python] Support add_jars() for Python StreamExecutionEnvironment.

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13292:
URL: https://github.com/apache/flink/pull/13292#issuecomment-684516974


   
   ## CI report:
   
   * 44b87544d58616f56eeb8105764a37011c07ea9d Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6075)
 
   * a7b36aa890a9edc52d4bc6d173be798f94f23917 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #13279: [FLINK-19090][docs-zh] Translate "Local Cluster" of "Clusters & Depolyment" page into Chinese

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13279:
URL: https://github.com/apache/flink/pull/13279#issuecomment-683391097


   
   ## CI report:
   
   * aa1a04a8c48aba5f3a8a4a2c222d589f045ec81f Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5992)
 
   * 55564089efdd5b17f61b92c51213a6b0d019bdda UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #13232: [FLINK-18951][python][docs] Add documentation for Configurations in Python DataStream API.

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13232:
URL: https://github.com/apache/flink/pull/13232#issuecomment-679225070


   
   ## CI report:
   
   * 99c5a7392b35bef45802cee74c59667e64e8f239 Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6078)
 
   * 492cf7e87bfb8ac6f3fafc73617345b1de1eefe5 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6127)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #13230: [FLINK-18950][python][docs] Add documentation for Operations in Python DataStream API.

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13230:
URL: https://github.com/apache/flink/pull/13230#issuecomment-679164841


   
   ## CI report:
   
   * 86c48415d9aace2a986d704154f1f26af3fe2cc1 Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6079)
 
   * 7a79a9dcb38787f5be0f7be9903fb81bb1408dce UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Commented] (FLINK-18858) Kinesis Flink SQL Connector

2020-09-02 Thread Tzu-Li (Gordon) Tai (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-18858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189810#comment-17189810
 ] 

Tzu-Li (Gordon) Tai commented on FLINK-18858:
-

cc [~danny.cranmer] reckoned you might find this interesting, since we briefly 
chatted about a Kinesis SQL connector before.
Perhaps you can provide some help together with [~whummer] on a first 
implementation?

I might not have the capacity to actively push for the initial implementation 
and reviews, but a big +1 to the feature as well.
I'll be happy to take a look near the end if there is no other committer 
already looking at the contribution.

> Kinesis Flink SQL Connector
> ---
>
> Key: FLINK-18858
> URL: https://issues.apache.org/jira/browse/FLINK-18858
> Project: Flink
>  Issue Type: Improvement
>  Components: Connectors / Kinesis, Table SQL / Ecosystem
>Reporter: Waldemar Hummer
>Priority: Major
>
> Hi all,
> as far as I can see in the [list of 
> connectors|https://github.com/apache/flink/tree/master/flink-connectors], we 
> have a 
> {{[flink-connector-kinesis|https://github.com/apache/flink/tree/master/flink-connectors/flink-connector-kinesis]}}
>  for *programmatic access* to Kinesis streams, but there does not yet seem to 
> exist a *Kinesis SQL connector* (something like 
> {{flink-sql-connector-kinesis}}, analogous to {{flink-sql-connector-kafka}}).
> Our use case would be to enable SQL queries with direct access to Kinesis 
> sources (and potentially sinks), to enable something like the following Flink 
> SQL queries:
> {code:java}
>  $ bin/sql-client.sh embedded
> ...
> Flink SQL> CREATE TABLE Orders(`user` string, amount int, rowtime TIME) WITH 
> ('connector' = 'kinesis', ...);
> ...
> Flink SQL> SELECT * FROM Orders ...;
> ...{code}
>  
> I was wondering if this is something that has been considered, or is already 
> actively being worked on? If one of you can provide some guidance, we may be 
> able to work on a PoC implementation to add this functionality.
>  
> (Wasn't able to find an existing issue in the backlog - if this is a 
> duplicate, then please let me know as well.)
> Thanks!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (FLINK-19126) Failed to run job in yarn-cluster mode due to No Executor found.

2020-09-02 Thread Yang Wang (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-19126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189809#comment-17189809
 ] 

Yang Wang commented on FLINK-19126:
---

[~klion26] is right. You need to either set the {{HADOOP_CLASSPATH}} 
environment or put the {{flink-shaded-hadoop-xxx.jar}} in the 
{{FLINK_HOME/lib}} directory.

> Failed to run job in yarn-cluster mode due to No Executor found.
> 
>
> Key: FLINK-19126
> URL: https://issues.apache.org/jira/browse/FLINK-19126
> Project: Flink
>  Issue Type: Bug
>  Components: Deployment / YARN
>Affects Versions: 1.11.1
>Reporter: Tang Yan
>Priority: Major
>
> I've build the flink package successfully, but when I run the below command, 
> it failed to submit the jobs.
> [yanta@flink-1.11]$ bin/flink run -m yarn-cluster -p 2 -c 
> org.apache.flink.examples.java.wordcount.WordCount 
> examples/batch/WordCount.jar  --input hdfs:///user/yanta/aa.txt --output 
> hdfs:///user/yanta/result.txt
> Setting HADOOP_CONF_DIR=/etc/hadoop/conf because no HADOOP_CONF_DIR or 
> HADOOP_CLASSPATH was set.
>  The program 
> finished with the following exception:
> java.lang.IllegalStateException: No Executor found. Please make sure to 
> export the HADOOP_CLASSPATH environment variable or have hadoop in your 
> classpath. For more information refer to the "Deployment & Operations" 
> section of the official Apache Flink documentation. at 
> org.apache.flink.yarn.cli.FallbackYarnSessionCli.isActive(FallbackYarnSessionCli.java:59)
>  at 
> org.apache.flink.client.cli.CliFrontend.validateAndGetActiveCommandLine(CliFrontend.java:1090)
>  at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:218) at 
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:916) 
> at 
> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:992) 
> at 
> org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
>  at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:992)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [flink] flinkbot edited a comment on pull request #13311: [FLINK-18721] Migrate YarnResourceManager to the new YarnResourceManagerDriver

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13311:
URL: https://github.com/apache/flink/pull/13311#issuecomment-686196434


   
   ## CI report:
   
   * 6d73ba60af6a8aa17399a144e2414f197f40ed21 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6126)
 
   * b7660009b48d48261b767855d8973c63741ab493 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] shuiqiangchen commented on pull request #13230: [FLINK-18950][python][docs] Add documentation for Operations in Python DataStream API.

2020-09-02 Thread GitBox


shuiqiangchen commented on pull request #13230:
URL: https://github.com/apache/flink/pull/13230#issuecomment-686216502


   @HuangXingBo, highly appreciate for your comments, I have updated the PR, 
please have a look at it.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] Shawn-Hx commented on pull request #13279: [FLINK-19090][docs-zh] Translate "Local Cluster" of "Clusters & Depolyment" page into Chinese

2020-09-02 Thread GitBox


Shawn-Hx commented on pull request #13279:
URL: https://github.com/apache/flink/pull/13279#issuecomment-686217970


   Hi, @klion26 
   This PR is ready to be reviewed. Could you please help to review it at your 
convenience ?
   Thank you !



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink-statefun] tzulitai commented on pull request #140: [FLINK-19019] [docker] Add common filesystem plugins to flink-statefun image

2020-09-02 Thread GitBox


tzulitai commented on pull request #140:
URL: https://github.com/apache/flink-statefun/pull/140#issuecomment-686215727


   thanks, merging!



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Closed] (FLINK-19019) Add HDFS / S3 / GCS support to Flink-StateFun Docker image.

2020-09-02 Thread Tzu-Li (Gordon) Tai (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-19019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tzu-Li (Gordon) Tai closed FLINK-19019.
---
Resolution: Fixed

Fixed in statefun-master via 3978b1ff2361c2727a7ce3d0cd5d111a34d59a61

> Add HDFS / S3 / GCS support to Flink-StateFun Docker image.
> ---
>
> Key: FLINK-19019
> URL: https://issues.apache.org/jira/browse/FLINK-19019
> Project: Flink
>  Issue Type: Improvement
>  Components: Stateful Functions
>Reporter: Igal Shilman
>Assignee: Tzu-Li (Gordon) Tai
>Priority: Major
>  Labels: pull-request-available
>
> The 3rd party filesystem support is not enabled by default. To make 
> checkpointing work in s3 / gcs etc' we need to add the relevant plugins.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [flink-statefun] tzulitai closed pull request #140: [FLINK-19019] [docker] Add common filesystem plugins to flink-statefun image

2020-09-02 Thread GitBox


tzulitai closed pull request #140:
URL: https://github.com/apache/flink-statefun/pull/140


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #13232: [FLINK-18951][python][docs] Add documentation for Configurations in Python DataStream API.

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13232:
URL: https://github.com/apache/flink/pull/13232#issuecomment-679225070


   
   ## CI report:
   
   * 99c5a7392b35bef45802cee74c59667e64e8f239 Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6078)
 
   * 492cf7e87bfb8ac6f3fafc73617345b1de1eefe5 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] dianfu commented on pull request #13304: [FLINK-19118][python] Support Expression in the operations of Python Table API

2020-09-02 Thread GitBox


dianfu commented on pull request #13304:
URL: https://github.com/apache/flink/pull/13304#issuecomment-686217682


   @WeiZhong94 Thanks for the review. Updated the PR.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] shuiqiangchen commented on a change in pull request #13230: [FLINK-18950][python][docs] Add documentation for Operations in Python DataStream API.

2020-09-02 Thread GitBox


shuiqiangchen commented on a change in pull request #13230:
URL: https://github.com/apache/flink/pull/13230#discussion_r482670348



##
File path: docs/dev/python/user-guide/datastream/operators.md
##
@@ -0,0 +1,87 @@
+---
+title: "Operators"
+nav-parent_id: python_datastream_api
+nav-pos: 20
+---
+
+
+
+Operators transform one or more DataStreams into a new DataStream. Programs 
can combine multiple transformations into 
+sophisticated dataflow topologies.
+
+This section give a description of the basic transformations Python DataStream 
API provides, the effective physical 
+partitioning after applying those as well as insights into Flink's operator 
chaining.
+
+* This will be replaced by the TOC
+{:toc}
+
+# DataStream Transformations
+
+DataStream programs in Flink are regular programs that implement 
transformations on data streams (e.g., mapping, 
+filtering, reducing). Please see [operators]({% link 
dev/stream/operators/index.md %}
+?code_tab=python) for an overview of the available stream transformations in 
Python DataStream API.
+
+# Functions
+Most operators require a user-defined function. The following will describe 
different ways of how they can be specified.
+
+## Implementing Function Interfaces
+Function interfaces for different operations are provided in Python DataStream 
API. Users can implement a Function and 
+pass it to the corresponding operation. Take MapFunction for instance:
+
+{% highlight python %}
+# Implement a MapFunction that returns plus one value of input value.
+class MyMapFunction(MapFunction):
+
+def map(value):
+return value + 1
+
+data_stream = env.from_collection([1, 2, 3, 4, 5],type_info=Types.INT())
+mapped_stream = data_stream.map(MyMapFunction(), output_type=Types.INT())
+{% endhighlight %}
+
+Note In Python DataStream API, users are 
able to defined the output type information of the operation. If not 
+defined, the output type will be `Types.PICKLED_BYTE_ARRAY` so that data will 
be in a form of byte array generated by 
+pickle seriallizer. For more details about the `Pickle Serialization`, please 
refer to [DataTypes]({% link dev/python/user-guide/datastream/data_types.md
+ %}#pickle-serialization).
+
+## Lambda Functions
+As shown in previous examples, all operations can also accept a lambda 
function for describing the operation:

Review comment:
   The 'previous examples' means the example in index page, here is a more 
concrete illustration.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Updated] (FLINK-19129) Helm charts are missing the latest log4j-console file

2020-09-02 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-19129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated FLINK-19129:
---
Labels: pull-request-available  (was: )

> Helm charts are missing the latest log4j-console file
> -
>
> Key: FLINK-19129
> URL: https://issues.apache.org/jira/browse/FLINK-19129
> Project: Flink
>  Issue Type: Improvement
>  Components: Stateful Functions
>Reporter: Igal Shilman
>Priority: Major
>  Labels: pull-request-available
> Fix For: statefun-2.2.0
>
>
> Our Helm charts still contain the old log4j file (from flink-1.10.x) but 
> logging was changed in Flink 1.11.x.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [flink] flinkbot edited a comment on pull request #13227: [FLINK-18959][Runtime] Try to revert MiniDispatcher for archiveExecutionGraph

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13227:
URL: https://github.com/apache/flink/pull/13227#issuecomment-679046587


   
   ## CI report:
   
   * 213695b1635186819bcde49ed3c94f0238943ee2 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5841)
 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5838)
 
   * 160a51baaf35f77f30c65ce358116446c007d4f3 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6124)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #13295: [FLINK-18598][python][docs] Add instructions for asynchronous execute in PyFlink doc.

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13295:
URL: https://github.com/apache/flink/pull/13295#issuecomment-684586420


   
   ## CI report:
   
   * 9f2e570c493ec1ca97e2c2ef1e73a6e227117016 Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6049)
 
   * f67e466f192eaecfc167467334d844a571123afe Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6125)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink-statefun] tzulitai commented on pull request #141: [FLINK-19129] [k8s] Update log4j-console in template Helm chart

2020-09-02 Thread GitBox


tzulitai commented on pull request #141:
URL: https://github.com/apache/flink-statefun/pull/141#issuecomment-686215615


   cc @igalshilman 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink-statefun] tzulitai opened a new pull request #141: [FLINK-19129] [k8s] Update log4j-console in template Helm chart

2020-09-02 Thread GitBox


tzulitai opened a new pull request #141:
URL: https://github.com/apache/flink-statefun/pull/141


   This needs to be updated since logging was changed in Flink from 1.10.x to 
1.11.x.
   
   The new contents is identical to how it is in Flink 1.11.x: 
https://github.com/apache/flink/blob/release-1.11/flink-dist/src/main/flink-bin/conf/log4j-console.properties



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Commented] (FLINK-19125) Avoid memory fragmentation when running flink docker image

2020-09-02 Thread Yang Wang (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-19125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189800#comment-17189800
 ] 

Yang Wang commented on FLINK-19125:
---

[~yunta] Thanks a lot for the investigation. I believe your analysis in on 
point. Just have one quick question. Could we upgrade the JDK version to fix 
the memory issue? I think the higher version JDK may have the newer 
{{libjemalloc.so}}.

> Avoid memory fragmentation when running flink docker image
> --
>
> Key: FLINK-19125
> URL: https://issues.apache.org/jira/browse/FLINK-19125
> Project: Flink
>  Issue Type: Improvement
>  Components: Deployment / Kubernetes, Runtime / State Backends
>Affects Versions: 1.11.1
>Reporter: Yun Tang
>Priority: Major
>
> This ticket tracks the problem of memory fragmentation when launching default 
> Flink docker image.
> In FLINK-18712, user reported if he submits job with rocksDB state backend on 
> a k8s session cluster again and again once it finished, the memory usage of 
> task manager grows continuously until OOM killed. 
>  I reproduce this problem with official Flink docker image no matter how we 
> use rocksDB (whether to enable managed memory).
> I dig into the problem and found this is due to the memory fragmentation 
> caused by {{glibc}}, which would not return memory to kernel gracefully 
> (please refer to [glibc 
> bugzilla|https://sourceware.org/bugzilla/show_bug.cgi?id=15321] and [glibc 
> manual|https://www.gnu.org/software/libc/manual/html_mono/libc.html#Freeing-after-Malloc])
> I found if limiting MALLOC_ARENA_MAX to 2 could mitigate this problem (please 
> refer to 
> [choose-for-malloc_arena_max|https://devcenter.heroku.com/articles/tuning-glibc-memory-behavior#what-value-to-choose-for-malloc_arena_max]
>  for more details).
> And if we choose to use jemalloc to allocate memory via rebuilding another 
> docker image, the problem would be gone. 
> {code:java}
> apt-get -y install libjemalloc-dev
> ENV LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so
> {code}
> Jemalloc intends to [emphasize fragmentation 
> avoidance|https://github.com/jemalloc/jemalloc /wiki/Background#intended-use] 
> and we might consider to re-factor our Dockerfile to base on jemalloc to 
> avoid memory fragmentation.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [flink] HuangXingBo commented on pull request #13232: [FLINK-18951][python][docs] Add documentation for Configurations in Python DataStream API.

2020-09-02 Thread GitBox


HuangXingBo commented on pull request #13232:
URL: https://github.com/apache/flink/pull/13232#issuecomment-686214805


   @shuiqiangchen Thanks a lot for the update. LGTM



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Commented] (FLINK-19054) KafkaTableITCase.testKafkaSourceSink hangs

2020-09-02 Thread Dian Fu (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-19054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189801#comment-17189801
 ] 

Dian Fu commented on FLINK-19054:
-

[https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=6117=logs=c5f0071e-1851-543e-9a45-9ac140befc32=684b1416-4c17-504e-d5ab-97ee44e08a20]

> KafkaTableITCase.testKafkaSourceSink hangs
> --
>
> Key: FLINK-19054
> URL: https://issues.apache.org/jira/browse/FLINK-19054
> Project: Flink
>  Issue Type: Bug
>  Components: Connectors / Kafka, Table SQL / API
>Affects Versions: 1.11.2
>Reporter: Dian Fu
>Priority: Major
>  Labels: test-stability
>
> [https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=5844=logs=c5f0071e-1851-543e-9a45-9ac140befc32=684b1416-4c17-504e-d5ab-97ee44e08a20]
> {code}
> 2020-08-25T09:04:57.3569768Z "Kafka Fetcher for Source: 
> KafkaTableSource(price, currency, log_date, log_time, log_ts) -> 
> SourceConversion(table=[default_catalog.default_database.kafka, source: 
> [KafkaTableSource(price, currency, log_date, log_time, log_ts)]], 
> fields=[price, currency, log_date, log_time, log_ts]) -> Calc(select=[(price 
> + 1.0:DECIMAL(2, 1)) AS computed-price, price, currency, log_date, log_time, 
> log_ts, (log_ts + 1000:INTERVAL SECOND) AS ts]) -> 
> WatermarkAssigner(rowtime=[ts], watermark=[ts]) -> Calc(select=[ts, log_date, 
> log_time, CAST(ts) AS ts0, price]) (1/1)" #1501 daemon prio=5 os_prio=0 
> tid=0x7f25b800 nid=0x22b8 runnable [0x7f2127efd000]
> 2020-08-25T09:04:57.3571373Zjava.lang.Thread.State: RUNNABLE
> 2020-08-25T09:04:57.3571672Z  at sun.nio.ch.FileDispatcherImpl.read0(Native 
> Method)
> 2020-08-25T09:04:57.3572191Z  at 
> sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
> 2020-08-25T09:04:57.3572921Z  at 
> sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
> 2020-08-25T09:04:57.3573419Z  at sun.nio.ch.IOUtil.read(IOUtil.java:197)
> 2020-08-25T09:04:57.3573957Z  at 
> sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:377)
> 2020-08-25T09:04:57.3574809Z  - locked <0xfde5a308> (a 
> java.lang.Object)
> 2020-08-25T09:04:57.3575448Z  at 
> org.apache.kafka.common.network.PlaintextTransportLayer.read(PlaintextTransportLayer.java:103)
> 2020-08-25T09:04:57.3576309Z  at 
> org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:117)
> 2020-08-25T09:04:57.3577086Z  at 
> org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:424)
> 2020-08-25T09:04:57.3577727Z  at 
> org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:385)
> 2020-08-25T09:04:57.3578403Z  at 
> org.apache.kafka.common.network.Selector.attemptRead(Selector.java:651)
> 2020-08-25T09:04:57.3579486Z  at 
> org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:572)
> 2020-08-25T09:04:57.3580240Z  at 
> org.apache.kafka.common.network.Selector.poll(Selector.java:483)
> 2020-08-25T09:04:57.3580880Z  at 
> org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:547)
> 2020-08-25T09:04:57.3581756Z  at 
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:262)
> 2020-08-25T09:04:57.3583015Z  at 
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:233)
> 2020-08-25T09:04:57.3583847Z  at 
> org.apache.kafka.clients.consumer.KafkaConsumer.pollForFetches(KafkaConsumer.java:1300)
> 2020-08-25T09:04:57.3584555Z  at 
> org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1240)
> 2020-08-25T09:04:57.3585197Z  at 
> org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1168)
> 2020-08-25T09:04:57.3585961Z  at 
> org.apache.flink.streaming.connectors.kafka.internal.KafkaConsumerThread.run(KafkaConsumerThread.java:253)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [flink] shuiqiangchen commented on pull request #13232: [FLINK-18951][python][docs] Add documentation for Configurations in Python DataStream API.

2020-09-02 Thread GitBox


shuiqiangchen commented on pull request #13232:
URL: https://github.com/apache/flink/pull/13232#issuecomment-686207820


   @HuangXingBo Thank you for your comments, I have updated the PR according to 
your suggestions, please have a look.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot commented on pull request #13311: [FLINK-18721] Migrate YarnResourceManager to the new YarnResourceManagerDriver

2020-09-02 Thread GitBox


flinkbot commented on pull request #13311:
URL: https://github.com/apache/flink/pull/13311#issuecomment-686196434


   
   ## CI report:
   
   * 6d73ba60af6a8aa17399a144e2414f197f40ed21 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] WeiZhong94 commented on a change in pull request #13304: [FLINK-19118][python] Support Expression in the operations of Python Table API

2020-09-02 Thread GitBox


WeiZhong94 commented on a change in pull request #13304:
URL: https://github.com/apache/flink/pull/13304#discussion_r482665098



##
File path: flink-python/pyflink/table/tests/test_window.py
##
@@ -40,8 +41,8 @@ class BatchTableWindowTests(PyFlinkBatchTableTestCase):
 
 def test_tumble_window(self):
 t = self.t_env.from_elements([(1, 1, "Hello")], ["a", "b", "c"])
-result = t.window(Tumble.over("2.rows").on("a").alias("w"))\
-.group_by("w, c").select("b.sum")
+result = t.window(Tumble.over(E.lit(2).rows).on("a").alias("w"))\

Review comment:
   ".rows" seems not supported. Maybe we should support it or use 
"row_interval()" instead?

##
File path: flink-python/pyflink/table/tests/test_pandas_udf.py
##
@@ -58,8 +59,8 @@ def test_basic_functionality(self):
 
 t = self.t_env.from_elements([(1, 2, 3), (2, 5, 6), (3, 1, 9)], ['a', 
'b', 'c'])
 exec_insert_table(
-t.where("add_one(b) <= 3").select("a, b + 1, add(a + 1, 
subtract_one(c)) + 2, "
-  "add(add_one(a), 1L)"),
+t.where(E.call('add_one', t.b) <= 3)
+ .select("a, b + 1, add(a + 1, subtract_one(c)) + 2, " 
"add(add_one(a), 1L)"),

Review comment:
   combine to one string?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #13295: [FLINK-18598][python][docs] Add instructions for asynchronous execute in PyFlink doc.

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13295:
URL: https://github.com/apache/flink/pull/13295#issuecomment-684586420


   
   ## CI report:
   
   * 9f2e570c493ec1ca97e2c2ef1e73a6e227117016 Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6049)
 
   * f67e466f192eaecfc167467334d844a571123afe UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #13309: [Flink-14942][state-processor-api] savepoint deep copy

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13309:
URL: https://github.com/apache/flink/pull/13309#issuecomment-685901869


   
   ## CI report:
   
   * 7691536155927e3b0a665322bb8146a2e7659d8c Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6120)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #13307: [FLINK-19078][table-runtime] Import rowtime join temporal operator

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13307:
URL: https://github.com/apache/flink/pull/13307#issuecomment-685763264


   
   ## CI report:
   
   * f148274d5bcdf7177561a2f659ba9fc718af0054 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6106)
 
   * da1db6af426818bc3bd0224c244dd201bd81799a Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6123)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Reopened] (FLINK-18815) AbstractCloseableRegistryTest.testClose unstable

2020-09-02 Thread Dian Fu (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-18815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dian Fu reopened FLINK-18815:
-

> AbstractCloseableRegistryTest.testClose unstable
> 
>
> Key: FLINK-18815
> URL: https://issues.apache.org/jira/browse/FLINK-18815
> Project: Flink
>  Issue Type: Bug
>  Components: FileSystems, Tests
>Affects Versions: 1.10.1, 1.12.0, 1.11.1
>Reporter: Robert Metzger
>Assignee: Kezhu Wang
>Priority: Critical
>  Labels: pull-request-available, test-stability
> Fix For: 1.10.2, 1.12.0, 1.11.2
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=5164=logs=0da23115-68bb-5dcd-192c-bd4c8adebde1=05b74a19-4ee4-5036-c46f-ada307df6cf0
> {code}
> [ERROR] Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.509 
> s <<< FAILURE! - in org.apache.flink.core.fs.SafetyNetCloseableRegistryTest
> [ERROR] testClose(org.apache.flink.core.fs.SafetyNetCloseableRegistryTest)  
> Time elapsed: 1.15 s  <<< FAILURE!
> java.lang.AssertionError: expected:<0> but was:<-1>
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.failNotEquals(Assert.java:834)
>   at org.junit.Assert.assertEquals(Assert.java:645)
>   at org.junit.Assert.assertEquals(Assert.java:631)
>   at 
> org.apache.flink.core.fs.AbstractCloseableRegistryTest.testClose(AbstractCloseableRegistryTest.java:93)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [flink] flinkbot edited a comment on pull request #13227: [FLINK-18959][Runtime] Try to revert MiniDispatcher for archiveExecutionGraph

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13227:
URL: https://github.com/apache/flink/pull/13227#issuecomment-679046587


   
   ## CI report:
   
   * 213695b1635186819bcde49ed3c94f0238943ee2 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5841)
 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=5838)
 
   * 160a51baaf35f77f30c65ce358116446c007d4f3 UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Commented] (FLINK-18815) AbstractCloseableRegistryTest.testClose unstable

2020-09-02 Thread Dian Fu (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-18815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189796#comment-17189796
 ] 

Dian Fu commented on FLINK-18815:
-

This issue reproduced on master:
[https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=6116=logs=3b6ec2fd-a816-5e75-c775-06fb87cb6670=2aff8966-346f-518f-e6ce-de64002a5034]

> AbstractCloseableRegistryTest.testClose unstable
> 
>
> Key: FLINK-18815
> URL: https://issues.apache.org/jira/browse/FLINK-18815
> Project: Flink
>  Issue Type: Bug
>  Components: FileSystems, Tests
>Affects Versions: 1.10.1, 1.12.0, 1.11.1
>Reporter: Robert Metzger
>Assignee: Kezhu Wang
>Priority: Critical
>  Labels: pull-request-available, test-stability
> Fix For: 1.10.2, 1.12.0, 1.11.2
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=5164=logs=0da23115-68bb-5dcd-192c-bd4c8adebde1=05b74a19-4ee4-5036-c46f-ada307df6cf0
> {code}
> [ERROR] Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.509 
> s <<< FAILURE! - in org.apache.flink.core.fs.SafetyNetCloseableRegistryTest
> [ERROR] testClose(org.apache.flink.core.fs.SafetyNetCloseableRegistryTest)  
> Time elapsed: 1.15 s  <<< FAILURE!
> java.lang.AssertionError: expected:<0> but was:<-1>
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.failNotEquals(Assert.java:834)
>   at org.junit.Assert.assertEquals(Assert.java:645)
>   at org.junit.Assert.assertEquals(Assert.java:631)
>   at 
> org.apache.flink.core.fs.AbstractCloseableRegistryTest.testClose(AbstractCloseableRegistryTest.java:93)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (FLINK-19127) Provide a replacement of StreamExecutionEnvironment.createRemoteEnvironment for TableEnvironment

2020-09-02 Thread Jark Wu (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-19127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189797#comment-17189797
 ] 

Jark Wu commented on FLINK-19127:
-

I think we can leverage the existing runtime configurations, such as 
{{execution.target}}, {{jobmanager.rpc.port}}, {{jobmanager.rpc.port}}.
So that we don't need to introduce any new interface, users only need to set 
the configurations in {{TableConfig}} and TableEnvironment will use them to 
create the correct {{PipelineExecutor}}.

> Provide a replacement of StreamExecutionEnvironment.createRemoteEnvironment 
> for TableEnvironment
> 
>
> Key: FLINK-19127
> URL: https://issues.apache.org/jira/browse/FLINK-19127
> Project: Flink
>  Issue Type: Improvement
>  Components: Table SQL / API
>Reporter: Timo Walther
>Priority: Major
>
> Connecting to a remote cluster from the unified TableEnvironment is neither 
> tested nor documented. Since StreamExecutionEnvironment is not necessary 
> anymore, users should be able to do the same in TableEnvironment easily. This 
> is in particular useful for interactive sessions that run in an IDE.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [flink] HuangXingBo edited a comment on pull request #13295: [FLINK-18598][python][docs] Add instructions for asynchronous execute in PyFlink doc.

2020-09-02 Thread GitBox


HuangXingBo edited a comment on pull request #13295:
URL: https://github.com/apache/flink/pull/13295#issuecomment-686193377


   @shuiqiangchen Thanks for the update. LGTM



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] HuangXingBo commented on pull request #13295: [FLINK-18598][python][docs] Add instructions for asynchronous execute in PyFlink doc.

2020-09-02 Thread GitBox


HuangXingBo commented on pull request #13295:
URL: https://github.com/apache/flink/pull/13295#issuecomment-686193377


   @shuiqiangchen Thanks for the PR. LGTM



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] tzulitai commented on pull request #12944: [FLINK-18513][Kinesis] Add AWS SDK v2.x dependency and KinesisProxyV2

2020-09-02 Thread GitBox


tzulitai commented on pull request #12944:
URL: https://github.com/apache/flink/pull/12944#issuecomment-686192678


   @dannycranmer 
   
   I've been fixing a few bugs in the PR that has failed CI, before merging it.
   So far have bumped into this:
   
   ```
   2020-09-02T10:18:54.5913921Z [ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-shade-plugin:3.1.1:shade (shade-flink) on 
project flink-connector-kinesis_2.11: Error creating shaded jar: duplicate 
entry: 
META-INF/services/org.apache.flink.kinesis.shaded.software.amazon.awssdk.http.async.SdkAsyncHttpService
 -> [Help 1]
   2020-09-02T10:18:54.5915252Z [ERROR] 
   2020-09-02T10:18:54.5916062Z [ERROR] To see the full stack trace of the 
errors, re-run Maven with the -e switch.
   2020-09-02T10:18:54.5916918Z [ERROR] Re-run Maven using the -X switch to 
enable full debug logging.
   2020-09-02T10:18:54.5917384Z [ERROR] 
   2020-09-02T10:18:54.5918070Z [ERROR] For more information about the errors 
and possible solutions, please read the following articles:
   2020-09-02T10:18:54.5918907Z [ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
   2020-09-02T10:18:54.5919331Z [ERROR] 
   2020-09-02T10:18:54.5919700Z [ERROR] After correcting the problems, you can 
resume the build with the command
   2020-09-02T10:18:54.5920396Z [ERROR]   mvn  -rf 
:flink-connector-kinesis_2.11
   ```
   
   Original build logs:
   
https://dev.azure.com/tzulitai/tzulitai-flink/_build/results?buildId=35=logs=66592496-52df-56bb-d03e-37509e1d9d0f=66592496-52df-56bb-d03e-37509e1d9d0f=ae0269db-6796-5583-2e5f-d84757d711aa



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] tzulitai edited a comment on pull request #12944: [FLINK-18513][Kinesis] Add AWS SDK v2.x dependency and KinesisProxyV2

2020-09-02 Thread GitBox


tzulitai edited a comment on pull request #12944:
URL: https://github.com/apache/flink/pull/12944#issuecomment-686192678


   @dannycranmer 
   
   I've been fixing a few bugs in the PR that has failed CI, before merging it.
   So far have bumped into this, could you take a look?
   
   ```
   2020-09-02T10:18:54.5913921Z [ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-shade-plugin:3.1.1:shade (shade-flink) on 
project flink-connector-kinesis_2.11: Error creating shaded jar: duplicate 
entry: 
META-INF/services/org.apache.flink.kinesis.shaded.software.amazon.awssdk.http.async.SdkAsyncHttpService
 -> [Help 1]
   2020-09-02T10:18:54.5915252Z [ERROR] 
   2020-09-02T10:18:54.5916062Z [ERROR] To see the full stack trace of the 
errors, re-run Maven with the -e switch.
   2020-09-02T10:18:54.5916918Z [ERROR] Re-run Maven using the -X switch to 
enable full debug logging.
   2020-09-02T10:18:54.5917384Z [ERROR] 
   2020-09-02T10:18:54.5918070Z [ERROR] For more information about the errors 
and possible solutions, please read the following articles:
   2020-09-02T10:18:54.5918907Z [ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
   2020-09-02T10:18:54.5919331Z [ERROR] 
   2020-09-02T10:18:54.5919700Z [ERROR] After correcting the problems, you can 
resume the build with the command
   2020-09-02T10:18:54.5920396Z [ERROR]   mvn  -rf 
:flink-connector-kinesis_2.11
   ```
   
   Original build logs:
   
https://dev.azure.com/tzulitai/tzulitai-flink/_build/results?buildId=35=logs=66592496-52df-56bb-d03e-37509e1d9d0f=66592496-52df-56bb-d03e-37509e1d9d0f=ae0269db-6796-5583-2e5f-d84757d711aa



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] shuiqiangchen commented on pull request #13295: [FLINK-18598][python][docs] Add instructions for asynchronous execute in PyFlink doc.

2020-09-02 Thread GitBox


shuiqiangchen commented on pull request #13295:
URL: https://github.com/apache/flink/pull/13295#issuecomment-686192102


   @HuangXingBo Thank you for your comments, I have updated the PR according to 
your comments, please have a look.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Commented] (FLINK-19126) Failed to run job in yarn-cluster mode due to No Executor found.

2020-09-02 Thread Congxian Qiu(klion26) (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-19126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189788#comment-17189788
 ] 

Congxian Qiu(klion26) commented on FLINK-19126:
---

Hi [~Tang Yan] As the exception said, could you please have a try after export 
{{HADOOP_CLASSPATH}}?

> Failed to run job in yarn-cluster mode due to No Executor found.
> 
>
> Key: FLINK-19126
> URL: https://issues.apache.org/jira/browse/FLINK-19126
> Project: Flink
>  Issue Type: Bug
>  Components: Deployment / YARN
>Affects Versions: 1.11.1
>Reporter: Tang Yan
>Priority: Major
>
> I've build the flink package successfully, but when I run the below command, 
> it failed to submit the jobs.
> [yanta@flink-1.11]$ bin/flink run -m yarn-cluster -p 2 -c 
> org.apache.flink.examples.java.wordcount.WordCount 
> examples/batch/WordCount.jar  --input hdfs:///user/yanta/aa.txt --output 
> hdfs:///user/yanta/result.txt
> Setting HADOOP_CONF_DIR=/etc/hadoop/conf because no HADOOP_CONF_DIR or 
> HADOOP_CLASSPATH was set.
>  The program 
> finished with the following exception:
> java.lang.IllegalStateException: No Executor found. Please make sure to 
> export the HADOOP_CLASSPATH environment variable or have hadoop in your 
> classpath. For more information refer to the "Deployment & Operations" 
> section of the official Apache Flink documentation. at 
> org.apache.flink.yarn.cli.FallbackYarnSessionCli.isActive(FallbackYarnSessionCli.java:59)
>  at 
> org.apache.flink.client.cli.CliFrontend.validateAndGetActiveCommandLine(CliFrontend.java:1090)
>  at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:218) at 
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:916) 
> at 
> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:992) 
> at 
> org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
>  at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:992)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [flink] flinkbot edited a comment on pull request #13309: [Flink-14942][state-processor-api] savepoint deep copy

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13309:
URL: https://github.com/apache/flink/pull/13309#issuecomment-685901869


   
   ## CI report:
   
   * d6026937ceea29e808b8a4e6e2846de209ee831f Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6119)
 
   * 7691536155927e3b0a665322bb8146a2e7659d8c Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6120)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot commented on pull request #13311: [FLINK-18721] Migrate YarnResourceManager to the new YarnResourceManagerDriver

2020-09-02 Thread GitBox


flinkbot commented on pull request #13311:
URL: https://github.com/apache/flink/pull/13311#issuecomment-686191324


   Thanks a lot for your contribution to the Apache Flink project. I'm the 
@flinkbot. I help the community
   to review your pull request. We will use this comment to track the progress 
of the review.
   
   
   ## Automated Checks
   Last check on commit 6d73ba60af6a8aa17399a144e2414f197f40ed21 (Thu Sep 03 
02:15:44 UTC 2020)
   
   **Warnings:**
* No documentation files were touched! Remember to keep the Flink docs up 
to date!
   
   
   Mention the bot in a comment to re-run the automated checks.
   ## Review Progress
   
   * ❓ 1. The [description] looks good.
   * ❓ 2. There is [consensus] that the contribution should go into to Flink.
   * ❓ 3. Needs [attention] from.
   * ❓ 4. The change fits into the overall [architecture].
   * ❓ 5. Overall code [quality] is good.
   
   Please see the [Pull Request Review 
Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full 
explanation of the review process.
The Bot is tracking the review progress through labels. Labels are applied 
according to the order of the review items. For consensus, approval by a Flink 
committer of PMC member is required Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot approve description` to approve one or more aspects (aspects: 
`description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until 
`architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's 
attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #13307: [FLINK-19078][table-runtime] Import rowtime join temporal operator

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13307:
URL: https://github.com/apache/flink/pull/13307#issuecomment-685763264


   
   ## CI report:
   
   * f148274d5bcdf7177561a2f659ba9fc718af0054 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6106)
 
   * da1db6af426818bc3bd0224c244dd201bd81799a UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Updated] (FLINK-18721) Migrate YarnResourceManager to the new YarnResourceManagerDriver

2020-09-02 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-18721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated FLINK-18721:
---
Labels: pull-request-available  (was: )

> Migrate YarnResourceManager to the new YarnResourceManagerDriver
> 
>
> Key: FLINK-18721
> URL: https://issues.apache.org/jira/browse/FLINK-18721
> Project: Flink
>  Issue Type: Sub-task
>  Components: Deployment / YARN, Runtime / Coordination
>Reporter: Xintong Song
>Assignee: Yangze Guo
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.12.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [flink] flinkbot edited a comment on pull request #13217: [FLINK-16866] Make job submission non-blocking

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13217:
URL: https://github.com/apache/flink/pull/13217#issuecomment-678285884


   
   ## CI report:
   
   * 3655fcea1966bfbcb85c86d6a159c354f20d6cc7 UNKNOWN
   * 31d810f7c4d93043d8444a78f921b263f1258646 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6118)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #13128: [FLINK-18795][hbase] Support for HBase 2

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13128:
URL: https://github.com/apache/flink/pull/13128#issuecomment-672766836


   
   ## CI report:
   
   * 313b80f4474455d7013b0852929b5a8458f391a1 UNKNOWN
   * 4797c5f9ce51ad9763e50a3647a86c4a5e7bdf02 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6110)
 
   * 26bfe01baddf52cb691082fcd97542c73828f18c Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6121)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] KarmaGYZ opened a new pull request #13311: [FLINK-18721] Migrate YarnResourceManager to the new YarnResourceManagerDriver

2020-09-02 Thread GitBox


KarmaGYZ opened a new pull request #13311:
URL: https://github.com/apache/flink/pull/13311


   
   
   ## What is the purpose of the change
   
   Migrate YarnResourceManager to the new YarnResourceManagerDriver
   
   ## Brief change log
   
   - 1716752 : Introduce the `TaskExecutorProcessSpecContainerResourceAdapter`, 
which is the substitute of `WorkerResourceSpecContainerResourceAdapter`.
   - 15c7b09 : Add Builder class for `TestingYarnNMClientAsync` and 
`TestingYarnAMRMClientAsync`.
   - 6869e49 : Introduce the `YarnResourceManagerDriver`.
   - 6d73ba6 : Switch the yarn deployment to new `ActiveResourceManager`
   
   
   ## Verifying this change
   
   This change added tests and can be verified as follows:
   
   - Existing UTs, IT cases and e2e tests.
   - `YarnResourceManagerDriverTest`
   - `TaskExecutorProcessSpecContainerResourceAdapterTest`
   
   ## Does this pull request potentially affect one of the following parts:
   
 - Dependencies (does it add or upgrade a dependency): no
 - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: no
 - The serializers: no
 - The runtime per-record code paths (performance sensitive): no
 - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: yes
 - The S3 file system connector: no
   
   ## Documentation
   
 - Does this pull request introduce a new feature? no
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] dianfu commented on a change in pull request #13297: [FLINK-19105][docs]update demo error

2020-09-02 Thread GitBox


dianfu commented on a change in pull request #13297:
URL: https://github.com/apache/flink/pull/13297#discussion_r482651242



##
File path: docs/dev/table/common.md
##
@@ -743,9 +743,9 @@ TableEnvironment tableEnv = ...; // see "Create a 
TableEnvironment" section
 final Schema schema = new Schema()
 .field("a", DataTypes.INT())
 .field("b", DataTypes.STRING())
-.field("c", DataTypes.LONG());
+.field("c", DataTypes.BIGINT());
 
-tableEnv.connect(new FileSystem("/path/to/file"))
+tableEnv.connect(new FileSystem().path("/path/to/file"))

Review comment:
   This error exists in several places in the documentation. Could you also 
update them?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Updated] (FLINK-19110) Flatten current PyFlink documentation structure

2020-09-02 Thread Dian Fu (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-19110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dian Fu updated FLINK-19110:

Component/s: API / Python

> Flatten current PyFlink documentation structure
> ---
>
> Key: FLINK-19110
> URL: https://issues.apache.org/jira/browse/FLINK-19110
> Project: Flink
>  Issue Type: Sub-task
>  Components: API / Python, Documentation
>Reporter: Wei Zhong
>Assignee: Wei Zhong
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.12.0
>
>
> The navigation for this entire section is overly complex. I would much rather 
> see something flatter, like this:
>  * Python API
>  ** Installation
>  ** Table API Tutorial
>  ** Table API User's Guide
>  ** DataStream API User's Guide
>  ** FAQ



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Closed] (FLINK-19110) Flatten current PyFlink documentation structure

2020-09-02 Thread Dian Fu (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-19110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dian Fu closed FLINK-19110.
---
Fix Version/s: 1.12.0
   Resolution: Fixed

master: 85c15064d1e094a6c5cf1b35b728fb70f0370398

> Flatten current PyFlink documentation structure
> ---
>
> Key: FLINK-19110
> URL: https://issues.apache.org/jira/browse/FLINK-19110
> Project: Flink
>  Issue Type: Sub-task
>  Components: Documentation
>Reporter: Wei Zhong
>Assignee: Wei Zhong
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.12.0
>
>
> The navigation for this entire section is overly complex. I would much rather 
> see something flatter, like this:
>  * Python API
>  ** Installation
>  ** Table API Tutorial
>  ** Table API User's Guide
>  ** DataStream API User's Guide
>  ** FAQ



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Assigned] (FLINK-19110) Flatten current PyFlink documentation structure

2020-09-02 Thread Dian Fu (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-19110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dian Fu reassigned FLINK-19110:
---

Assignee: Wei Zhong

> Flatten current PyFlink documentation structure
> ---
>
> Key: FLINK-19110
> URL: https://issues.apache.org/jira/browse/FLINK-19110
> Project: Flink
>  Issue Type: Sub-task
>  Components: Documentation
>Reporter: Wei Zhong
>Assignee: Wei Zhong
>Priority: Major
>  Labels: pull-request-available
>
> The navigation for this entire section is overly complex. I would much rather 
> see something flatter, like this:
>  * Python API
>  ** Installation
>  ** Table API Tutorial
>  ** Table API User's Guide
>  ** DataStream API User's Guide
>  ** FAQ



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Assigned] (FLINK-18721) Migrate YarnResourceManager to the new YarnResourceManagerDriver

2020-09-02 Thread Xintong Song (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-18721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xintong Song reassigned FLINK-18721:


Assignee: Yangze Guo

> Migrate YarnResourceManager to the new YarnResourceManagerDriver
> 
>
> Key: FLINK-18721
> URL: https://issues.apache.org/jira/browse/FLINK-18721
> Project: Flink
>  Issue Type: Sub-task
>  Components: Deployment / YARN, Runtime / Coordination
>Reporter: Xintong Song
>Assignee: Yangze Guo
>Priority: Major
> Fix For: 1.12.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [flink] dianfu closed pull request #13302: [FLINK-19110][docs][python] Flatten current PyFlink documentation structure.

2020-09-02 Thread GitBox


dianfu closed pull request #13302:
URL: https://github.com/apache/flink/pull/13302


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Assigned] (FLINK-18722) Migrate MesosResourceManager to the new MesosResourceManagerDriver

2020-09-02 Thread Xintong Song (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-18722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xintong Song reassigned FLINK-18722:


Assignee: Xintong Song

> Migrate MesosResourceManager to the new MesosResourceManagerDriver
> --
>
> Key: FLINK-18722
> URL: https://issues.apache.org/jira/browse/FLINK-18722
> Project: Flink
>  Issue Type: Sub-task
>  Components: Deployment / Mesos, Runtime / Coordination
>Reporter: Xintong Song
>Assignee: Xintong Song
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [flink] YongjinZhou commented on pull request #13297: [FLINK-19105][docs]update demo error

2020-09-02 Thread GitBox


YongjinZhou commented on pull request #13297:
URL: https://github.com/apache/flink/pull/13297#issuecomment-686186024


   cc @dianfu  could plz have a review?



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] dianfu commented on pull request #13302: [FLINK-19110][docs][python] Flatten current PyFlink documentation structure.

2020-09-02 Thread GitBox


dianfu commented on pull request #13302:
URL: https://github.com/apache/flink/pull/13302#issuecomment-686182786


   @sunjincheng121 Thanks for the remind. Merging...



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Commented] (FLINK-19067) FileNotFoundException when run flink examples on standby JobManager

2020-09-02 Thread JieFang.He (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-19067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189783#comment-17189783
 ] 

JieFang.He commented on FLINK-19067:


[~rmetzger]  This situation does not always happen. My cluster no longer 
appears after reinstallation. I am now trying to reproduce the problem.

> FileNotFoundException when run flink examples on standby JobManager
> ---
>
> Key: FLINK-19067
> URL: https://issues.apache.org/jira/browse/FLINK-19067
> Project: Flink
>  Issue Type: Bug
>  Components: Runtime / Coordination
>Affects Versions: 1.11.1
>Reporter: JieFang.He
>Priority: Major
>
> 1、When run examples/batch/WordCount.jar on standby JobManager,it will fail 
> with the exception:
> Caused by: java.io.FileNotFoundException: 
> /data2/storageDir/default/blob/job_d29414828f614d5466e239be4d3889ac/blob_p-a2ebe1c5aa160595f214b4bd0f39d80e42ee2e93-f458f1c12dc023e78d25f191de1d7c4b
>  (No such file or directory)
>  at java.io.FileInputStream.open0(Native Method)
>  at java.io.FileInputStream.open(FileInputStream.java:195)
>  at java.io.FileInputStream.(FileInputStream.java:138)
>  at 
> org.apache.flink.core.fs.local.LocalDataInputStream.(LocalDataInputStream.java:50)
>  at 
> org.apache.flink.core.fs.local.LocalFileSystem.open(LocalFileSystem.java:143)
>  at 
> org.apache.flink.runtime.blob.FileSystemBlobStore.get(FileSystemBlobStore.java:105)
>  at 
> org.apache.flink.runtime.blob.FileSystemBlobStore.get(FileSystemBlobStore.java:87)
>  at 
> org.apache.flink.runtime.blob.BlobServer.getFileInternal(BlobServer.java:501)
>  at 
> org.apache.flink.runtime.blob.BlobServerConnection.get(BlobServerConnection.java:231)
>  at 
> org.apache.flink.runtime.blob.BlobServerConnection.run(BlobServerConnection.java:117)
>  
> 2、Run examples success on other nodes
> 3、After run success on the other node, it can run success on the Standby 
> JobManager. But run again will fail
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (FLINK-19126) Failed to run job in yarn-cluster mode due to No Executor found.

2020-09-02 Thread Tang Yan (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-19126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tang Yan updated FLINK-19126:
-
Component/s: (was: Legacy Components / Flink on Tez)
 (was: flink-contrib)
 Deployment / YARN

> Failed to run job in yarn-cluster mode due to No Executor found.
> 
>
> Key: FLINK-19126
> URL: https://issues.apache.org/jira/browse/FLINK-19126
> Project: Flink
>  Issue Type: Bug
>  Components: Deployment / YARN
>Affects Versions: 1.11.1
>Reporter: Tang Yan
>Priority: Major
>
> I've build the flink package successfully, but when I run the below command, 
> it failed to submit the jobs.
> [yanta@flink-1.11]$ bin/flink run -m yarn-cluster -p 2 -c 
> org.apache.flink.examples.java.wordcount.WordCount 
> examples/batch/WordCount.jar  --input hdfs:///user/yanta/aa.txt --output 
> hdfs:///user/yanta/result.txt
> Setting HADOOP_CONF_DIR=/etc/hadoop/conf because no HADOOP_CONF_DIR or 
> HADOOP_CLASSPATH was set.
>  The program 
> finished with the following exception:
> java.lang.IllegalStateException: No Executor found. Please make sure to 
> export the HADOOP_CLASSPATH environment variable or have hadoop in your 
> classpath. For more information refer to the "Deployment & Operations" 
> section of the official Apache Flink documentation. at 
> org.apache.flink.yarn.cli.FallbackYarnSessionCli.isActive(FallbackYarnSessionCli.java:59)
>  at 
> org.apache.flink.client.cli.CliFrontend.validateAndGetActiveCommandLine(CliFrontend.java:1090)
>  at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:218) at 
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:916) 
> at 
> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:992) 
> at 
> org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
>  at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:992)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (FLINK-19126) Failed to run job in yarn-cluster mode due to No Executor found.

2020-09-02 Thread Tang Yan (Jira)


 [ 
https://issues.apache.org/jira/browse/FLINK-19126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tang Yan updated FLINK-19126:
-
Component/s: Legacy Components / Flink on Tez

> Failed to run job in yarn-cluster mode due to No Executor found.
> 
>
> Key: FLINK-19126
> URL: https://issues.apache.org/jira/browse/FLINK-19126
> Project: Flink
>  Issue Type: Bug
>  Components: flink-contrib, Legacy Components / Flink on Tez
>Affects Versions: 1.11.1
>Reporter: Tang Yan
>Priority: Major
>
> I've build the flink package successfully, but when I run the below command, 
> it failed to submit the jobs.
> [yanta@flink-1.11]$ bin/flink run -m yarn-cluster -p 2 -c 
> org.apache.flink.examples.java.wordcount.WordCount 
> examples/batch/WordCount.jar  --input hdfs:///user/yanta/aa.txt --output 
> hdfs:///user/yanta/result.txt
> Setting HADOOP_CONF_DIR=/etc/hadoop/conf because no HADOOP_CONF_DIR or 
> HADOOP_CLASSPATH was set.
>  The program 
> finished with the following exception:
> java.lang.IllegalStateException: No Executor found. Please make sure to 
> export the HADOOP_CLASSPATH environment variable or have hadoop in your 
> classpath. For more information refer to the "Deployment & Operations" 
> section of the official Apache Flink documentation. at 
> org.apache.flink.yarn.cli.FallbackYarnSessionCli.isActive(FallbackYarnSessionCli.java:59)
>  at 
> org.apache.flink.client.cli.CliFrontend.validateAndGetActiveCommandLine(CliFrontend.java:1090)
>  at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:218) at 
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:916) 
> at 
> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:992) 
> at 
> org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
>  at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:992)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [flink] flinkbot edited a comment on pull request #13128: [FLINK-18795][hbase] Support for HBase 2

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13128:
URL: https://github.com/apache/flink/pull/13128#issuecomment-672766836


   
   ## CI report:
   
   * 313b80f4474455d7013b0852929b5a8458f391a1 UNKNOWN
   * 4797c5f9ce51ad9763e50a3647a86c4a5e7bdf02 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6110)
 
   * 26bfe01baddf52cb691082fcd97542c73828f18c UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] sunjincheng121 commented on pull request #13302: [FLINK-19110][docs][python] Flatten current PyFlink documentation structure.

2020-09-02 Thread GitBox


sunjincheng121 commented on pull request #13302:
URL: https://github.com/apache/flink/pull/13302#issuecomment-686168662


   @flinkbot approve all



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] sunjincheng121 edited a comment on pull request #13302: [FLINK-19110][docs][python] Flatten current PyFlink documentation structure.

2020-09-02 Thread GitBox


sunjincheng121 edited a comment on pull request #13302:
URL: https://github.com/apache/flink/pull/13302#issuecomment-686168411


   +1 to merge. 
   BTW, @dianfu we can using the Bot commands during  as much as possible.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] sunjincheng121 commented on pull request #13302: [FLINK-19110][docs][python] Flatten current PyFlink documentation structure.

2020-09-02 Thread GitBox


sunjincheng121 commented on pull request #13302:
URL: https://github.com/apache/flink/pull/13302#issuecomment-686168411


   @flinkbot approve all 
   
   +1 to merge. 
   BTW, @dianfu we can using the Bot commands during  as much as possible.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #13310: [FLINK-19086] Revert "[FLINK-19052] [serialization] add constructor c…

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13310:
URL: https://github.com/apache/flink/pull/13310#issuecomment-685961917


   
   ## CI report:
   
   * b486100151ebd674a026078c8615708198435785 Azure: 
[FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6115)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #13217: [FLINK-16866] Make job submission non-blocking

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13217:
URL: https://github.com/apache/flink/pull/13217#issuecomment-678285884


   
   ## CI report:
   
   * 3655fcea1966bfbcb85c86d6a159c354f20d6cc7 UNKNOWN
   * c52eaf11a6832049ad9a18c784d2661101d91b8c Azure: 
[CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6114)
 
   * 31d810f7c4d93043d8444a78f921b263f1258646 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6118)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #13309: [Flink-14942][state-processor-api] savepoint deep copy

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13309:
URL: https://github.com/apache/flink/pull/13309#issuecomment-685901869


   
   ## CI report:
   
   * 06806d984f97f1f4d2384203d376e45dcac19d0d Azure: 
[CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6112)
 
   * d6026937ceea29e808b8a4e6e2846de209ee831f Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6119)
 
   * 7691536155927e3b0a665322bb8146a2e7659d8c Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6120)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #13309: [Flink-14942][state-processor-api] savepoint deep copy

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13309:
URL: https://github.com/apache/flink/pull/13309#issuecomment-685901869


   
   ## CI report:
   
   * 06806d984f97f1f4d2384203d376e45dcac19d0d Azure: 
[CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6112)
 
   * d6026937ceea29e808b8a4e6e2846de209ee831f Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6119)
 
   * 7691536155927e3b0a665322bb8146a2e7659d8c UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #13309: [Flink-14942][state-processor-api] savepoint deep copy

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13309:
URL: https://github.com/apache/flink/pull/13309#issuecomment-685901869


   
   ## CI report:
   
   * 06806d984f97f1f4d2384203d376e45dcac19d0d Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6112)
 
   * d6026937ceea29e808b8a4e6e2846de209ee831f Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6119)
 
   * 7691536155927e3b0a665322bb8146a2e7659d8c UNKNOWN
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [flink] flinkbot edited a comment on pull request #13217: [FLINK-16866] Make job submission non-blocking

2020-09-02 Thread GitBox


flinkbot edited a comment on pull request #13217:
URL: https://github.com/apache/flink/pull/13217#issuecomment-678285884


   
   ## CI report:
   
   * 3655fcea1966bfbcb85c86d6a159c354f20d6cc7 UNKNOWN
   * 08f8c0a7fbfecf7a6eba88653140439f3673c2cc Azure: 
[SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6051)
 
   * c52eaf11a6832049ad9a18c784d2661101d91b8c Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6114)
 
   * 31d810f7c4d93043d8444a78f921b263f1258646 Azure: 
[PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=6118)
 
   
   
   Bot commands
 The @flinkbot bot supports the following commands:
   
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




  1   2   3   4   >