[jira] [Commented] (FLINK-22541) add json format filter params
[
https://issues.apache.org/jira/browse/FLINK-22541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17340006#comment-17340006
]
sandy du commented on FLINK-22541:
--
yes,I will add some codes in JsonRowDataDeserializationSchema.java
!image-2021-05-06-13-38-37-284.png!
> add json format filter params
> --
>
> Key: FLINK-22541
> URL: https://issues.apache.org/jira/browse/FLINK-22541
> Project: Flink
> Issue Type: Improvement
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile), Table
> SQL / Ecosystem
>Affects Versions: 1.11.0, 1.12.0
>Reporter: sandy du
>Priority: Minor
> Attachments: image-2021-05-06-13-37-49-160.png,
> image-2021-05-06-13-38-37-284.png
>
>
> In my case,one kafka topic store multiple table data，for example：
>
> \{"id":"121","source":"users","content":{"name":"test01","age":20,"addr":"addr1"}}
>
> \{"id":"122","source":"users","content":{"name":"test02","age":23,"addr":"addr2"}}
>
> \{"id":"124","source":"users","content":{"name":"test03","age":34,"addr":"addr3"}}
>
> \{"id":"124","source":"order","content":{"orderId":"11","price":34,"addr":"addr1231"}}
>
> \{"id":"125","source":"order","content":{"orderId":"12","price":34,"addr":"addr1232"}}
>
> \{"id":"126","source":"order","content":{"orderId":"13","price":34,"addr":"addr1233"}}
>
> I just want to consume data from talbe order，flink sql ddl like this：
> CREATE TABLE order (
> orderId STRING,
> age INT,
> addr STRING
> )
> with (
> 'connector'='kafka',
> 'topic'='kafkatopic',
> 'properties.bootstrap.servers'='localhost:9092',
> 'properties.group.id'='testGroup',
> 'scan.startup.mode'='earliest-offset',
> 'format'='json',
> 'path-fliter'='$[?(@.source=="order")]',
> 'path-data'='$.content'
> );
>
> path-fliter and path-data can use JsonPath
> ([https://github.com/json-path/JsonPath])
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-22541) add json format filter params
[
https://issues.apache.org/jira/browse/FLINK-22541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sandy du updated FLINK-22541:
-
Attachment: image-2021-05-06-13-38-37-284.png
> add json format filter params
> --
>
> Key: FLINK-22541
> URL: https://issues.apache.org/jira/browse/FLINK-22541
> Project: Flink
> Issue Type: Improvement
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile), Table
> SQL / Ecosystem
>Affects Versions: 1.11.0, 1.12.0
>Reporter: sandy du
>Priority: Minor
> Attachments: image-2021-05-06-13-37-49-160.png,
> image-2021-05-06-13-38-37-284.png
>
>
> In my case,one kafka topic store multiple table data，for example：
>
> \{"id":"121","source":"users","content":{"name":"test01","age":20,"addr":"addr1"}}
>
> \{"id":"122","source":"users","content":{"name":"test02","age":23,"addr":"addr2"}}
>
> \{"id":"124","source":"users","content":{"name":"test03","age":34,"addr":"addr3"}}
>
> \{"id":"124","source":"order","content":{"orderId":"11","price":34,"addr":"addr1231"}}
>
> \{"id":"125","source":"order","content":{"orderId":"12","price":34,"addr":"addr1232"}}
>
> \{"id":"126","source":"order","content":{"orderId":"13","price":34,"addr":"addr1233"}}
>
> I just want to consume data from talbe order，flink sql ddl like this：
> CREATE TABLE order (
> orderId STRING,
> age INT,
> addr STRING
> )
> with (
> 'connector'='kafka',
> 'topic'='kafkatopic',
> 'properties.bootstrap.servers'='localhost:9092',
> 'properties.group.id'='testGroup',
> 'scan.startup.mode'='earliest-offset',
> 'format'='json',
> 'path-fliter'='$[?(@.source=="order")]',
> 'path-data'='$.content'
> );
>
> path-fliter and path-data can use JsonPath
> ([https://github.com/json-path/JsonPath])
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #15712: [FLINK-22400][hive connect]fix NPE problem when convert flink object for Map
flinkbot edited a comment on pull request #15712: URL: https://github.com/apache/flink/pull/15712#issuecomment-824189664 ## CI report: * f2fe08ada02c5c9e20ed397163d3ab7a34594994 UNKNOWN * e90715e4ff43abca28c7da6d8cc719abb857a6f6 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17605) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15683: [FLINK-22362]Don't appear target address when taskmanager connect to jobmanager error
flinkbot edited a comment on pull request #15683: URL: https://github.com/apache/flink/pull/15683#issuecomment-822960823 ## CI report: * b4a559f1f2fd1aea9d614fe3e13c4993651d0fc8 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17606) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-22541) add json format filter params
[
https://issues.apache.org/jira/browse/FLINK-22541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sandy du updated FLINK-22541:
-
Attachment: image-2021-05-06-13-37-49-160.png
> add json format filter params
> --
>
> Key: FLINK-22541
> URL: https://issues.apache.org/jira/browse/FLINK-22541
> Project: Flink
> Issue Type: Improvement
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile), Table
> SQL / Ecosystem
>Affects Versions: 1.11.0, 1.12.0
>Reporter: sandy du
>Priority: Minor
> Attachments: image-2021-05-06-13-37-49-160.png
>
>
> In my case,one kafka topic store multiple table data，for example：
>
> \{"id":"121","source":"users","content":{"name":"test01","age":20,"addr":"addr1"}}
>
> \{"id":"122","source":"users","content":{"name":"test02","age":23,"addr":"addr2"}}
>
> \{"id":"124","source":"users","content":{"name":"test03","age":34,"addr":"addr3"}}
>
> \{"id":"124","source":"order","content":{"orderId":"11","price":34,"addr":"addr1231"}}
>
> \{"id":"125","source":"order","content":{"orderId":"12","price":34,"addr":"addr1232"}}
>
> \{"id":"126","source":"order","content":{"orderId":"13","price":34,"addr":"addr1233"}}
>
> I just want to consume data from talbe order，flink sql ddl like this：
> CREATE TABLE order (
> orderId STRING,
> age INT,
> addr STRING
> )
> with (
> 'connector'='kafka',
> 'topic'='kafkatopic',
> 'properties.bootstrap.servers'='localhost:9092',
> 'properties.group.id'='testGroup',
> 'scan.startup.mode'='earliest-offset',
> 'format'='json',
> 'path-fliter'='$[?(@.source=="order")]',
> 'path-data'='$.content'
> );
>
> path-fliter and path-data can use JsonPath
> ([https://github.com/json-path/JsonPath])
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-22427) Support DEFAULT value syntax to simplify data clean process
[
https://issues.apache.org/jira/browse/FLINK-22427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339984#comment-17339984
]
Moses commented on FLINK-22427:
---
[~jark] Agree.
> Support DEFAULT value syntax to simplify data clean process
> ---
>
> Key: FLINK-22427
> URL: https://issues.apache.org/jira/browse/FLINK-22427
> Project: Flink
> Issue Type: New Feature
> Components: Table SQL / API
> Environment: Flink version 1.11.1
>Reporter: Moses
>Priority: Major
>
> The SQL statement is designed as below:
> {code:sql}
> CREATE TABLE user_behavior (
> user_id BIGINT,
> item_id BIGINT,
> category_id BIGINT,
> behavior STRING default 'soo', Here I wanna use default value
> ) WITH (
> 'connector' = 'kafka',
> 'topic' = 'user_behavior',
> 'scan.startup.mode' = 'earliest-offset',
> 'properties.bootstrap.servers' = 'kafka:9094'
> 'format' = 'json'
> );
> {code}
> Throws Exception:
> {code:java}
> [ERROR] Could not execute SQL statement. Reason:
> org.apache.flink.sql.parser.impl.ParseException: Encountered "default"
> at line 5, column 21.
> Was expecting one of:
> "CONSTRAINT" ...
> {code}
> It's more friendly for us to support that allowing users to set default
> value during table schema definition. That will reduce UDXF usage and
> simplify data clean process to some extent.
> There are two points need to be solved:
> * Support SQL grammar.
> * Set field to specified value while field value is missing.
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] lindong28 commented on pull request #15840: [release-1.12][FLINK-20114][connector/kafka,common] Fix a few KafkaSource-related bugs
lindong28 commented on pull request #15840: URL: https://github.com/apache/flink/pull/15840#issuecomment-833204973 Thank you @tweise for creating the PR! LGTM. I have verified that this PR only contains changes from https://github.com/apache/flink/pull/15161. And this PR contains all bug fixes that are in https://github.com/apache/flink/pull/15161 but not currently in the release-1.12 branch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15683: [FLINK-22362]Don't appear target address when taskmanager connect to jobmanager error
flinkbot edited a comment on pull request #15683: URL: https://github.com/apache/flink/pull/15683#issuecomment-822960823 ## CI report: * c31947044463827b662c8239d630b52fdad3eca1 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16840) * b4a559f1f2fd1aea9d614fe3e13c4993651d0fc8 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17606) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15712: [FLINK-22400][hive connect]fix NPE problem when convert flink object for Map
flinkbot edited a comment on pull request #15712: URL: https://github.com/apache/flink/pull/15712#issuecomment-824189664 ## CI report: * f2fe08ada02c5c9e20ed397163d3ab7a34594994 UNKNOWN * 5fdb0880e9465be2711e8ac4b9df01388fc5aa2f Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17532) * e90715e4ff43abca28c7da6d8cc719abb857a6f6 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17605) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-19481) Add support for a flink native GCS FileSystem
[ https://issues.apache.org/jira/browse/FLINK-19481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339981#comment-17339981 ] Xintong Song commented on FLINK-19481: -- Thanks for the discussion. I agree with Ben, that Galen's PR is mostly focusing on adding support for RecoverableWriter, and is not depending on the specific file system implementation. Migrating from hadoop to native gcs file system implementation should not require much reworking. However, before deciding dong it, I'd like to understand the benefits of implementing a native gcs file system. Galen's PR does introduce a GSFileSystem, which simply wraps GoogleHadoopFileSystem. It seems to me this already solves most of the problems. - "gs://" scheme can be supported - Users no longer need to deal with the dependencies and FileSystem hierarchies. They should simply add the new flink-gs-fs-hadoop artifact, and ideally everything else needed should be included. Is there any significant benefits that I overlooked, that can only be achieved by a native gcs file system implementation rather than the GSFileSystem in Galen's PR? > Add support for a flink native GCS FileSystem > - > > Key: FLINK-19481 > URL: https://issues.apache.org/jira/browse/FLINK-19481 > Project: Flink > Issue Type: Improvement > Components: Connectors / FileSystem, FileSystems >Affects Versions: 1.12.0 >Reporter: Ben Augarten >Priority: Minor > Labels: auto-deprioritized-major > > Currently, GCS is supported but only by using the hadoop connector[1] > > The objective of this improvement is to add support for checkpointing to > Google Cloud Storage with the Flink File System, > > This would allow the `gs://` scheme to be used for savepointing and > checkpointing. Long term, it would be nice if we could use the GCS FileSystem > as a source and sink in flink jobs as well. > > Long term, I hope that implementing a flink native GCS FileSystem will > simplify usage of GCS because the hadoop FileSystem ends up bringing in many > unshaded dependencies. > > [1] > [https://github.com/GoogleCloudDataproc/hadoop-connectors|https://github.com/GoogleCloudDataproc/hadoop-connectors)] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] xintongsong commented on pull request #15599: [FLINK-11838][flink-gs-fs-hadoop] Create Google Storage file system with recoverable writer support
xintongsong commented on pull request #15599: URL: https://github.com/apache/flink/pull/15599#issuecomment-833201079 @rmetzger, thanks for the pointer :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (FLINK-22512) Can't call current_timestamp with hive dialect for hive-3.1
[ https://issues.apache.org/jira/browse/FLINK-22512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li resolved FLINK-22512. Resolution: Fixed Fixed in master: c688bf3c83e72155ccf5d04fe397b7c0a1274fd1 Fixed in release-1.13: 2f71141639826cc29006ae0e114a6219bdb4f0fa > Can't call current_timestamp with hive dialect for hive-3.1 > --- > > Key: FLINK-22512 > URL: https://issues.apache.org/jira/browse/FLINK-22512 > Project: Flink > Issue Type: Bug > Components: Connectors / Hive >Reporter: Rui Li >Assignee: 罗宇侠 >Priority: Major > Labels: pull-request-available > Fix For: 1.14.0, 1.13.1 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #15840: [release-1.12][FLINK-20114][connector/kafka,common] Fix a few KafkaSource-related bugs
flinkbot edited a comment on pull request #15840: URL: https://github.com/apache/flink/pull/15840#issuecomment-832879704 ## CI report: * 61e736fe55c8c34c46bd32ab073fc0f9e660ccaf Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17599) Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17580) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15712: [FLINK-22400][hive connect]fix NPE problem when convert flink object for Map
flinkbot edited a comment on pull request #15712: URL: https://github.com/apache/flink/pull/15712#issuecomment-824189664 ## CI report: * f2fe08ada02c5c9e20ed397163d3ab7a34594994 UNKNOWN * 5fdb0880e9465be2711e8ac4b9df01388fc5aa2f Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17532) * e90715e4ff43abca28c7da6d8cc719abb857a6f6 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15683: [FLINK-22362]Don't appear target address when taskmanager connect to jobmanager error
flinkbot edited a comment on pull request #15683: URL: https://github.com/apache/flink/pull/15683#issuecomment-822960823 ## CI report: * c31947044463827b662c8239d630b52fdad3eca1 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16840) * b4a559f1f2fd1aea9d614fe3e13c4993651d0fc8 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-22541) add json format filter params
[
https://issues.apache.org/jira/browse/FLINK-22541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339977#comment-17339977
]
Rui Li commented on FLINK-22541:
Hmm I'm not very familiar with JSON format. But this seems to me more of a
filter pushdown feature. If user specifies a WHERE clause and kafka table
source supports filter pushdown, will that satisfy the requirements here?
> add json format filter params
> --
>
> Key: FLINK-22541
> URL: https://issues.apache.org/jira/browse/FLINK-22541
> Project: Flink
> Issue Type: Improvement
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile), Table
> SQL / Ecosystem
>Affects Versions: 1.11.0, 1.12.0
>Reporter: sandy du
>Priority: Minor
>
> In my case,one kafka topic store multiple table data，for example：
>
> \{"id":"121","source":"users","content":{"name":"test01","age":20,"addr":"addr1"}}
>
> \{"id":"122","source":"users","content":{"name":"test02","age":23,"addr":"addr2"}}
>
> \{"id":"124","source":"users","content":{"name":"test03","age":34,"addr":"addr3"}}
>
> \{"id":"124","source":"order","content":{"orderId":"11","price":34,"addr":"addr1231"}}
>
> \{"id":"125","source":"order","content":{"orderId":"12","price":34,"addr":"addr1232"}}
>
> \{"id":"126","source":"order","content":{"orderId":"13","price":34,"addr":"addr1233"}}
>
> I just want to consume data from talbe order，flink sql ddl like this：
> CREATE TABLE order (
> orderId STRING,
> age INT,
> addr STRING
> )
> with (
> 'connector'='kafka',
> 'topic'='kafkatopic',
> 'properties.bootstrap.servers'='localhost:9092',
> 'properties.group.id'='testGroup',
> 'scan.startup.mode'='earliest-offset',
> 'format'='json',
> 'path-fliter'='$[?(@.source=="order")]',
> 'path-data'='$.content'
> );
>
> path-fliter and path-data can use JsonPath
> ([https://github.com/json-path/JsonPath])
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (FLINK-22528) Document latency tracking metrics for state accesses
[ https://issues.apache.org/jira/browse/FLINK-22528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yun Tang reassigned FLINK-22528: Assignee: Yun Tang > Document latency tracking metrics for state accesses > > > Key: FLINK-22528 > URL: https://issues.apache.org/jira/browse/FLINK-22528 > Project: Flink > Issue Type: Improvement > Components: Documentation, Runtime / State Backends >Affects Versions: 1.13.0 >Reporter: Till Rohrmann >Assignee: Yun Tang >Priority: Major > Fix For: 1.14.0 > > > With FLINK-21736 we introduced latency tracking metrics for state accesses. > We should also document how the user can use them and what they say and what > their limitations are. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-22561) Restore checkpoint from another cluster
[
https://issues.apache.org/jira/browse/FLINK-22561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339975#comment-17339975
]
Yun Tang commented on FLINK-22561:
--
[~haoyan.zhang], could you use
[Checkpoints#loadCheckpointMetadata|https://github.com/apache/flink/blob/c688bf3c83e72155ccf5d04fe397b7c0a1274fd1/flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/Checkpoints.java#L99]
to load your {{_metadata}} to see whether the shared path have been stored as
absolute path. I believe that they should store as absolute path with hdfs
schema, and if so, the problem might be your local hdfs client cannot download
data from other cluster instead of looking into Flink.
BTW, if using incremental checkpoint, please be cautious to restore from
another cluster as previous checkpoints might contain files still useful in
newer checkpoints.
> Restore checkpoint from another cluster
> ---
>
> Key: FLINK-22561
> URL: https://issues.apache.org/jira/browse/FLINK-22561
> Project: Flink
> Issue Type: Improvement
> Environment: flink-1.12.1
> hadoop 3.0.0-cdh6.3.2
>Reporter: zhang haoyan
>Priority: Major
>
> Hi, we have two hadoop clusters and run some flink streaming job on it
> When we migrate some jobs from one cluster to another, we start the job just
> like:
> flink - s hdfs://clusterA/xx/xxx/chk-35 xxx.jar, but we will get
> an error like 'java.io.FileNotFoundException: File does not exist:
> /xxx/flink-checkpoints/cb46fc9afab25ced935e751dab49bee2/shared/329f6b79-292a-4d5d-b47b-a1cb63d02fe5',
>
> it seems can't read shared path from another cluster
> We don't want copy the share path manually every time and switch jobs between
> this two cluster will be very often.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #15842: [FLINK-22573][datastream] Fix AsyncIO calls timeout on completed element. [1.12]
flinkbot edited a comment on pull request #15842: URL: https://github.com/apache/flink/pull/15842#issuecomment-832978484 ## CI report: * 42aedbdeb13fb478cc2aa566de0eb0bf043336cf Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17597) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] hehuiyuan commented on pull request #15712: [FLINK-22400][hive connect]fix NPE problem when convert flink object for Map
hehuiyuan commented on pull request #15712: URL: https://github.com/apache/flink/pull/15712#issuecomment-833192628 @flinkbot re-run -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-22580) web 404
[ https://issues.apache.org/jira/browse/FLINK-22580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dong updated FLINK-22580: - Component/s: Documentation Affects Version/s: 1.13.0 Summary: web 404 (was: web404) > web 404 > > > Key: FLINK-22580 > URL: https://issues.apache.org/jira/browse/FLINK-22580 > Project: Flink > Issue Type: Improvement > Components: Documentation >Affects Versions: 1.13.0 >Reporter: dong >Priority: Major > Attachments: 微信截图_20210506110237.png > > > [https://ci.apache.org/projects/flink/flink-docs-release-1.13/try-flink/index.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] lirui-apache closed pull request #15819: [FLINK-22512][Hive] Fix issue that can't call current_timestamp with hive dialect for hive-3.1
lirui-apache closed pull request #15819: URL: https://github.com/apache/flink/pull/15819 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] gentlewangyu commented on a change in pull request #15683: [FLINK-22362]Don't appear target address when taskmanager connect to jobmanager error
gentlewangyu commented on a change in pull request #15683:
URL: https://github.com/apache/flink/pull/15683#discussion_r627045298
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/net/ConnectionUtils.java
##
@@ -352,7 +352,7 @@ private static boolean tryToConnect(
return true;
} catch (Exception ex) {
String message =
-"Failed to connect from address '" + fromAddress + "': " +
ex.getMessage();
+"Failed to connect from address '" + fromAddress + "' to
address '" + toSocket + "': " + ex.getMessage();
Review comment:
@pnowojski Thanks for your reminder, I have corrected it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (FLINK-22580) web404
dong created FLINK-22580: Summary: web404 Key: FLINK-22580 URL: https://issues.apache.org/jira/browse/FLINK-22580 Project: Flink Issue Type: Improvement Reporter: dong Attachments: 微信截图_20210506110237.png [https://ci.apache.org/projects/flink/flink-docs-release-1.13/try-flink/index.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] wuchong closed pull request #15754: [hotfix][hive / test] Add an ITCase that checks partition-time commit pretty well
wuchong closed pull request #15754: URL: https://github.com/apache/flink/pull/15754 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (FLINK-22579) The description content of Kubernetes config Incomplete display
dong created FLINK-22579: Summary: The description content of Kubernetes config Incomplete display Key: FLINK-22579 URL: https://issues.apache.org/jira/browse/FLINK-22579 Project: Flink Issue Type: Improvement Components: Deployment / Kubernetes Affects Versions: 1.13.0 Reporter: dong Attachments: 94680B45-7F34-4ad4-AE21-8A5ED486FAF9.png https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/deployment/config/#kubernetes The description content of Kubernetes config Incomplete display -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-22541) add json format filter params
[
https://issues.apache.org/jira/browse/FLINK-22541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339972#comment-17339972
]
sandy du commented on FLINK-22541:
--
Params path-fliter and path-data are optional, they don't affect normal
logic。The view paln is flawed because the map type does not support nested.
> add json format filter params
> --
>
> Key: FLINK-22541
> URL: https://issues.apache.org/jira/browse/FLINK-22541
> Project: Flink
> Issue Type: Improvement
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile), Table
> SQL / Ecosystem
>Affects Versions: 1.11.0, 1.12.0
>Reporter: sandy du
>Priority: Minor
>
> In my case,one kafka topic store multiple table data，for example：
>
> \{"id":"121","source":"users","content":{"name":"test01","age":20,"addr":"addr1"}}
>
> \{"id":"122","source":"users","content":{"name":"test02","age":23,"addr":"addr2"}}
>
> \{"id":"124","source":"users","content":{"name":"test03","age":34,"addr":"addr3"}}
>
> \{"id":"124","source":"order","content":{"orderId":"11","price":34,"addr":"addr1231"}}
>
> \{"id":"125","source":"order","content":{"orderId":"12","price":34,"addr":"addr1232"}}
>
> \{"id":"126","source":"order","content":{"orderId":"13","price":34,"addr":"addr1233"}}
>
> I just want to consume data from talbe order，flink sql ddl like this：
> CREATE TABLE order (
> orderId STRING,
> age INT,
> addr STRING
> )
> with (
> 'connector'='kafka',
> 'topic'='kafkatopic',
> 'properties.bootstrap.servers'='localhost:9092',
> 'properties.group.id'='testGroup',
> 'scan.startup.mode'='earliest-offset',
> 'format'='json',
> 'path-fliter'='$[?(@.source=="order")]',
> 'path-data'='$.content'
> );
>
> path-fliter and path-data can use JsonPath
> ([https://github.com/json-path/JsonPath])
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (FLINK-22233) Spelling error of word "constant" in source code
[ https://issues.apache.org/jira/browse/FLINK-22233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jark Wu closed FLINK-22233. --- Fix Version/s: 1.13.1 1.14.0 Resolution: Fixed Fixed in - master: ba010e33a1e63d1e89bce76a230bd0985f53844c - release-1.13: 0dc9ee44ab9f3d7b246799a95c0edfa6d051dca5 > Spelling error of word "constant" in source code > > > Key: FLINK-22233 > URL: https://issues.apache.org/jira/browse/FLINK-22233 > Project: Flink > Issue Type: Improvement > Components: Table SQL / API >Affects Versions: 1.12.0 >Reporter: 王高超 >Assignee: 王高超 >Priority: Minor > Labels: pull-request-available, starter > Fix For: 1.14.0, 1.13.1 > > > When I write the data to MySQL and forget to add the primary key, I report > such an error： > ' org.apache.flink.table.api.ValidationException: Flink doesn't support > ENFORCED mode for PRIMARY KEY constaint. ' > As mentioned above, The last word 'constant' is misspelled . And the correct > spelling is 'constraint'. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] wuchong merged pull request #15656: [FLINK-22233][Table SQL / API]Modified the spelling error of word "constant" in source code
wuchong merged pull request #15656: URL: https://github.com/apache/flink/pull/15656 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-22541) add json format filter params
[
https://issues.apache.org/jira/browse/FLINK-22541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339969#comment-17339969
]
Jark Wu commented on FLINK-22541:
-
I'm concerning this will make the json format too complex.
Personally, I think this can be addressed by following a {{WHERE}} condition,
users can also define a view on the table for convenient.
> add json format filter params
> --
>
> Key: FLINK-22541
> URL: https://issues.apache.org/jira/browse/FLINK-22541
> Project: Flink
> Issue Type: Improvement
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile), Table
> SQL / Ecosystem
>Affects Versions: 1.11.0, 1.12.0
>Reporter: sandy du
>Priority: Minor
>
> In my case,one kafka topic store multiple table data，for example：
>
> \{"id":"121","source":"users","content":{"name":"test01","age":20,"addr":"addr1"}}
>
> \{"id":"122","source":"users","content":{"name":"test02","age":23,"addr":"addr2"}}
>
> \{"id":"124","source":"users","content":{"name":"test03","age":34,"addr":"addr3"}}
>
> \{"id":"124","source":"order","content":{"orderId":"11","price":34,"addr":"addr1231"}}
>
> \{"id":"125","source":"order","content":{"orderId":"12","price":34,"addr":"addr1232"}}
>
> \{"id":"126","source":"order","content":{"orderId":"13","price":34,"addr":"addr1233"}}
>
> I just want to consume data from talbe order，flink sql ddl like this：
> CREATE TABLE order (
> orderId STRING,
> age INT,
> addr STRING
> )
> with (
> 'connector'='kafka',
> 'topic'='kafkatopic',
> 'properties.bootstrap.servers'='localhost:9092',
> 'properties.group.id'='testGroup',
> 'scan.startup.mode'='earliest-offset',
> 'format'='json',
> 'path-fliter'='$[?(@.source=="order")]',
> 'path-data'='$.content'
> );
>
> path-fliter and path-data can use JsonPath
> ([https://github.com/json-path/JsonPath])
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] 95chenjz commented on pull request #15767: [FLINK-22393][docs-zh] Translate the page of "Execution Mode (Batch/Streaming)" into Chinese
95chenjz commented on pull request #15767: URL: https://github.com/apache/flink/pull/15767#issuecomment-833182918 @PatrickRen Hi Patrick, is everything looking alright now? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-22541) add json format filter params
[
https://issues.apache.org/jira/browse/FLINK-22541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339968#comment-17339968
]
Jark Wu commented on FLINK-22541:
-
cc [~lirui]
> add json format filter params
> --
>
> Key: FLINK-22541
> URL: https://issues.apache.org/jira/browse/FLINK-22541
> Project: Flink
> Issue Type: Improvement
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile), Table
> SQL / Ecosystem
>Affects Versions: 1.11.0, 1.12.0
>Reporter: sandy du
>Priority: Minor
>
> In my case,one kafka topic store multiple table data，for example：
>
> \{"id":"121","source":"users","content":{"name":"test01","age":20,"addr":"addr1"}}
>
> \{"id":"122","source":"users","content":{"name":"test02","age":23,"addr":"addr2"}}
>
> \{"id":"124","source":"users","content":{"name":"test03","age":34,"addr":"addr3"}}
>
> \{"id":"124","source":"order","content":{"orderId":"11","price":34,"addr":"addr1231"}}
>
> \{"id":"125","source":"order","content":{"orderId":"12","price":34,"addr":"addr1232"}}
>
> \{"id":"126","source":"order","content":{"orderId":"13","price":34,"addr":"addr1233"}}
>
> I just want to consume data from talbe order，flink sql ddl like this：
> CREATE TABLE order (
> orderId STRING,
> age INT,
> addr STRING
> )
> with (
> 'connector'='kafka',
> 'topic'='kafkatopic',
> 'properties.bootstrap.servers'='localhost:9092',
> 'properties.group.id'='testGroup',
> 'scan.startup.mode'='earliest-offset',
> 'format'='json',
> 'path-fliter'='$[?(@.source=="order")]',
> 'path-data'='$.content'
> );
>
> path-fliter and path-data can use JsonPath
> ([https://github.com/json-path/JsonPath])
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-22541) add json format filter params
[
https://issues.apache.org/jira/browse/FLINK-22541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu updated FLINK-22541:
Component/s: Table SQL / Ecosystem
> add json format filter params
> --
>
> Key: FLINK-22541
> URL: https://issues.apache.org/jira/browse/FLINK-22541
> Project: Flink
> Issue Type: Improvement
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile), Table
> SQL / Ecosystem
>Affects Versions: 1.11.0, 1.12.0
>Reporter: sandy du
>Priority: Minor
>
> In my case,one kafka topic store multiple table data，for example：
>
> \{"id":"121","source":"users","content":{"name":"test01","age":20,"addr":"addr1"}}
>
> \{"id":"122","source":"users","content":{"name":"test02","age":23,"addr":"addr2"}}
>
> \{"id":"124","source":"users","content":{"name":"test03","age":34,"addr":"addr3"}}
>
> \{"id":"124","source":"order","content":{"orderId":"11","price":34,"addr":"addr1231"}}
>
> \{"id":"125","source":"order","content":{"orderId":"12","price":34,"addr":"addr1232"}}
>
> \{"id":"126","source":"order","content":{"orderId":"13","price":34,"addr":"addr1233"}}
>
> I just want to consume data from talbe order，flink sql ddl like this：
> CREATE TABLE order (
> orderId STRING,
> age INT,
> addr STRING
> )
> with (
> 'connector'='kafka',
> 'topic'='kafkatopic',
> 'properties.bootstrap.servers'='localhost:9092',
> 'properties.group.id'='testGroup',
> 'scan.startup.mode'='earliest-offset',
> 'format'='json',
> 'path-fliter'='$[?(@.source=="order")]',
> 'path-data'='$.content'
> );
>
> path-fliter and path-data can use JsonPath
> ([https://github.com/json-path/JsonPath])
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-22578) deployment configuration page miss content navigate menu
[ https://issues.apache.org/jira/browse/FLINK-22578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu updated FLINK-22578: --- Fix Version/s: 1.13.1 1.14.0 > deployment configuration page miss content navigate menu > > > Key: FLINK-22578 > URL: https://issues.apache.org/jira/browse/FLINK-22578 > Project: Flink > Issue Type: Improvement > Components: Documentation >Affects Versions: 1.13.0 >Reporter: 谢波 >Priority: Major > Fix For: 1.14.0, 1.13.1 > > > [https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/deployment/config/] > The page miss content navigate menu cuase Inconvenient to use. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-22541) add json format filter params
[
https://issues.apache.org/jira/browse/FLINK-22541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339967#comment-17339967
]
sandy du commented on FLINK-22541:
--
Hi [~jark] ，I wanna contribute this issue，can you assess the issue.
> add json format filter params
> --
>
> Key: FLINK-22541
> URL: https://issues.apache.org/jira/browse/FLINK-22541
> Project: Flink
> Issue Type: Improvement
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
>Affects Versions: 1.11.0, 1.12.0
>Reporter: sandy du
>Priority: Minor
>
> In my case,one kafka topic store multiple table data，for example：
>
> \{"id":"121","source":"users","content":{"name":"test01","age":20,"addr":"addr1"}}
>
> \{"id":"122","source":"users","content":{"name":"test02","age":23,"addr":"addr2"}}
>
> \{"id":"124","source":"users","content":{"name":"test03","age":34,"addr":"addr3"}}
>
> \{"id":"124","source":"order","content":{"orderId":"11","price":34,"addr":"addr1231"}}
>
> \{"id":"125","source":"order","content":{"orderId":"12","price":34,"addr":"addr1232"}}
>
> \{"id":"126","source":"order","content":{"orderId":"13","price":34,"addr":"addr1233"}}
>
> I just want to consume data from talbe order，flink sql ddl like this：
> CREATE TABLE order (
> orderId STRING,
> age INT,
> addr STRING
> )
> with (
> 'connector'='kafka',
> 'topic'='kafkatopic',
> 'properties.bootstrap.servers'='localhost:9092',
> 'properties.group.id'='testGroup',
> 'scan.startup.mode'='earliest-offset',
> 'format'='json',
> 'path-fliter'='$[?(@.source=="order")]',
> 'path-data'='$.content'
> );
>
> path-fliter and path-data can use JsonPath
> ([https://github.com/json-path/JsonPath])
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-22562) Translate recent updates to backpressure and checkpointing docs
[ https://issues.apache.org/jira/browse/FLINK-22562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339966#comment-17339966 ] Jianzhang Chen commented on FLINK-22562: Hi [~pnowojski], can I take this task? > Translate recent updates to backpressure and checkpointing docs > --- > > Key: FLINK-22562 > URL: https://issues.apache.org/jira/browse/FLINK-22562 > Project: Flink > Issue Type: Bug > Components: chinese-translation >Affects Versions: 1.14.0, 1.13.1 >Reporter: Piotr Nowojski >Priority: Major > > https://issues.apache.org/jira/browse/FLINK-22253 > https://github.com/apache/flink/pull/15811 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-22578) deployment configuration page miss content navigate menu
谢波 created FLINK-22578: -- Summary: deployment configuration page miss content navigate menu Key: FLINK-22578 URL: https://issues.apache.org/jira/browse/FLINK-22578 Project: Flink Issue Type: Improvement Components: Documentation Affects Versions: 1.13.0 Reporter: 谢波 [https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/deployment/config/] The page miss content navigate menu cuase Inconvenient to use. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-22572) JsonFsStreamSinkITCase.testPart fails running in a timeout
[ https://issues.apache.org/jira/browse/FLINK-22572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jark Wu updated FLINK-22572: Component/s: Table SQL / Ecosystem > JsonFsStreamSinkITCase.testPart fails running in a timeout > -- > > Key: FLINK-22572 > URL: https://issues.apache.org/jira/browse/FLINK-22572 > Project: Flink > Issue Type: Bug > Components: Connectors / FileSystem, Formats (JSON, Avro, Parquet, > ORC, SequenceFile), Table SQL / Ecosystem >Affects Versions: 1.14.0 >Reporter: Matthias >Priority: Major > Labels: test-stability > > [This build|https://dev.azure.com/mapohl/flink/_build/results?buildId=408] > failed (not exclusively) due to a timeout in > [testPart(org.apache.flink.formats.json.JsonFsStreamSinkITCase)|https://dev.azure.com/mapohl/flink/_build/results?buildId=408=logs=dafbab6d-4616-5d7b-ee37-3c54e4828fd7=777327ab-6d4e-582e-3e76-4a9391c57e59=11295]. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #15843: Add missing title in the page
flinkbot edited a comment on pull request #15843: URL: https://github.com/apache/flink/pull/15843#issuecomment-832990496 ## CI report: * e7f0bf915e740fc851fd27ad47ee9ee9b6ea85d9 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17598) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15841: [FLINK-22573][datastream] Fix AsyncIO calls timeout on completed element. [1.13]
flinkbot edited a comment on pull request #15841: URL: https://github.com/apache/flink/pull/15841#issuecomment-832957376 ## CI report: * 573a5554c041c5ebf3aaa279b9dac58136a63066 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17592) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15121: The method $(String) is undefined for the type TableExample
flinkbot edited a comment on pull request #15121: URL: https://github.com/apache/flink/pull/15121#issuecomment-793480240 ## CI report: * cf26ce895a7956d258acc073817f578558e78227 UNKNOWN * 709c4110370b845f42d916a06e56c6026cf2fac8 UNKNOWN * 4ea99332e1997eebca1f3f0a9d9229b8265fe32c UNKNOWN * 9a256ec6b644a0f795376076f4e3ff4e85aaad1e UNKNOWN * b7dca63a2e98f09c8c780a21091b5268d897b220 UNKNOWN * 2b7bb05a4dbbb0883db566589dfe1975fdbb279d UNKNOWN * c9b51568d56b57a905d28b898408166db1f66a15 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17588) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15771: [FLINK-18934] Idle stream does not advance watermark in connected stream
flinkbot edited a comment on pull request #15771: URL: https://github.com/apache/flink/pull/15771#issuecomment-826939649 ## CI report: * f8403b5c81170289456915341eb8568f7fc2fa1c Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17591) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15834: [FLINK-17170][kinesis] Fix deadlock during stop-with-savepoint. [1.12]
flinkbot edited a comment on pull request #15834: URL: https://github.com/apache/flink/pull/15834#issuecomment-832455659 ## CI report: * 3f4a3c265a770b7645d328da448cefe9980f048a Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17589) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15840: [release-1.12][FLINK-20114][connector/kafka,common] Fix a few KafkaSource-related bugs
flinkbot edited a comment on pull request #15840: URL: https://github.com/apache/flink/pull/15840#issuecomment-832879704 ## CI report: * 61e736fe55c8c34c46bd32ab073fc0f9e660ccaf Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17599) Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17580) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15840: [release-1.12][FLINK-20114][connector/kafka,common] Fix a few KafkaSource-related bugs
flinkbot edited a comment on pull request #15840: URL: https://github.com/apache/flink/pull/15840#issuecomment-832879704 ## CI report: * 61e736fe55c8c34c46bd32ab073fc0f9e660ccaf Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17580) Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17599) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15833: [FLINK-17170][kinesis] Fix deadlock during stop-with-savepoint. [1.13]
flinkbot edited a comment on pull request #15833: URL: https://github.com/apache/flink/pull/15833#issuecomment-832455584 ## CI report: * 10851cae323b5d27f27fb4528e5b27f2d3cf644b Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17583) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] tweise commented on pull request #15840: [release-1.12][FLINK-20114][connector/kafka,common] Fix a few KafkaSource-related bugs
tweise commented on pull request #15840: URL: https://github.com/apache/flink/pull/15840#issuecomment-833007726 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] tweise commented on pull request #15840: [release-1.12][FLINK-20114][connector/kafka,common] Fix a few KafkaSource-related bugs
tweise commented on pull request #15840: URL: https://github.com/apache/flink/pull/15840#issuecomment-833007410 Unrelated e2e test failure: `2021-05-05T19:51:17.8939009Z May 05 19:51:17 [FAIL] 'Running Kerberized YARN application on Docker test (default input)' failed after 6 minutes and 49 seconds! Test exited with exit code 1` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15843: Add missing title in the page
flinkbot edited a comment on pull request #15843: URL: https://github.com/apache/flink/pull/15843#issuecomment-832990496 ## CI report: * e7f0bf915e740fc851fd27ad47ee9ee9b6ea85d9 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17598) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15826: [FLINK-17170][kinesis] Fix deadlock during stop-with-savepoint.
flinkbot edited a comment on pull request #15826: URL: https://github.com/apache/flink/pull/15826#issuecomment-831762997 ## CI report: * 1285619ec11d6170f5bf632e118821e6c5a00cea Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17582) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #15843: Add missing title in the page
flinkbot commented on pull request #15843: URL: https://github.com/apache/flink/pull/15843#issuecomment-832990496 ## CI report: * e7f0bf915e740fc851fd27ad47ee9ee9b6ea85d9 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15842: [FLINK-22573][datastream] Fix AsyncIO calls timeout on completed element. [1.12]
flinkbot edited a comment on pull request #15842: URL: https://github.com/apache/flink/pull/15842#issuecomment-832978484 ## CI report: * 42aedbdeb13fb478cc2aa566de0eb0bf043336cf Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17597) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15826: [FLINK-17170][kinesis] Fix deadlock during stop-with-savepoint.
flinkbot edited a comment on pull request #15826: URL: https://github.com/apache/flink/pull/15826#issuecomment-831762997 ## CI report: * a25e0f732a4c83d56e6415e95eb7f2c960c2f1e0 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17564) Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17581) Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17574) * 1285619ec11d6170f5bf632e118821e6c5a00cea Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17582) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #15842: [FLINK-22573][datastream] Fix AsyncIO calls timeout on completed element. [1.12]
flinkbot commented on pull request #15842: URL: https://github.com/apache/flink/pull/15842#issuecomment-832978484 ## CI report: * 42aedbdeb13fb478cc2aa566de0eb0bf043336cf UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15841: [FLINK-22573][datastream] Fix AsyncIO calls timeout on completed element. [1.13]
flinkbot edited a comment on pull request #15841: URL: https://github.com/apache/flink/pull/15841#issuecomment-832957376 ## CI report: * 573a5554c041c5ebf3aaa279b9dac58136a63066 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17592) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15840: [release-1.12][FLINK-20114][connector/kafka,common] Fix a few KafkaSource-related bugs
flinkbot edited a comment on pull request #15840: URL: https://github.com/apache/flink/pull/15840#issuecomment-832879704 ## CI report: * 61e736fe55c8c34c46bd32ab073fc0f9e660ccaf Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17580) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15771: [FLINK-18934] Idle stream does not advance watermark in connected stream
flinkbot edited a comment on pull request #15771: URL: https://github.com/apache/flink/pull/15771#issuecomment-826939649 ## CI report: * 602e377ea7ab131bc415b102a1e6e6a2c835e990 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17578) * f8403b5c81170289456915341eb8568f7fc2fa1c Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17591) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15121: The method $(String) is undefined for the type TableExample
flinkbot edited a comment on pull request #15121: URL: https://github.com/apache/flink/pull/15121#issuecomment-793480240 ## CI report: * cf26ce895a7956d258acc073817f578558e78227 UNKNOWN * 709c4110370b845f42d916a06e56c6026cf2fac8 UNKNOWN * 4ea99332e1997eebca1f3f0a9d9229b8265fe32c UNKNOWN * 9a256ec6b644a0f795376076f4e3ff4e85aaad1e UNKNOWN * b7dca63a2e98f09c8c780a21091b5268d897b220 UNKNOWN * 2b7bb05a4dbbb0883db566589dfe1975fdbb279d UNKNOWN * 4303e9844bcaf301d79371a008525a8f8b944db1 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17523) * c9b51568d56b57a905d28b898408166db1f66a15 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17588) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #15843: Add missing title in the page
flinkbot commented on pull request #15843: URL: https://github.com/apache/flink/pull/15843#issuecomment-832976852 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit e7f0bf915e740fc851fd27ad47ee9ee9b6ea85d9 (Wed May 05 20:14:06 UTC 2021) **Warnings:** * Documentation files were touched, but no `docs/content.zh/` files: Update Chinese documentation or file Jira ticket. * **Invalid pull request title: No valid Jira ID provided** Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] andrewinci opened a new pull request #15843: Add missing title in the page
andrewinci opened a new pull request #15843: URL: https://github.com/apache/flink/pull/15843 Add title in the page like in `flink/docs/content/docs/dev/table/concepts/legacy.md` ## What is the purpose of the change *(For example: This pull request makes task deployment go through the blob server, rather than through RPC. That way we avoid re-transferring them on each deployment (during recovery).)* ## Brief change log *(for example:)* - *The TaskInfo is stored in the blob store on job creation time as a persistent artifact* - *Deployments RPC transmits only the blob storage reference* - *TaskManagers retrieve the TaskInfo from the blob cache* ## Verifying this change *(Please pick either of the following options)* This change is a trivial rework / code cleanup without any test coverage. *(or)* This change is already covered by existing tests, such as *(please describe tests)*. *(or)* This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (100MB)* - *Extended integration test for recovery after master (JobManager) failure* - *Added test that validates that TaskInfo is transferred only once across recoveries* - *Manually verified the change by running a 4 node cluser with 2 JobManagers and 4 TaskManagers, a stateful streaming program, and killing one JobManager and two TaskManagers during the execution, verifying that recovery happens correctly.* ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / no) - The serializers: (yes / no / don't know) - The runtime per-record code paths (performance sensitive): (yes / no / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / no / don't know) - The S3 file system connector: (yes / no / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15838: [FLINK-21095][ci] Remove legacy slot management profile
flinkbot edited a comment on pull request #15838: URL: https://github.com/apache/flink/pull/15838#issuecomment-832740396 ## CI report: * 72979bff8efeda231ba7d75f004d592178c44051 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17572) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #15841: [FLINK-22573][datastream] Fix AsyncIO calls timeout on completed element. [1.13]
flinkbot commented on pull request #15841: URL: https://github.com/apache/flink/pull/15841#issuecomment-832957376 ## CI report: * 573a5554c041c5ebf3aaa279b9dac58136a63066 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15834: [FLINK-17170][kinesis] Fix deadlock during stop-with-savepoint. [1.12]
flinkbot edited a comment on pull request #15834: URL: https://github.com/apache/flink/pull/15834#issuecomment-832455659 ## CI report: * d8014bc0b666508c4d3f8f06bb63c0a4fa057394 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17563) * 3f4a3c265a770b7645d328da448cefe9980f048a Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17589) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15771: [FLINK-18934] Idle stream does not advance watermark in connected stream
flinkbot edited a comment on pull request #15771: URL: https://github.com/apache/flink/pull/15771#issuecomment-826939649 ## CI report: * 602e377ea7ab131bc415b102a1e6e6a2c835e990 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17578) * f8403b5c81170289456915341eb8568f7fc2fa1c UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15121: The method $(String) is undefined for the type TableExample
flinkbot edited a comment on pull request #15121: URL: https://github.com/apache/flink/pull/15121#issuecomment-793480240 ## CI report: * cf26ce895a7956d258acc073817f578558e78227 UNKNOWN * 709c4110370b845f42d916a06e56c6026cf2fac8 UNKNOWN * 4ea99332e1997eebca1f3f0a9d9229b8265fe32c UNKNOWN * 9a256ec6b644a0f795376076f4e3ff4e85aaad1e UNKNOWN * b7dca63a2e98f09c8c780a21091b5268d897b220 UNKNOWN * 2b7bb05a4dbbb0883db566589dfe1975fdbb279d UNKNOWN * 4303e9844bcaf301d79371a008525a8f8b944db1 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17523) * c9b51568d56b57a905d28b898408166db1f66a15 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #15842: [FLINK-22573][datastream] Fix AsyncIO calls timeout on completed element. [1.12]
flinkbot commented on pull request #15842: URL: https://github.com/apache/flink/pull/15842#issuecomment-832955405 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 42aedbdeb13fb478cc2aa566de0eb0bf043336cf (Wed May 05 19:36:23 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] AHeise opened a new pull request #15842: [FLINK-22573][datastream] Fix AsyncIO calls timeout on completed element. [1.12]
AHeise opened a new pull request #15842: URL: https://github.com/apache/flink/pull/15842 Unchanged backport of #15836 (well squashed into 1 commit). ## What is the purpose of the change As long as the mailbox is blocked, timers are not cancelled, such that a completed element might still get a timeout. ## Brief change log The fix is to check the completed flag when the timer triggers. ## Verifying this change Added `AsyncWaitOperatorTest#testTimeoutAfterComplete`. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / **no**) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**) - The serializers: (yes / **no** / don't know) - The runtime per-record code paths (performance sensitive): (**yes** / no / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / **no** / don't know) - The S3 file system connector: (yes / **no** / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / **no**) - If yes, how is the feature documented? (**not applicable** / docs / JavaDocs / not documented) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #15841: [FLINK-22573][datastream] Fix AsyncIO calls timeout on completed element. [1.13]
flinkbot commented on pull request #15841: URL: https://github.com/apache/flink/pull/15841#issuecomment-832949119 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 573a5554c041c5ebf3aaa279b9dac58136a63066 (Wed May 05 19:24:53 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] AHeise opened a new pull request #15841: [FLINK-22573][datastream] Fix AsyncIO calls timeout on completed element. [1.13]
AHeise opened a new pull request #15841: URL: https://github.com/apache/flink/pull/15841 Unchanged backport of #15836 (well squashed into 1 commit). ## What is the purpose of the change As long as the mailbox is blocked, timers are not cancelled, such that a completed element might still get a timeout. ## Brief change log The fix is to check the completed flag when the timer triggers. ## Verifying this change Added `AsyncWaitOperatorTest#testTimeoutAfterComplete`. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / **no**) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**) - The serializers: (yes / **no** / don't know) - The runtime per-record code paths (performance sensitive): (**yes** / no / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / **no** / don't know) - The S3 file system connector: (yes / **no** / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / **no**) - If yes, how is the feature documented? (**not applicable** / docs / JavaDocs / not documented) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15834: [FLINK-17170][kinesis] Fix deadlock during stop-with-savepoint. [1.12]
flinkbot edited a comment on pull request #15834: URL: https://github.com/apache/flink/pull/15834#issuecomment-832455659 ## CI report: * d8014bc0b666508c4d3f8f06bb63c0a4fa057394 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17563) * 3f4a3c265a770b7645d328da448cefe9980f048a UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-22573) AsyncIO can timeout elements after completion
[ https://issues.apache.org/jira/browse/FLINK-22573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339861#comment-17339861 ] Arvid Heise commented on FLINK-22573: - Merged the fix into master as 56e292b92ec0dce383becf5150a2dd92f3fcae7a..0790768167f1a43d90df89ca4ceb2248841d91d3. > AsyncIO can timeout elements after completion > - > > Key: FLINK-22573 > URL: https://issues.apache.org/jira/browse/FLINK-22573 > Project: Flink > Issue Type: Bug > Components: API / DataStream >Affects Versions: 1.11.3, 1.13.0, 1.14.0, 1.12.3 >Reporter: Arvid Heise >Assignee: Arvid Heise >Priority: Major > > AsyncIO emits completed elements over the mailbox at which any timer is also > canceled. However, if the mailbox cannot process (heavy backpressure), it may > be that the timer still triggers on a completed element. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] AHeise merged pull request #15836: [FLINK-22573][datastream] Fix AsyncIO calls timeout on completed element.
AHeise merged pull request #15836: URL: https://github.com/apache/flink/pull/15836 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] AHeise commented on pull request #15836: [FLINK-22573][datastream] Fix AsyncIO calls timeout on completed element.
AHeise commented on pull request #15836: URL: https://github.com/apache/flink/pull/15836#issuecomment-832943232 > LGTM - assuming test failure is unrelated. > > One last question, have you tried looping this test locally for a couple of hundred/thousands times, just to be sure? I looped it 1k times successfully. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Closed] (FLINK-22442) Using scala api to change the TimeCharacteristic of the PatternStream is invalid
[
https://issues.apache.org/jira/browse/FLINK-22442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dawid Wysakowicz closed FLINK-22442.
Fix Version/s: 1.12.4
1.13.1
1.14.0
Resolution: Fixed
Fixed in:
* master
** cb0d8eb85048cfb08be54e75f2f90b1d590b7ae0
* 1.13.1
** 32c64bb4d30499f602079cd34c73371106fe09c3
* 1.12.4
** c9b51568d56b57a905d28b898408166db1f66a15
> Using scala api to change the TimeCharacteristic of the PatternStream is
> invalid
>
>
> Key: FLINK-22442
> URL: https://issues.apache.org/jira/browse/FLINK-22442
> Project: Flink
> Issue Type: Bug
> Components: Library / CEP
>Affects Versions: 1.12.0, 1.12.1, 1.12.2
>Reporter: Cedric Chen
>Assignee: Cedric Chen
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.14.0, 1.13.1, 1.12.4
>
>
> Using scala api to change the TimeCharacteristic of the PatternStream is
> invalid
> you can only use the eventTime for PatternStream
> the bug is ：
> in the code in org.apache.flink.cep.scala.PatternStream
> when we called function like inProcessingTime()
> the real JPatternStream in the object not be updated
> {code:java}
> // org.apache.flink.cep.scala.PatternStream
> class PatternStream[T](jPatternStream: JPatternStream[T]) {
> private[flink] def wrappedPatternStream = jPatternStream
> ..
> def sideOutputLateData(lateDataOutputTag: OutputTag[T]): PatternStream[T] = {
>jPatternStream.sideOutputLateData(lateDataOutputTag)
>this
> }
> def inProcessingTime(): PatternStream[T] = {
> jPatternStream.inProcessingTime()
> this
> }
> def inEventTime(): PatternStream[T] = {
> jPatternStream.inEventTime()
> this
> }
> ｝
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] dawidwys closed pull request #15742: [FLINK-22442][CEP] Fix Using scala api to change the TimeCharacteristic o…
dawidwys closed pull request #15742: URL: https://github.com/apache/flink/pull/15742 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] dawidwys commented on pull request #15742: [FLINK-22442][CEP] Fix Using scala api to change the TimeCharacteristic o…
dawidwys commented on pull request #15742: URL: https://github.com/apache/flink/pull/15742#issuecomment-832938976 Thanks for the fix and sorry for the delay. I am good with the current state. I'll merge it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-22488) KafkaSourceLegacyITCase.testOneToOneSources failed due to "OperatorEvent from an OperatorCoordinator to a task was lost"
[ https://issues.apache.org/jira/browse/FLINK-22488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339857#comment-17339857 ] Dawid Wysakowicz commented on FLINK-22488: -- Merged logging additional info in: * master ** 567c900f76e8678d0d29bb600e4c28b841499ba9 * 1.13 ** 2b71b12b1f9f32899509298e8fa3795851e1ea98 * 1.12 ** 6c2501fc58006fababca18ec9be9e691e7eecb7c > KafkaSourceLegacyITCase.testOneToOneSources failed due to "OperatorEvent from > an OperatorCoordinator to a task was lost" > > > Key: FLINK-22488 > URL: https://issues.apache.org/jira/browse/FLINK-22488 > Project: Flink > Issue Type: Improvement >Reporter: Dong Lin >Priority: Minor > Labels: pull-request-available > > According to [1], the test KafkaSourceLegacyITCase.testOneToOneSources failed > because it runs a streaming job (which uses KafkaSource) with > restartAttempts=1. In addition to the failover explicitly triggered by the > FailingIdentityMapper, the job additionally failed due to > "org.apache.flink.util.FlinkException: An OperatorEvent from an > OperatorCoordinator to a task was lost. Triggering task failover to ensure > consistency", which is unexpected by the test. > Note that SubtaskGatewayImpl was updated by [2] on 4/14 which triggers task > failover if any OperatorEvent was lost. This could explain why those Kafka > tests start to fail due to the exception described above. > In order to make this test stable, let's try to understand why there is such > a high chance of loosing OperatorEvent in the Azure test pipeline. And if we > could not avoid loosing OperatorEvent in the test pipeline, we probably need > to update the test to allow the pipeline being restarted arbitrary times (and > still be able to stop the test on the happy path). > [1] > https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=17212=logs=c5f0071e-1851-543e-9a45-9ac140befc32=1fb1a56f-e8b5-5a82-00a0-a2db7757b4f5=6960 > [2] https://github.com/apache/flink/pull/15605 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #15833: [FLINK-17170][kinesis] Fix deadlock during stop-with-savepoint. [1.13]
flinkbot edited a comment on pull request #15833: URL: https://github.com/apache/flink/pull/15833#issuecomment-832455584 ## CI report: * 2cacafb7857f25e99a42cba0f9136f3794e0f5a4 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17575) Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17562) * 10851cae323b5d27f27fb4528e5b27f2d3cf644b Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17583) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15826: [FLINK-17170][kinesis] Fix deadlock during stop-with-savepoint.
flinkbot edited a comment on pull request #15826: URL: https://github.com/apache/flink/pull/15826#issuecomment-831762997 ## CI report: * a25e0f732a4c83d56e6415e95eb7f2c960c2f1e0 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17564) Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17581) Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17574) * 1285619ec11d6170f5bf632e118821e6c5a00cea Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17582) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15833: [FLINK-17170][kinesis] Fix deadlock during stop-with-savepoint. [1.13]
flinkbot edited a comment on pull request #15833: URL: https://github.com/apache/flink/pull/15833#issuecomment-832455584 ## CI report: * 2cacafb7857f25e99a42cba0f9136f3794e0f5a4 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17575) Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17562) * 10851cae323b5d27f27fb4528e5b27f2d3cf644b UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15826: [FLINK-17170][kinesis] Fix deadlock during stop-with-savepoint.
flinkbot edited a comment on pull request #15826: URL: https://github.com/apache/flink/pull/15826#issuecomment-831762997 ## CI report: * a25e0f732a4c83d56e6415e95eb7f2c960c2f1e0 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17564) Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17581) Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17574) * 1285619ec11d6170f5bf632e118821e6c5a00cea UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15833: [FLINK-17170][kinesis] Fix deadlock during stop-with-savepoint. [1.13]
flinkbot edited a comment on pull request #15833: URL: https://github.com/apache/flink/pull/15833#issuecomment-832455584 ## CI report: * 2cacafb7857f25e99a42cba0f9136f3794e0f5a4 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17575) Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17562) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15840: [release-1.12][FLINK-20114][connector/kafka,common] Fix a few KafkaSource-related bugs
flinkbot edited a comment on pull request #15840: URL: https://github.com/apache/flink/pull/15840#issuecomment-832879704 ## CI report: * 61e736fe55c8c34c46bd32ab073fc0f9e660ccaf Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17580) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15826: [FLINK-17170][kinesis] Fix deadlock during stop-with-savepoint.
flinkbot edited a comment on pull request #15826: URL: https://github.com/apache/flink/pull/15826#issuecomment-831762997 ## CI report: * a25e0f732a4c83d56e6415e95eb7f2c960c2f1e0 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17574) Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17564) Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17581) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-21439) Adaptive Scheduler: Add support for exception history
[
https://issues.apache.org/jira/browse/FLINK-21439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339813#comment-17339813
]
Robert Metzger commented on FLINK-21439:
What's the status of this effort?
> Adaptive Scheduler: Add support for exception history
> -
>
> Key: FLINK-21439
> URL: https://issues.apache.org/jira/browse/FLINK-21439
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.13.0
>Reporter: Matthias
>Assignee: John Phelan
>Priority: Major
> Labels: reactive
> Time Spent: 3h
> Remaining Estimate: 0h
>
> {{SchedulerNG.requestJob}} returns an {{ExecutionGraphInfo}} that was
> introduced in FLINK-21188. This {{ExecutionGraphInfo}} holds the information
> about the {{ArchivedExecutionGraph}} and exception history information.
> Currently, it's a list of {{ErrorInfos}}. This might change due to ongoing
> work in FLINK-21190 where we might introduced a wrapper class with more
> information on the failure.
> The goal of this ticket is to implement the exception history for the
> {{AdaptiveScheduler}}, i.e. collecting the exceptions that caused restarts.
> This collection of failures should be forwarded through
> {{SchedulerNG.requestJob}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] AHeise commented on pull request #15826: [FLINK-17170][kinesis] Fix deadlock during stop-with-savepoint.
AHeise commented on pull request #15826: URL: https://github.com/apache/flink/pull/15826#issuecomment-832888061 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15836: [FLINK-22573][datastream] Fix AsyncIO calls timeout on completed element.
flinkbot edited a comment on pull request #15836: URL: https://github.com/apache/flink/pull/15836#issuecomment-832638447 ## CI report: * 178b0364d2cf843a15d481ba533edf6cffa981f9 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17579) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #15840: [release-1.12][FLINK-20114][connector/kafka,common] Fix a few KafkaSource-related bugs
flinkbot commented on pull request #15840: URL: https://github.com/apache/flink/pull/15840#issuecomment-832879704 ## CI report: * 61e736fe55c8c34c46bd32ab073fc0f9e660ccaf UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-22516) ResourceManager cannot establish leadership
[
https://issues.apache.org/jira/browse/FLINK-22516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339796#comment-17339796
]
Ricky Burnett commented on FLINK-22516:
---
Thanks for replying [~mapohl]. I am aware this this is outside of the normal
support. I wanted to bring it up just in case this is an unknown/unfixed
issue. This seems like a critical flaw in leadership election. I couldn't
find any tickets related to this issue, so I don't think it's fixed.
> ResourceManager cannot establish leadership
> ---
>
> Key: FLINK-22516
> URL: https://issues.apache.org/jira/browse/FLINK-22516
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
>Affects Versions: 1.9.3
> Environment: 1.9.3 Flink version, on kubernetes.
>Reporter: Ricky Burnett
>Priority: Major
> Attachments: jobmanager_leadership.log
>
>
> We are running Flink clusters with 2 Jobmanagers in HA mode. After a
> Zookeeper restart the two JMs begin leadership election end up in state where
> they are both trying to start their ResourceManager and until one of them
> writes to `leader//resource_manager_lock` and the other Jobmanager's
> JobMaster proceeds to execute `notifyOfNewResourceManagerLeader` which
> restarts the ResourceManager. This in turn writes to
> `leader//resource_manager_lock` which triggers the first JobMaster to
> restart it's ResourceManager. We can see this in the logs from the
> "ResourceManager leader changed to new address" log, that goes back and forth
> between the two JMs and the two IP addresses. This cycle appears to continue
> indefinitely with outside interruption.
> I've attached combined logs from two JMs in our environment that got into
> this state. The logs start with the loss of connection and end with a couple
> of cycles of back and forth. The two relevant hosts are
> "flink-jm-828d4aa2-d4d4-457b-995d-feb56d08c1fb-784cdb9c57-tsxb7" and
> "flink-jm-828d4aa2-d4d4-457b-995d-feb56d08c1fb-784cdb9c57-mpf9x".
> *-tsxb7 appears to be the last host that was granted leadership.
> {code:java}
> {"thread":"Curator-Framework-0-EventThread","level":"INFO","loggerName":"org.apache.flink.runtime.jobmaster.JobManagerRunner","message":"JobManager
> runner for job tenant: ssademo, pipeline:
> 828d4aa2-d4d4-457b-995d-feb56d08c1fb, name: integration-test-detection
> (33e12948df69077ab3b33316eacbb5e4) was granted leadership with session id
> 97992805-9c60-40ba-8260-aaf036694cde at
> akka.tcp://flink@100.97.92.73:6123/user/jobmanager_3.","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","instant":{"epochSecond":1617129712,"nanoOfSecond":44700},"contextMap":{},"threadId":152,"threadPriority":5,"source":{"class":"org.apache.flink.runtime.jobmaster.JobManagerRunner","method":"startJobMaster","file":"JobManagerRunner.java","line":313},"service":"streams","time":"2021-03-30T18:41:52.447UTC","hostname":"flink-jm-828d4aa2-d4d4-457b-995d-feb56d08c1fb-784cdb9c57-tsxb7"}
> {code}
> But *-mpf9x continues to try to wrestle control back.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-22516) ResourceManager cannot establish leadership
[
https://issues.apache.org/jira/browse/FLINK-22516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339784#comment-17339784
]
Matthias commented on FLINK-22516:
--
Thanks for reporting this issue. Please keep in mind that the community usually
only supports the two most recent releases.
[~trohrmann] do you know out of your head whether this issue is a known issue
that was already fixed in one of the previous versions or whether it is related
to one of the issues that are currently still open?
> ResourceManager cannot establish leadership
> ---
>
> Key: FLINK-22516
> URL: https://issues.apache.org/jira/browse/FLINK-22516
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
>Affects Versions: 1.9.3
> Environment: 1.9.3 Flink version, on kubernetes.
>Reporter: Ricky Burnett
>Priority: Major
> Attachments: jobmanager_leadership.log
>
>
> We are running Flink clusters with 2 Jobmanagers in HA mode. After a
> Zookeeper restart the two JMs begin leadership election end up in state where
> they are both trying to start their ResourceManager and until one of them
> writes to `leader//resource_manager_lock` and the other Jobmanager's
> JobMaster proceeds to execute `notifyOfNewResourceManagerLeader` which
> restarts the ResourceManager. This in turn writes to
> `leader//resource_manager_lock` which triggers the first JobMaster to
> restart it's ResourceManager. We can see this in the logs from the
> "ResourceManager leader changed to new address" log, that goes back and forth
> between the two JMs and the two IP addresses. This cycle appears to continue
> indefinitely with outside interruption.
> I've attached combined logs from two JMs in our environment that got into
> this state. The logs start with the loss of connection and end with a couple
> of cycles of back and forth. The two relevant hosts are
> "flink-jm-828d4aa2-d4d4-457b-995d-feb56d08c1fb-784cdb9c57-tsxb7" and
> "flink-jm-828d4aa2-d4d4-457b-995d-feb56d08c1fb-784cdb9c57-mpf9x".
> *-tsxb7 appears to be the last host that was granted leadership.
> {code:java}
> {"thread":"Curator-Framework-0-EventThread","level":"INFO","loggerName":"org.apache.flink.runtime.jobmaster.JobManagerRunner","message":"JobManager
> runner for job tenant: ssademo, pipeline:
> 828d4aa2-d4d4-457b-995d-feb56d08c1fb, name: integration-test-detection
> (33e12948df69077ab3b33316eacbb5e4) was granted leadership with session id
> 97992805-9c60-40ba-8260-aaf036694cde at
> akka.tcp://flink@100.97.92.73:6123/user/jobmanager_3.","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","instant":{"epochSecond":1617129712,"nanoOfSecond":44700},"contextMap":{},"threadId":152,"threadPriority":5,"source":{"class":"org.apache.flink.runtime.jobmaster.JobManagerRunner","method":"startJobMaster","file":"JobManagerRunner.java","line":313},"service":"streams","time":"2021-03-30T18:41:52.447UTC","hostname":"flink-jm-828d4aa2-d4d4-457b-995d-feb56d08c1fb-784cdb9c57-tsxb7"}
> {code}
> But *-mpf9x continues to try to wrestle control back.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #15839: [FLINK-22557][build] Run japicmp against 1.12.2
flinkbot edited a comment on pull request #15839: URL: https://github.com/apache/flink/pull/15839#issuecomment-832740531 ## CI report: * 445021d89541ddb782051a45f9041de751fa372e Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17573) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-22516) ResourceManager cannot establish leadership
[
https://issues.apache.org/jira/browse/FLINK-22516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias updated FLINK-22516:
-
Component/s: Runtime / Coordination
> ResourceManager cannot establish leadership
> ---
>
> Key: FLINK-22516
> URL: https://issues.apache.org/jira/browse/FLINK-22516
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
>Affects Versions: 1.9.3
> Environment: 1.9.3 Flink version, on kubernetes.
>Reporter: Ricky Burnett
>Priority: Major
> Attachments: jobmanager_leadership.log
>
>
> We are running Flink clusters with 2 Jobmanagers in HA mode. After a
> Zookeeper restart the two JMs begin leadership election end up in state where
> they are both trying to start their ResourceManager and until one of them
> writes to `leader//resource_manager_lock` and the other Jobmanager's
> JobMaster proceeds to execute `notifyOfNewResourceManagerLeader` which
> restarts the ResourceManager. This in turn writes to
> `leader//resource_manager_lock` which triggers the first JobMaster to
> restart it's ResourceManager. We can see this in the logs from the
> "ResourceManager leader changed to new address" log, that goes back and forth
> between the two JMs and the two IP addresses. This cycle appears to continue
> indefinitely with outside interruption.
> I've attached combined logs from two JMs in our environment that got into
> this state. The logs start with the loss of connection and end with a couple
> of cycles of back and forth. The two relevant hosts are
> "flink-jm-828d4aa2-d4d4-457b-995d-feb56d08c1fb-784cdb9c57-tsxb7" and
> "flink-jm-828d4aa2-d4d4-457b-995d-feb56d08c1fb-784cdb9c57-mpf9x".
> *-tsxb7 appears to be the last host that was granted leadership.
> {code:java}
> {"thread":"Curator-Framework-0-EventThread","level":"INFO","loggerName":"org.apache.flink.runtime.jobmaster.JobManagerRunner","message":"JobManager
> runner for job tenant: ssademo, pipeline:
> 828d4aa2-d4d4-457b-995d-feb56d08c1fb, name: integration-test-detection
> (33e12948df69077ab3b33316eacbb5e4) was granted leadership with session id
> 97992805-9c60-40ba-8260-aaf036694cde at
> akka.tcp://flink@100.97.92.73:6123/user/jobmanager_3.","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger","instant":{"epochSecond":1617129712,"nanoOfSecond":44700},"contextMap":{},"threadId":152,"threadPriority":5,"source":{"class":"org.apache.flink.runtime.jobmaster.JobManagerRunner","method":"startJobMaster","file":"JobManagerRunner.java","line":313},"service":"streams","time":"2021-03-30T18:41:52.447UTC","hostname":"flink-jm-828d4aa2-d4d4-457b-995d-feb56d08c1fb-784cdb9c57-tsxb7"}
> {code}
> But *-mpf9x continues to try to wrestle control back.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot commented on pull request #15840: [release-1.12][FLINK-20114][connector/kafka,common] Fix a few KafkaSource-related bugs
flinkbot commented on pull request #15840: URL: https://github.com/apache/flink/pull/15840#issuecomment-832865638 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 61e736fe55c8c34c46bd32ab073fc0f9e660ccaf (Wed May 05 17:14:16 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-20114).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] tweise commented on pull request #15840: [release-1.12][FLINK-20114][connector/kafka,common] Fix a few KafkaSource-related bugs
tweise commented on pull request #15840: URL: https://github.com/apache/flink/pull/15840#issuecomment-832864931 CC: @lindong28 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] tweise opened a new pull request #15840: [release-1.12][FLINK-20114][connector/kafka,common] Fix a few KafkaSource-related bugs
tweise opened a new pull request #15840: URL: https://github.com/apache/flink/pull/15840 Backport of https://github.com/apache/flink/pull/15161 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15827: [FLINK-22555][build][python] Exclude leftover jboss files
flinkbot edited a comment on pull request #15827: URL: https://github.com/apache/flink/pull/15827#issuecomment-831839655 ## CI report: * 7a82a2746db1f7e073849b4885dcff2c98d13d4b Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=17571) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-22521) flink stop 00000000000000000000000000000000 error
[
https://issues.apache.org/jira/browse/FLINK-22521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias updated FLINK-22521:
-
Component/s: Runtime / Coordination
> flink stop error
> --
>
> Key: FLINK-22521
> URL: https://issues.apache.org/jira/browse/FLINK-22521
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
>Affects Versions: 1.12.1
>Reporter: 陈孝忠
>Priority: Major
>
> Flink K8S applicaton mode, using K8S HA. After running for two days, the
> program could not be stopped. The running log is as follows:
> {code:java}
> /opt/flink/bin/flink stop --target
> kubernetes-application -Dkubernetes.namespace=middle-flink
> -Dkubernetes.config.file=/opt/flink/conf/kubeconfig
> -Dkubernetes.cluster-id=xxx
> Suspending job "" with a savepoint.
> 2021-04-29 03:43:56,397 INFO
> org.apache.flink.kubernetes.KubernetesClusterDescriptor [] - Retrieve flink
> cluster xxx successfully, JobManager Web Interface:
> [http:/xxx:16206|http://stock-calc-task-03-rest.middle-flink:16206/]
> rs=1
>
> The program finished with the following exception:
> org.apache.flink.util.FlinkException: Could not stop with a savepoint job
> "".
> at
> org.apache.flink.client.cli.CliFrontend.lambda$stop$5(CliFrontend.java:585)
> at
> org.apache.flink.client.cli.CliFrontend.runClusterAction(CliFrontend.java:1006)
> at org.apache.flink.client.cli.CliFrontend.stop(CliFrontend.java:573)
> at org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1073)
> at
> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1136)
> at
> org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:28)
> at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1136)
> Caused by: java.util.concurrent.ExecutionException:
> org.apache.flink.runtime.rest.util.RestClientException:
> [org.apache.flink.runtime.rest.NotFoundException: Operation not found under
> key: org.apache.flink.runtime.rest.handl
> er.job.AsynchronousJobOperationKey@38ca1e1b
> at
> org.apache.flink.runtime.rest.handler.async.AbstractAsynchronousOperationHandlers$StatusHandler.handleRequest(AbstractAsynchronousOperationHandlers.java:182)
> at
> org.apache.flink.runtime.rest.handler.job.savepoints.SavepointHandlers$SavepointStatusHandler.handleRequest(SavepointHandlers.java:219)
> at
> org.apache.flink.runtime.rest.handler.AbstractRestHandler.respondToRequest(AbstractRestHandler.java:83)
> at
> org.apache.flink.runtime.rest.handler.AbstractHandler.respondAsLeader(AbstractHandler.java:195)
> at
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.lambda$channelRead0$0(LeaderRetrievalHandler.java:83)
> at java.util.Optional.ifPresent(Optional.java:159)
> at org.apache.flink.util.OptionalConsumer.ifPresent(OptionalConsumer.java:45)
> at
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:80)
> at
> org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:49)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
> at
> org.apache.flink.runtime.rest.handler.router.RouterHandler.routed(RouterHandler.java:115)
> at
> org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:94)
> at
> org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:55)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
> at org.apache.flink.shaded.netty4.io.netty.channel.Abst
>
> ractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
> at
>
[jira] [Updated] (FLINK-22521) flink stop 00000000000000000000000000000000 error
[
https://issues.apache.org/jira/browse/FLINK-22521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias updated FLINK-22521:
-
Description:
Flink K8S applicaton mode, using K8S HA. After running for two days, the
program could not be stopped. The running log is as follows:
{code:java}
/opt/flink/bin/flink stop --target
kubernetes-application -Dkubernetes.namespace=middle-flink
-Dkubernetes.config.file=/opt/flink/conf/kubeconfig -Dkubernetes.cluster-id=xxx
Suspending job "" with a savepoint.
2021-04-29 03:43:56,397 INFO
org.apache.flink.kubernetes.KubernetesClusterDescriptor [] - Retrieve flink
cluster xxx successfully, JobManager Web Interface:
[http:/xxx:16206|http://stock-calc-task-03-rest.middle-flink:16206/]
rs=1
The program finished with the following exception:
org.apache.flink.util.FlinkException: Could not stop with a savepoint job
"".
at org.apache.flink.client.cli.CliFrontend.lambda$stop$5(CliFrontend.java:585)
at
org.apache.flink.client.cli.CliFrontend.runClusterAction(CliFrontend.java:1006)
at org.apache.flink.client.cli.CliFrontend.stop(CliFrontend.java:573)
at org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1073)
at
org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1136)
at
org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:28)
at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1136)
Caused by: java.util.concurrent.ExecutionException:
org.apache.flink.runtime.rest.util.RestClientException:
[org.apache.flink.runtime.rest.NotFoundException: Operation not found under
key: org.apache.flink.runtime.rest.handl
er.job.AsynchronousJobOperationKey@38ca1e1b
at
org.apache.flink.runtime.rest.handler.async.AbstractAsynchronousOperationHandlers$StatusHandler.handleRequest(AbstractAsynchronousOperationHandlers.java:182)
at
org.apache.flink.runtime.rest.handler.job.savepoints.SavepointHandlers$SavepointStatusHandler.handleRequest(SavepointHandlers.java:219)
at
org.apache.flink.runtime.rest.handler.AbstractRestHandler.respondToRequest(AbstractRestHandler.java:83)
at
org.apache.flink.runtime.rest.handler.AbstractHandler.respondAsLeader(AbstractHandler.java:195)
at
org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.lambda$channelRead0$0(LeaderRetrievalHandler.java:83)
at java.util.Optional.ifPresent(Optional.java:159)
at org.apache.flink.util.OptionalConsumer.ifPresent(OptionalConsumer.java:45)
at
org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:80)
at
org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:49)
at
org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at
org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at
org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at
org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at
org.apache.flink.runtime.rest.handler.router.RouterHandler.routed(RouterHandler.java:115)
at
org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:94)
at
org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:55)
at
org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at org.apache.flink.shaded.netty4.io.netty.channel.Abst
ractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at
org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at
org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at
org.apache.flink.shaded.netty4.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
at
org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at
org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at
org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at
org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:208)
at
[jira] [Updated] (FLINK-22521) flink stop 00000000000000000000000000000000 error
[
https://issues.apache.org/jira/browse/FLINK-22521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias updated FLINK-22521:
-
Description:
Flink K8S applicaton mode, using K8S HA. After running for two days, the
program could not be stopped. The running log is as follows:
{code:java}
/opt/flink/bin/flink stop --target
kubernetes-application -Dkubernetes.namespace=middle-flink
-Dkubernetes.config.file=/opt/flink/conf/kubeconfig -Dkubernetes.cluster-id=xxx
Suspending job "" with a savepoint.
2021-04-29 03:43:56,397 INFO
org.apache.flink.kubernetes.KubernetesClusterDescriptor [] - Retrieve flink
cluster xxx successfully, JobManager Web Interface:
[http:/xxx:16206|http://stock-calc-task-03-rest.middle-flink:16206/]
rs=1
The program finished with the following exception:
org.apache.flink.util.FlinkException: Could not stop with a savepoint job
"".
at org.apache.flink.client.cli.CliFrontend.lambda$stop$5(CliFrontend.java:585)
at
org.apache.flink.client.cli.CliFrontend.runClusterAction(CliFrontend.java:1006)
at org.apache.flink.client.cli.CliFrontend.stop(CliFrontend.java:573)
at org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1073)
at
org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1136)
at
org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:28)
at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1136)
Caused by: java.util.concurrent.ExecutionException:
org.apache.flink.runtime.rest.util.RestClientException:
[org.apache.flink.runtime.rest.NotFoundException: Operation not found under
key: org.apache.flink.runtime.rest.handl
er.job.AsynchronousJobOperationKey@38ca1e1b
at
org.apache.flink.runtime.rest.handler.async.AbstractAsynchronousOperationHandlers$StatusHandler.handleRequest(AbstractAsynchronousOperationHandlers.java:182)
at
org.apache.flink.runtime.rest.handler.job.savepoints.SavepointHandlers$SavepointStatusHandler.handleRequest(SavepointHandlers.java:219)
at
org.apache.flink.runtime.rest.handler.AbstractRestHandler.respondToRequest(AbstractRestHandler.java:83)
at
org.apache.flink.runtime.rest.handler.AbstractHandler.respondAsLeader(AbstractHandler.java:195)
at
org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.lambda$channelRead0$0(LeaderRetrievalHandler.java:83)
at java.util.Optional.ifPresent(Optional.java:159)
at org.apache.flink.util.OptionalConsumer.ifPresent(OptionalConsumer.java:45)
at
org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:80)
at
org.apache.flink.runtime.rest.handler.LeaderRetrievalHandler.channelRead0(LeaderRetrievalHandler.java:49)
at
org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at
org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at
org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at
org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at
org.apache.flink.runtime.rest.handler.router.RouterHandler.routed(RouterHandler.java:115)
at
org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:94)
at
org.apache.flink.runtime.rest.handler.router.RouterHandler.channelRead0(RouterHandler.java:55)
at
org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at org.apache.flink.shaded.netty4.io.netty.channel.Abst
ractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at
org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at
org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at
org.apache.flink.shaded.netty4.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
at
org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at
org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at
org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at
org.apache.flink.runtime.rest.FileUploadHandler.channelRead0(FileUploadHandler.java:208)
at
[jira] [Updated] (FLINK-22577) KubernetesLeaderElectionAndRetrievalITCase is failing
[
https://issues.apache.org/jira/browse/FLINK-22577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias updated FLINK-22577:
-
Description:
{{KubernetesLeaderElectionAndRetrievalITCase}} is failing constantly. Running
it locally results in an {{AssertionError}}:
{code}
3069 [KubernetesLeaderElector-ExecutorService-thread-1] DEBUG
io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector [] - Loop
thread interrupted
java.lang.InterruptedException: null
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:998)
~[?:1.8.0_265]
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
~[?:1.8.0_265]
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
~[?:1.8.0_265]
at
io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector.loop(LeaderElector.java:200)
~[kubernetes-client-4.9.2.jar:?]
at
io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector.renewWithTimeout(LeaderElector.java:100)
~[kubernetes-client-4.9.2.jar:?]
at
io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector.run(LeaderElector.java:71)
~[kubernetes-client-4.9.2.jar:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_265]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_265]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_265]
3078 [KubernetesLeaderElector-ExecutorService-thread-1] ERROR
org.apache.flink.runtime.util.FatalExitExceptionHandler [] - FATAL: Thread
'KubernetesLeaderElector-ExecutorService-thread-1' produced an uncaught
exception. Stopping the process...
java.lang.AssertionError: null
at
org.apache.flink.kubernetes.highavailability.KubernetesLeaderElectionDriver.writeLeaderInformation(KubernetesLeaderElectionDriver.java:130)
~[classes/:?]
at
org.apache.flink.runtime.leaderelection.TestingLeaderElectionEventHandler.lambda$onRevokeLeadership$1(TestingLeaderElectionEventHandler.java:69)
~[test-classes/:?]
at
org.apache.flink.runtime.leaderelection.TestingLeaderElectionEventHandler.waitForInitialization(TestingLeaderElectionEventHandler.java:93)
~[test-classes/:?]
at
org.apache.flink.runtime.leaderelection.TestingLeaderElectionEventHandler.onRevokeLeadership(TestingLeaderElectionEventHandler.java:66)
~[test-classes/:?]
at
org.apache.flink.kubernetes.highavailability.KubernetesLeaderElectionDriver$LeaderCallbackHandlerImpl.notLeader(KubernetesLeaderElectionDriver.java:202)
~[classes/:?]
at
io.fabric8.kubernetes.client.extended.leaderelection.LeaderCallbacks.onStopLeading(LeaderCallbacks.java:38)
~[kubernetes-client-4.9.2.jar:?]
at
io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector.run(LeaderElector.java:72)
~[kubernetes-client-4.9.2.jar:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
~[?:1.8.0_265]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
~[?:1.8.0_265]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_265]
{code}
The failure never popped up due to FLINK-22564
* [1.12 release
branch|https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=17554=logs=c88eea3b-64a0-564d-0031-9fdcd7b8abee=ff888d9b-cd34-53cc-d90f-3e446d355529=2701]
* [1.13 release
branch|https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=17558=logs=c88eea3b-64a0-564d-0031-9fdcd7b8abee=ff888d9b-cd34-53cc-d90f-3e446d355529=2760]
*
[master|https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=17560=logs=c88eea3b-64a0-564d-0031-9fdcd7b8abee=ff888d9b-cd34-53cc-d90f-3e446d355529=2757]
was:
{{KubernetesLeaderElectionAndRetrievalITCase}} is failing constantly. Running
it locally results in an {{AssertionError}}:
{code}
3069 [KubernetesLeaderElector-ExecutorService-thread-1] DEBUG
io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector [] - Loop
thread interrupted
java.lang.InterruptedException: null
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:998)
~[?:1.8.0_265]
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
~[?:1.8.0_265]
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
~[?:1.8.0_265]
at
io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector.loop(LeaderElector.java:200)
~[kubernetes-client-4.9.2.jar:?]
at
io.fabric8.kubernetes.client.extended.leaderelection.LeaderElector.renewWithTimeout(LeaderElector.java:100)
~[kubernetes-client-4.9.2.jar:?]
at
