[jira] [Commented] (FLINK-24512) Allow metadata columns can also be part of primary key
[
https://issues.apache.org/jira/browse/FLINK-24512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17427457#comment-17427457
]
Ingo Bürk commented on FLINK-24512:
---
This is a duplicate of https://issues.apache.org/jira/browse/FLINK-24050
> Allow metadata columns can also be part of primary key
> --
>
> Key: FLINK-24512
> URL: https://issues.apache.org/jira/browse/FLINK-24512
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / API
>Reporter: Jark Wu
>Priority: Major
>
> Currently, primary key can only be defined on physical columns. However, we
> encountered some cases need to define primary key on metadata columns.
> For example, there is MySQL sharding tables {{user_01}}, {{user_02}}, ...,
> {{user_99}} and they all use snowflake algorithm to generate global unique ID
> as the PK of each table. The user would like to read the sharding tables as
> one table and load into an OLAP with primary key of table name and ID. A
> Flink SQL can be following:
> {code}
> CREATE TABLE mysql_users (
> table_name STRING NOT NULL METADATA,
> id BIGINT NOT NULL,
> user_name STRING,
> address STRING,
> PRIMARY KEY (table_name, id) NOT ENFORCED
> ) WITH (
> 'connector' = 'mysql-cdc',
> 'database-name' = 'mydb',
> 'table-name' = 'user_.*',
> 'username' = 'xxx',
> 'password' = 'yyy'
> );
> {code}
> IMO, (non-virtual) metadata columns is similar to physical columns, because
> it is also some kind **physical** data in database. And our primary key
> definition is "NOT ENFORCED", which means we simply trust the unique
> constraint by users.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #16606: [FLINK-21357][runtime/statebackend]Periodic materialization for generalized incremental checkpoints
flinkbot edited a comment on pull request #16606: URL: https://github.com/apache/flink/pull/16606#issuecomment-887431748 ## CI report: * 061a4e7bb03dc54f0b909350bc01ec5936d08080 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=24943) * d71b68237ef0225a093b9f8392a38613a2787166 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=24974) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16606: [FLINK-21357][runtime/statebackend]Periodic materialization for generalized incremental checkpoints
flinkbot edited a comment on pull request #16606: URL: https://github.com/apache/flink/pull/16606#issuecomment-887431748 ## CI report: * 061a4e7bb03dc54f0b909350bc01ec5936d08080 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=24943) * d71b68237ef0225a093b9f8392a38613a2787166 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17452: [FLINK-20732][connector/pulsar] Introduction of Pulsar Sink
flinkbot edited a comment on pull request #17452: URL: https://github.com/apache/flink/pull/17452#issuecomment-940136217 ## CI report: * f3c9e05ce7ed2aa2155292b1be6e79f4c0aa6796 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=24969) * 8e3d80bc6fc29bf1865667fff9ab32495fd217f0 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=24973) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] jianyun8023 commented on pull request #17452: [FLINK-20732][connector/pulsar] Introduction of Pulsar Sink
jianyun8023 commented on pull request #17452: URL: https://github.com/apache/flink/pull/17452#issuecomment-940678179 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17452: [FLINK-20732][connector/pulsar] Introduction of Pulsar Sink
flinkbot edited a comment on pull request #17452: URL: https://github.com/apache/flink/pull/17452#issuecomment-940136217 ## CI report: * f3c9e05ce7ed2aa2155292b1be6e79f4c0aa6796 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=24969) * 8e3d80bc6fc29bf1865667fff9ab32495fd217f0 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-24376) Operator name in OperatorCoordinator should not use chained name
[
https://issues.apache.org/jira/browse/FLINK-24376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiangjie Qin updated FLINK-24376:
-
Fix Version/s: 1.14.1
> Operator name in OperatorCoordinator should not use chained name
>
>
> Key: FLINK-24376
> URL: https://issues.apache.org/jira/browse/FLINK-24376
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Task
>Affects Versions: 1.14.0, 1.12.5, 1.13.2, 1.14.1
>Reporter: Qingsheng Ren
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.14.1
>
>
> Currently the operator name passed to
> {{CoordinatedOperatorFactory#getCoordinatorProvider}} is a chained operator
> name (e.g. Source -> Map) instead of the name of coordinating operator, which
> might be misleading.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-24376) Operator name in OperatorCoordinator should not use chained name
[
https://issues.apache.org/jira/browse/FLINK-24376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiangjie Qin updated FLINK-24376:
-
Affects Version/s: (was: 1.14.1)
> Operator name in OperatorCoordinator should not use chained name
>
>
> Key: FLINK-24376
> URL: https://issues.apache.org/jira/browse/FLINK-24376
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Task
>Affects Versions: 1.14.0, 1.12.5, 1.13.2
>Reporter: Qingsheng Ren
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.14.1
>
>
> Currently the operator name passed to
> {{CoordinatedOperatorFactory#getCoordinatorProvider}} is a chained operator
> name (e.g. Source -> Map) instead of the name of coordinating operator, which
> might be misleading.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] becketqin merged pull request #17355: [FLINK-24376][runtime] Use operator name for constructing OperatorCoordinatorProvider instead of chained name
becketqin merged pull request #17355: URL: https://github.com/apache/flink/pull/17355 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-24512) Allow metadata columns can also be part of primary key
[
https://issues.apache.org/jira/browse/FLINK-24512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17427441#comment-17427441
]
Jark Wu commented on FLINK-24512:
-
What do you think [~twalthr]?
> Allow metadata columns can also be part of primary key
> --
>
> Key: FLINK-24512
> URL: https://issues.apache.org/jira/browse/FLINK-24512
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / API
>Reporter: Jark Wu
>Priority: Major
>
> Currently, primary key can only be defined on physical columns. However, we
> encountered some cases need to define primary key on metadata columns.
> For example, there is MySQL sharding tables {{user_01}}, {{user_02}}, ...,
> {{user_99}} and they all use snowflake algorithm to generate global unique ID
> as the PK of each table. The user would like to read the sharding tables as
> one table and load into an OLAP with primary key of table name and ID. A
> Flink SQL can be following:
> {code}
> CREATE TABLE mysql_users (
> table_name STRING NOT NULL METADATA,
> id BIGINT NOT NULL,
> user_name STRING,
> address STRING,
> PRIMARY KEY (table_name, id) NOT ENFORCED
> ) WITH (
> 'connector' = 'mysql-cdc',
> 'database-name' = 'mydb',
> 'table-name' = 'user_.*',
> 'username' = 'xxx',
> 'password' = 'yyy'
> );
> {code}
> IMO, (non-virtual) metadata columns is similar to physical columns, because
> it is also some kind **physical** data in database. And our primary key
> definition is "NOT ENFORCED", which means we simply trust the unique
> constraint by users.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-24512) Allow metadata columns can also be part of primary key
[
https://issues.apache.org/jira/browse/FLINK-24512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu updated FLINK-24512:
Description:
Currently, primary key can only be defined on physical columns. However, we
encountered some cases need to define primary key on metadata columns.
For example, there is MySQL sharding tables {{user_01}}, {{user_02}}, ...,
{{user_99}} and they all use snowflake algorithm to generate global unique ID
as the PK of each table. The user would like to read the sharding tables as one
table and load into an OLAP with primary key of table name and ID. A Flink SQL
can be following:
{code}
CREATE TABLE mysql_users (
table_name STRING NOT NULL METADATA,
id BIGINT NOT NULL,
user_name STRING,
address STRING,
PRIMARY KEY (table_name, id) NOT ENFORCED
) WITH (
'connector' = 'mysql-cdc',
'database-name' = 'mydb',
'table-name' = 'user_.*',
'username' = 'xxx',
'password' = 'yyy'
);
{code}
IMO, (non-virtual) metadata columns is similar to physical columns, because it
is also some kind **physical** data in database. And our primary key definition
is "NOT ENFORCED", which means we simply trust the unique constraint by users.
was:
Currently, primary key can only be defined on physical columns. However, we
encountered some cases need to define primary key on metadata columns.
For example, there is MySQL sharding tables {{user_01}}, {{user_02}}, ...,
{{user_99}} and they all use snowflake algorithm to generate global unique ID
as the PK of each table. The user would like to read the sharding tables as one
table and load into an OLAP with primary key of table name and ID. A Flink SQL
can be following:
{code}
CREATE TABLE mysql_users (
table_name STRING NOT NULL METADATA,
id BIGINT NOT NULL,
user_name STRING,
address STRING,
PRIMARY KEY (table_name, id) NOT ENFORCED
) WITH (
'connector' = 'mysql-cdc',
'database-name' = 'mydb',
'table-name' = 'user_.*',
'username' = 'xxx',
'password' = 'yyy'
);
{code}
> Allow metadata columns can also be part of primary key
> --
>
> Key: FLINK-24512
> URL: https://issues.apache.org/jira/browse/FLINK-24512
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / API
>Reporter: Jark Wu
>Priority: Major
>
> Currently, primary key can only be defined on physical columns. However, we
> encountered some cases need to define primary key on metadata columns.
> For example, there is MySQL sharding tables {{user_01}}, {{user_02}}, ...,
> {{user_99}} and they all use snowflake algorithm to generate global unique ID
> as the PK of each table. The user would like to read the sharding tables as
> one table and load into an OLAP with primary key of table name and ID. A
> Flink SQL can be following:
> {code}
> CREATE TABLE mysql_users (
> table_name STRING NOT NULL METADATA,
> id BIGINT NOT NULL,
> user_name STRING,
> address STRING,
> PRIMARY KEY (table_name, id) NOT ENFORCED
> ) WITH (
> 'connector' = 'mysql-cdc',
> 'database-name' = 'mydb',
> 'table-name' = 'user_.*',
> 'username' = 'xxx',
> 'password' = 'yyy'
> );
> {code}
> IMO, (non-virtual) metadata columns is similar to physical columns, because
> it is also some kind **physical** data in database. And our primary key
> definition is "NOT ENFORCED", which means we simply trust the unique
> constraint by users.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-24512) Allow metadata columns can also be part of primary key
Jark Wu created FLINK-24512:
---
Summary: Allow metadata columns can also be part of primary key
Key: FLINK-24512
URL: https://issues.apache.org/jira/browse/FLINK-24512
Project: Flink
Issue Type: Bug
Components: Table SQL / API
Reporter: Jark Wu
Currently, primary key can only be defined on physical columns. However, we
encountered some cases need to define primary key on metadata columns.
For example, there is MySQL sharding tables {{user_01}}, {{user_02}}, ...,
{{user_99}} and they all use snowflake algorithm to generate global unique ID
as the PK of each table. The user would like to read the sharding tables as one
table and load into an OLAP with primary key of table name and ID. A Flink SQL
can be following:
{code}
CREATE TABLE mysql_users (
table_name STRING NOT NULL METADATA,
id BIGINT NOT NULL,
user_name STRING,
address STRING,
PRIMARY KEY (table_name, id) NOT ENFORCED
) WITH (
'connector' = 'mysql-cdc',
'database-name' = 'mydb',
'table-name' = 'user_.*',
'username' = 'xxx',
'password' = 'yyy'
);
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-24511) "sink.parallelism" doesn't work for upsert "values" sink
Jark Wu created FLINK-24511: --- Summary: "sink.parallelism" doesn't work for upsert "values" sink Key: FLINK-24511 URL: https://issues.apache.org/jira/browse/FLINK-24511 Project: Flink Issue Type: Bug Components: Table SQL / Planner Affects Versions: 1.13.2, 1.14.0 Reporter: Jark Wu In values sink, "sink.parallelism" is only applied for insert-only sinks, not upsert sinks. This makes hard to construct a job with keyby shuffle before upsert sink. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-24122) Add support to do clean in history server
[
https://issues.apache.org/jira/browse/FLINK-24122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17427431#comment-17427431
]
zlzhang0122 commented on FLINK-24122:
-
[~trohrmann] IMO the new option is the addition for the retention number, if
users configure both retention number and retention time, they will take effect
at the same time, but if users just configure one of them, it should also take
effect separately.
> Add support to do clean in history server
> -
>
> Key: FLINK-24122
> URL: https://issues.apache.org/jira/browse/FLINK-24122
> Project: Flink
> Issue Type: Bug
> Components: Runtime / REST
>Affects Versions: 1.12.3, 1.13.2
>Reporter: zlzhang0122
>Priority: Minor
> Fix For: 1.14.1
>
>
> Now, the history server can clean history jobs by two means:
> # if users have configured
> {code:java}
> historyserver.archive.clean-expired-jobs: true{code}
> , then compare the files in hdfs over two clean interval and find the delete
> and clean the local cache file.
> # if users have configured the
> {code:java}
> historyserver.archive.retained-jobs:{code}
> a positive number, then clean the oldest files in hdfs and local.
> But the retained-jobs number is difficult to determine.
> For example, users may want to check the history jobs yesterday while many
> jobs failed today and exceed the retained-jobs number, then the history jobs
> of yesterday will be delete. So what if add a configuration which contain a
> retained-times that indicate the max time the history job retain?
> Also it can't clean the job history files which was no longer in hdfs but
> still cached in local filesystem and these files will store forever and can't
> be cleaned unless users manually do this. Maybe we can give a option and do
> this clean if the option says true.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-24510) Fix the duplicate log output of libraries/connectors，when logback is used
[
https://issues.apache.org/jira/browse/FLINK-24510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17427428#comment-17427428
]
xiangqiao commented on FLINK-24510:
---
I have contributed PR, who can help assign the ticket to me?
thank you.
> Fix the duplicate log output of libraries/connectors，when logback is used
> -
>
> Key: FLINK-24510
> URL: https://issues.apache.org/jira/browse/FLINK-24510
> Project: Flink
> Issue Type: Bug
> Components: Deployment / Scripts
>Affects Versions: 1.14.0
>Reporter: xiangqiao
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2021-10-12-10-52-22-389.png
>
>
> When logback is used, the logs of libraries / connectors are output
> repeatedly.
> !image-2021-10-12-10-52-22-389.png!
>
> By default, *appenders are cumulative*: a logger will log to the appenders
> attached to itself (if any) as well as all the appenders attached to its
> ancestors. Thus, attaching the same appender to multiple loggers will cause
> logging output to be duplicated.
> [http://logback.qos.ch/manual/configuration.html#cumulative]
>
> We can solve this problem in the following ways：
> {code:java}
>
>
>
>
>
>
>
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-24479) Bug fix RocksIteratorWrapper#seekToLast method
[ https://issues.apache.org/jira/browse/FLINK-24479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17427427#comment-17427427 ] xiangqiao commented on FLINK-24479: --- I have contributed PR, who can help assign the ticket to me? thank you. > Bug fix RocksIteratorWrapper#seekToLast method > -- > > Key: FLINK-24479 > URL: https://issues.apache.org/jira/browse/FLINK-24479 > Project: Flink > Issue Type: Bug > Components: Runtime / State Backends >Affects Versions: 1.14.0 >Reporter: xiangqiao >Priority: Major > Labels: pull-request-available > > RocksIteratorWrapper#seekToLast method, calling RocksIterator#seekToFirst > method incorrectly. > The RocksIteratorWrapper#seekToLast method is not currently used, so it has > no impact on the flink project. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot commented on pull request #17453: [FLINK-24510][Deployment/Scripts]Fix the duplicate log output of libraries/connectors，when logback is used
flinkbot commented on pull request #17453: URL: https://github.com/apache/flink/pull/17453#issuecomment-940623146 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 853bce04c09bf00f130a0dcebac03e73a799db9e (Tue Oct 12 03:27:31 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-24510).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17453: [FLINK-24510][Deployment/Scripts]Fix the duplicate log output of libraries/connectors，when logback is used
flinkbot edited a comment on pull request #17453: URL: https://github.com/apache/flink/pull/17453#issuecomment-940622443 ## CI report: * 853bce04c09bf00f130a0dcebac03e73a799db9e Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=24970) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-24384) Count checkpoints failed in trigger phase into numberOfFailedCheckpoints
[ https://issues.apache.org/jira/browse/FLINK-24384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17427423#comment-17427423 ] Feifan Wang commented on FLINK-24384: - Hi [~pnowojski], how do you think about this proposal ? > Count checkpoints failed in trigger phase into numberOfFailedCheckpoints > > > Key: FLINK-24384 > URL: https://issues.apache.org/jira/browse/FLINK-24384 > Project: Flink > Issue Type: Improvement > Components: Runtime / Checkpointing >Reporter: Feifan Wang >Priority: Major > > h1. *Problem* > In current implementation, checkpoints failed in trigger phase do not count > into metric 'numberOfFailedCheckpoints'. Such that users can not aware > checkpoint stoped by this metric. > As lang as users can use rules like _*'numberOfCompletedCheckpoints' not > increase in some minutes past*_ (maybe checkpoint interval + timeout) for > alerting, but I think it is ambages and can not alert timely. > > h1. *Proposal* > As the title, count checkpoints failed in trigger phase into > 'numberOfFailedCheckpoints'. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot commented on pull request #17453: [FLINK-24510][Deployment/Scripts]Fix the duplicate log output of libraries/connectors，when logback is used
flinkbot commented on pull request #17453: URL: https://github.com/apache/flink/pull/17453#issuecomment-940622443 ## CI report: * 853bce04c09bf00f130a0dcebac03e73a799db9e UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-24510) Fix the duplicate log output of libraries/connectors，when logback is used
[
https://issues.apache.org/jira/browse/FLINK-24510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-24510:
---
Labels: pull-request-available (was: )
> Fix the duplicate log output of libraries/connectors，when logback is used
> -
>
> Key: FLINK-24510
> URL: https://issues.apache.org/jira/browse/FLINK-24510
> Project: Flink
> Issue Type: Bug
> Components: Deployment / Scripts
>Affects Versions: 1.14.0
>Reporter: xiangqiao
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2021-10-12-10-52-22-389.png
>
>
> When logback is used, the logs of libraries / connectors are output
> repeatedly.
> !image-2021-10-12-10-52-22-389.png!
>
> By default, *appenders are cumulative*: a logger will log to the appenders
> attached to itself (if any) as well as all the appenders attached to its
> ancestors. Thus, attaching the same appender to multiple loggers will cause
> logging output to be duplicated.
> [http://logback.qos.ch/manual/configuration.html#cumulative]
>
> We can solve this problem in the following ways：
> {code:java}
>
>
>
>
>
>
>
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] xiangqiao123 opened a new pull request #17453: [FLINK-24510][Deployment/Scripts]Fix the duplicate log output of libraries/connectors，when logback is used
xiangqiao123 opened a new pull request #17453: URL: https://github.com/apache/flink/pull/17453 ## Fix the duplicate log output of libraries/connectors，when logback is used When logback is used, the logs of libraries / connectors are output repeatedly. By default, appenders are cumulative: a logger will log to the appenders attached to itself (if any) as well as all the appenders attached to its ancestors. Thus, attaching the same appender to multiple loggers will cause logging output to be duplicated. http://logback.qos.ch/manual/configuration.html#cumulative ## Verifying this change This change is a trivial rework / code cleanup without any test coverage. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / **no**) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**) - The serializers: (yes / **no** / don't know) - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / **no** / don't know) - The S3 file system connector: (yes / **no** / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / **no**) - If yes, how is the feature documented? (**not applicable** / docs / JavaDocs / not documented) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (FLINK-10230) Support 'SHOW CREATE VIEW' syntax to print the query of a view
[
https://issues.apache.org/jira/browse/FLINK-10230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17425935#comment-17425935
]
Roc Marshal edited comment on FLINK-10230 at 10/12/21, 3:21 AM:
Could you [~jark] [~MartijnVisser] [~twalthr] help me to merge it if there are
nothing inappropriate ? Thank you.
was (Author: rocmarshal):
Could you [~jark] [~MartijnVisser] help me to merge it if there are nothing
inappropriate ? Thank you.
> Support 'SHOW CREATE VIEW' syntax to print the query of a view
> --
>
> Key: FLINK-10230
> URL: https://issues.apache.org/jira/browse/FLINK-10230
> Project: Flink
> Issue Type: New Feature
> Components: Table SQL / API, Table SQL / Client
>Reporter: Timo Walther
>Assignee: Roc Marshal
>Priority: Minor
> Labels: auto-deprioritized-major, auto-unassigned,
> pull-request-available
> Fix For: 1.15.0
>
>
> FLINK-10163 added initial support for views in SQL Client. We should add a
> command that allows for printing the query of a view for debugging. MySQL
> offers {{SHOW CREATE VIEW}} for this. Hive generalizes this to {{SHOW CREATE
> TABLE}}. The latter one could be extended to also show information about the
> used table factories and properties.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-24510) Fix the duplicate log output of libraries/connectors，when logback is used
xiangqiao created FLINK-24510:
-
Summary: Fix the duplicate log output of libraries/connectors，when
logback is used
Key: FLINK-24510
URL: https://issues.apache.org/jira/browse/FLINK-24510
Project: Flink
Issue Type: Bug
Components: Deployment / Scripts
Affects Versions: 1.14.0
Reporter: xiangqiao
Attachments: image-2021-10-12-10-52-22-389.png
When logback is used, the logs of libraries / connectors are output repeatedly.
!image-2021-10-12-10-52-22-389.png!
By default, *appenders are cumulative*: a logger will log to the appenders
attached to itself (if any) as well as all the appenders attached to its
ancestors. Thus, attaching the same appender to multiple loggers will cause
logging output to be duplicated.
[http://logback.qos.ch/manual/configuration.html#cumulative]
We can solve this problem in the following ways：
{code:java}
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (FLINK-9465) Specify a separate savepoint timeout option via CLI
[ https://issues.apache.org/jira/browse/FLINK-9465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17426559#comment-17426559 ] Feifan Wang edited comment on FLINK-9465 at 10/12/21, 3:02 AM: --- Hi [~trohrmann], I open a [pull request|https://github.com/apache/flink/pull/17443] to resolve this, but there are still some unit test problems need to be complete. I will try fix it myself before request you review it. was (Author: feifan wang): Hi [~trohrmann], I open a [pull request|https://github.com/apache/flink/pull/17443] to resolve this, but there are still some unit test that I think need to be complete. Can you take a glance over this PR and give me some guidance on the unit test ? > Specify a separate savepoint timeout option via CLI > --- > > Key: FLINK-9465 > URL: https://issues.apache.org/jira/browse/FLINK-9465 > Project: Flink > Issue Type: Improvement > Components: Runtime / Checkpointing >Affects Versions: 1.5.0 >Reporter: Truong Duc Kien >Assignee: Feifan Wang >Priority: Minor > Labels: auto-deprioritized-major, pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Savepoint can take much longer time to perform than checkpoint, especially > with incremental checkpoint enabled. This leads to a couple of troubles: > * For our job, we currently have to set the checkpoint timeout much large > than necessary, otherwise we would be unable to perform savepoint. > * During rush hour, our cluster would encounter high rate of checkpoint > timeout due to backpressure, however we're unable to migrate to a larger > configuration, because savepoint also timeout. > In my opinion, the timeout for savepoint should be configurable separately, > both in the config file and as parameter to the savepoint command. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] zicat commented on pull request #17449: [FLINK-24504][table-planner] Flink common expression elimination
zicat commented on pull request #17449: URL: https://github.com/apache/flink/pull/17449#issuecomment-940602034 > A simple map with the code as key doesn't seem sufficient to me. What if the function is non-deterministic? What if it is called with arguments which are calls to non-deterministic functions? > > I think in general this should be handled in the optimizer, not the code generation. Thank you very much for your quick reply, I will try to research whether can optimize the RexNode to eliminate common expression. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17452: [FLINK-20732][connector/pulsar] Introduction of Pulsar Sink
flinkbot edited a comment on pull request #17452: URL: https://github.com/apache/flink/pull/17452#issuecomment-940136217 ## CI report: * f3c9e05ce7ed2aa2155292b1be6e79f4c0aa6796 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=24969) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] leonardBang commented on pull request #16639: [FLINK-23549][connector][kafka][docs] - Corrected syntax error in create table example
leonardBang commented on pull request #16639: URL: https://github.com/apache/flink/pull/16639#issuecomment-940591666 LGTM, I'll touch @wuchong to help merge this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-24480) EqualiserCodeGeneratorTest fails on azure
[
https://issues.apache.org/jira/browse/FLINK-24480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17427410#comment-17427410
]
Xintong Song commented on FLINK-24480:
--
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=24966=logs=955770d3-1fed-5a0a-3db6-0c7554c910cb=14447d61-56b4-5000-80c1-daa459247f6a=42433
> EqualiserCodeGeneratorTest fails on azure
> -
>
> Key: FLINK-24480
> URL: https://issues.apache.org/jira/browse/FLINK-24480
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.12.5, 1.13.2
>Reporter: Xintong Song
>Priority: Major
> Labels: test-stability
> Fix For: 1.12.6, 1.13.4
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=24809=logs=955770d3-1fed-5a0a-3db6-0c7554c910cb=14447d61-56b4-5000-80c1-daa459247f6a=42615
> {code}
> Oct 07 01:11:46 [ERROR] Tests run: 3, Failures: 1, Errors: 0, Skipped: 0,
> Time elapsed: 8.236 s <<< FAILURE! - in
> org.apache.flink.table.planner.codegen.EqualiserCodeGeneratorTest
> Oct 07 01:11:46 [ERROR]
> testManyFields(org.apache.flink.table.planner.codegen.EqualiserCodeGeneratorTest)
> Time elapsed: 8.21 s <<< FAILURE!
> Oct 07 01:11:46 java.lang.AssertionError: Expected compilation to succeed
> Oct 07 01:11:46 at org.junit.Assert.fail(Assert.java:88)
> Oct 07 01:11:46 at
> org.apache.flink.table.planner.codegen.EqualiserCodeGeneratorTest.testManyFields(EqualiserCodeGeneratorTest.java:102)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-24503) Security: native kubernetes exposes REST service via LoadBalancer in default
[ https://issues.apache.org/jira/browse/FLINK-24503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17427409#comment-17427409 ] Xintong Song commented on FLINK-24503: -- My understanding of "trying out Flink" does not include running it in a production environment, or a cloud cluster that is sensitive to public access. However, I think you do have a good point, that the default config brings more threats to the production use cases (easily exposing REST access to wild Internet) than benefits to the trying out use cases (one step less, configuring it to load balancer or setting a port forwarding). Let's make sure this change is well described in the release notes and user documentations. > Security: native kubernetes exposes REST service via LoadBalancer in default > > > Key: FLINK-24503 > URL: https://issues.apache.org/jira/browse/FLINK-24503 > Project: Flink > Issue Type: Improvement > Components: Deployment / Kubernetes >Affects Versions: 1.13.0, 1.14.0, 1.13.1, 1.13.2 > Environment: Flink 1.13.2, native kubernetes >Reporter: LI Zhennan >Priority: Major > Labels: security > > Hi, > > Flink native k8s deployment exposes REST service via LoadBalancer in default: > https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/config/#kubernetes-rest-service-exposed-type > I propose to consider it a security issue. > It is very likely for users to unconciously expose their Flink REST service > to the wild Internet, given they are deploying on a k8s cluster provided by > cloud service like AWS or Google Cloud. > Given access, anyone can browse and cancel Flink job on REST service. > Personally I noticed this issue after my staging deployment went online for 2 > days. > Here, I propose to alter the default value to `ClusterIP`, so that: > # the REST service is not exposed to Internet accidentally; > # the developer can use `kubectl port-forward` to access the service in > default; > # the developer can still expose REST service via LoadBalancer by expressing > it explicitly in `flink run-application` params. > If it is okay, I would like to contribute the fix. > > Thank you. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #17452: [FLINK-20732][connector/pulsar] Introduction of Pulsar Sink
flinkbot edited a comment on pull request #17452: URL: https://github.com/apache/flink/pull/17452#issuecomment-940136217 ## CI report: * 7a74987d989f699bc68c2da8e0cd9bf3fa8065a6 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=24963) * f3c9e05ce7ed2aa2155292b1be6e79f4c0aa6796 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=24969) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] jianyun8023 commented on pull request #17452: [FLINK-20732][connector/pulsar] Introduction of Pulsar Sink
jianyun8023 commented on pull request #17452: URL: https://github.com/apache/flink/pull/17452#issuecomment-940568811 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17452: [FLINK-20732][connector/pulsar] Introduction of Pulsar Sink
flinkbot edited a comment on pull request #17452: URL: https://github.com/apache/flink/pull/17452#issuecomment-940136217 ## CI report: * 7a74987d989f699bc68c2da8e0cd9bf3fa8065a6 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=24963) * f3c9e05ce7ed2aa2155292b1be6e79f4c0aa6796 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=24969) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17452: [FLINK-20732][connector/pulsar] Introduction of Pulsar Sink
flinkbot edited a comment on pull request #17452: URL: https://github.com/apache/flink/pull/17452#issuecomment-940136217 ## CI report: * 7a74987d989f699bc68c2da8e0cd9bf3fa8065a6 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=24963) * f3c9e05ce7ed2aa2155292b1be6e79f4c0aa6796 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-statefun] sjwiesman commented on pull request #274: [FLINK-22533] Add custom metrics
sjwiesman commented on pull request #274: URL: https://github.com/apache/flink-statefun/pull/274#issuecomment-940558983 The general implementation looks good. My only comment is on the API. It feels strange to me that we have `FunctionTypeScopedMetrics` and `RouterMetrics` as two separate interfaces. Could we unify them into one interface? I also really like the cleanup of `LocalFunctionGroup`, that whole part of the codebase is much easier to follow :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16639: [FLINK-23549][connector][kafka][docs] - Corrected syntax error in create table example
flinkbot edited a comment on pull request #16639: URL: https://github.com/apache/flink/pull/16639#issuecomment-889540294 ## CI report: * b01a85efd6798de784f114278518874a2974b830 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21184) Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=21199) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] mans2singh commented on pull request #16639: [FLINK-23549][connector][kafka][docs] - Corrected syntax error in create table example
mans2singh commented on pull request #16639: URL: https://github.com/apache/flink/pull/16639#issuecomment-940528878 @leonardBang @wuchong - Can you please review this PR and let me know your recommendations ? Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17345: [FLINK-24227][connectors] FLIP-171: Added Kinesis Data Streams Sink i…
flinkbot edited a comment on pull request #17345: URL: https://github.com/apache/flink/pull/17345#issuecomment-926109717 ## CI report: * 3e47c0e0650872e5435c03e7b1e0c44da1eb0587 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=24964) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-22545) JVM crashes when running OperatorEventSendingCheckpointITCase.testOperatorEventAckLost
[
https://issues.apache.org/jira/browse/FLINK-22545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler updated FLINK-22545:
-
Release Note: (was: Fixed in 1.12.6 ({{release-1.12}}) via
c22c3daa1ff661c32c21f6a8a16cfc0926d5895a)
> JVM crashes when running
> OperatorEventSendingCheckpointITCase.testOperatorEventAckLost
> --
>
> Key: FLINK-22545
> URL: https://issues.apache.org/jira/browse/FLINK-22545
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination, Tests
>Affects Versions: 1.12.3
>Reporter: Guowei Ma
>Assignee: Stephan Ewen
>Priority: Critical
> Labels: pull-request-available, test-stability
> Fix For: 1.14.0, 1.12.6, 1.13.3
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=17501=logs=39d5b1d5-3b41-54dc-6458-1e2ddd1cdcf3=a99e99c7-21cd-5a1f-7274-585e62b72f56=4287
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-24133) PartitionRequestClientFactoryTest.testThrowsWhenNetworkFailure fails on Azure
[
https://issues.apache.org/jira/browse/FLINK-24133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler updated FLINK-24133:
-
Issue Type: Technical Debt (was: Bug)
> PartitionRequestClientFactoryTest.testThrowsWhenNetworkFailure fails on Azure
> -
>
> Key: FLINK-24133
> URL: https://issues.apache.org/jira/browse/FLINK-24133
> Project: Flink
> Issue Type: Technical Debt
> Components: Runtime / Network
>Affects Versions: 1.14.0, 1.12.5, 1.13.2, 1.15.0
>Reporter: Till Rohrmann
>Assignee: Anton Kalashnikov
>Priority: Major
> Labels: pull-request-available, test-stability
> Fix For: 1.14.0, 1.12.6, 1.13.3
>
>
> The test {{PartitionRequestClientFactoryTest.testThrowsWhenNetworkFailure}}
> fails on Azure with
> {code}
> 2021-09-01T17:01:08.2338015Z Sep 01 17:01:08 [ERROR] Tests run: 7, Failures:
> 1, Errors: 0, Skipped: 0, Time elapsed: 1.465 s <<< FAILURE! - in
> org.apache.flink.runtime.io.network.netty.PartitionRequestClientFactoryTest
> 2021-09-01T17:01:08.2341002Z Sep 01 17:01:08 [ERROR]
> testThrowsWhenNetworkFailure Time elapsed: 0.02 s <<< FAILURE!
> 2021-09-01T17:01:08.2341956Z Sep 01 17:01:08 java.lang.AssertionError:
> Expected exception:
> org.apache.flink.runtime.io.network.netty.exception.RemoteTransportException
> 2021-09-01T17:01:08.2342854Z Sep 01 17:01:08 at
> org.junit.internal.runners.statements.ExpectException.evaluate(ExpectException.java:34)
> 2021-09-01T17:01:08.2343408Z Sep 01 17:01:08 at
> org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> 2021-09-01T17:01:08.2343916Z Sep 01 17:01:08 at
> org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
> 2021-09-01T17:01:08.2348096Z Sep 01 17:01:08 at
> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
> 2021-09-01T17:01:08.2483997Z Sep 01 17:01:08 at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
> 2021-09-01T17:01:08.2484833Z Sep 01 17:01:08 at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
> 2021-09-01T17:01:08.2485521Z Sep 01 17:01:08 at
> org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
> 2021-09-01T17:01:08.2486189Z Sep 01 17:01:08 at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
> 2021-09-01T17:01:08.2486892Z Sep 01 17:01:08 at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
> 2021-09-01T17:01:08.2487565Z Sep 01 17:01:08 at
> org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
> 2021-09-01T17:01:08.2488276Z Sep 01 17:01:08 at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
> 2021-09-01T17:01:08.2488999Z Sep 01 17:01:08 at
> org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> 2021-09-01T17:01:08.2489818Z Sep 01 17:01:08 at
> org.junit.runners.ParentRunner.run(ParentRunner.java:413)
> 2021-09-01T17:01:08.2490535Z Sep 01 17:01:08 at
> org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> 2021-09-01T17:01:08.2491159Z Sep 01 17:01:08 at
> org.junit.runner.JUnitCore.run(JUnitCore.java:115)
> 2021-09-01T17:01:08.2491860Z Sep 01 17:01:08 at
> org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:43)
> 2021-09-01T17:01:08.2492760Z Sep 01 17:01:08 at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
> 2021-09-01T17:01:08.2493506Z Sep 01 17:01:08 at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> 2021-09-01T17:01:08.2494192Z Sep 01 17:01:08 at
> java.util.Iterator.forEachRemaining(Iterator.java:116)
> 2021-09-01T17:01:08.2494910Z Sep 01 17:01:08 at
> java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
> 2021-09-01T17:01:08.2495889Z Sep 01 17:01:08 at
> java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
> 2021-09-01T17:01:08.2496648Z Sep 01 17:01:08 at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> 2021-09-01T17:01:08.2497416Z Sep 01 17:01:08 at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
> 2021-09-01T17:01:08.2498201Z Sep 01 17:01:08 at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
> 2021-09-01T17:01:08.2498950Z Sep 01 17:01:08 at
> java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> 2021-09-01T17:01:08.2499767Z Sep 01 17:01:08 at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
> 2021-09-01T17:01:08.2500567Z Sep 01 17:01:08 at
> org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:82)
> 2021-09-01T17:01:08.2501381Z Sep 01 17:01:08 at
>
[jira] [Updated] (FLINK-24120) Document MALLOC_ARENA_MAX as workaround for glibc memory leak
[ https://issues.apache.org/jira/browse/FLINK-24120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chesnay Schepler updated FLINK-24120: - Summary: Document MALLOC_ARENA_MAX as workaround for glibc memory leak (was: The memory will rise a little after each savepoint ends) > Document MALLOC_ARENA_MAX as workaround for glibc memory leak > - > > Key: FLINK-24120 > URL: https://issues.apache.org/jira/browse/FLINK-24120 > Project: Flink > Issue Type: Bug > Components: Documentation, flink-docker >Affects Versions: 1.14.0, 1.13.2 > Environment: flink on k8s > flink version : 1.13.2 > state.backend: rocksdb > state.checkpoints.dir: hdfs_url > state.savepoints.dir: hdfs_url > state.backend.rocksdb.memory.managed: true > >Reporter: no nO >Assignee: Jiayi Liao >Priority: Major > Labels: pull-request-available > Fix For: 1.14.0, 1.13.3 > > Attachments: image-2021-09-02-10-40-36-043.png, 截屏2021-09-02 > 上午10.44.10.png > > > !image-2021-09-02-10-40-36-043.png! > My task will do a savepoint every hour, so every hour will do a savepoint. > From the memory monitoring, it can be seen that the memory of each hour will > soar up, although the memory will drop a little later, but from every hour > From the point of view of the memory peak on the whole point, the memory > continues to rise little by little, and eventually rises to the limited > memory, which will lead to being killed by k8S > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-24120) Document MALLOC_ARENA_MAX as workaround for glibc memory leak
[ https://issues.apache.org/jira/browse/FLINK-24120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chesnay Schepler updated FLINK-24120: - Issue Type: Improvement (was: Bug) > Document MALLOC_ARENA_MAX as workaround for glibc memory leak > - > > Key: FLINK-24120 > URL: https://issues.apache.org/jira/browse/FLINK-24120 > Project: Flink > Issue Type: Improvement > Components: Documentation, flink-docker >Affects Versions: 1.14.0, 1.13.2 > Environment: flink on k8s > flink version : 1.13.2 > state.backend: rocksdb > state.checkpoints.dir: hdfs_url > state.savepoints.dir: hdfs_url > state.backend.rocksdb.memory.managed: true > >Reporter: no nO >Assignee: Jiayi Liao >Priority: Major > Labels: pull-request-available > Fix For: 1.14.0, 1.13.3 > > Attachments: image-2021-09-02-10-40-36-043.png, 截屏2021-09-02 > 上午10.44.10.png > > > !image-2021-09-02-10-40-36-043.png! > My task will do a savepoint every hour, so every hour will do a savepoint. > From the memory monitoring, it can be seen that the memory of each hour will > soar up, although the memory will drop a little later, but from every hour > From the point of view of the memory peak on the whole point, the memory > continues to rise little by little, and eventually rises to the limited > memory, which will lead to being killed by k8S > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-23845) Clarify metric deletion guarantees for Prometheus PushGateway reporter on shutdown
[ https://issues.apache.org/jira/browse/FLINK-23845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chesnay Schepler updated FLINK-23845: - Summary: Clarify metric deletion guarantees for Prometheus PushGateway reporter on shutdown (was: [DOCS]PushGateway metrics group not delete when job shutdown) > Clarify metric deletion guarantees for Prometheus PushGateway reporter on > shutdown > -- > > Key: FLINK-23845 > URL: https://issues.apache.org/jira/browse/FLINK-23845 > Project: Flink > Issue Type: Bug > Components: Documentation >Affects Versions: 1.14.0, 1.12.5, 1.13.2 >Reporter: camilesing >Assignee: camilesing >Priority: Minor > Labels: pull-request-available, stale-blocker > Fix For: 1.14.0, 1.13.3 > > > see https://issues.apache.org/jira/browse/FLINK-20691 . whatever the problem > has always existed, we should avoid other guys met it -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-23628) MeasurementInfoProviderTest.testNormalizingTags fails on Azure
[
https://issues.apache.org/jira/browse/FLINK-23628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler updated FLINK-23628:
-
Issue Type: Technical Debt (was: Bug)
> MeasurementInfoProviderTest.testNormalizingTags fails on Azure
> --
>
> Key: FLINK-23628
> URL: https://issues.apache.org/jira/browse/FLINK-23628
> Project: Flink
> Issue Type: Technical Debt
> Components: Runtime / Metrics
>Affects Versions: 1.12.6, 1.13.3
>Reporter: Dawid Wysakowicz
>Assignee: Chesnay Schepler
>Priority: Major
> Fix For: 1.12.6, 1.13.3
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=21496=logs=d44f43ce-542c-597d-bf94-b0718c71e5e8=03dca39c-73e8-5aaf-601d-328ae5c35f20=15164
> {code}
> [INFO] Running org.apache.flink.metrics.influxdb.InfluxdbReporterTest
> [ERROR] Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.609
> s <<< FAILURE! - in
> org.apache.flink.metrics.influxdb.MeasurementInfoProviderTest
> [ERROR]
> testNormalizingTags(org.apache.flink.metrics.influxdb.MeasurementInfoProviderTest)
> Time elapsed: 0.008 s <<< ERROR!
> java.lang.UnsupportedOperationException: unexpected method call
> at
> org.apache.flink.metrics.influxdb.MeasurementInfoProviderTest.lambda$testNormalizingTags$1(MeasurementInfoProviderTest.java:79)
> at
> org.mockito.internal.handler.MockHandlerImpl.handle(MockHandlerImpl.java:103)
> at
> org.mockito.internal.handler.NullResultGuardian.handle(NullResultGuardian.java:29)
> at
> org.mockito.internal.handler.InvocationNotifierHandler.handle(InvocationNotifierHandler.java:35)
> at
> org.mockito.internal.creation.bytebuddy.MockMethodInterceptor.doIntercept(MockMethodInterceptor.java:63)
> at
> org.mockito.internal.creation.bytebuddy.MockMethodInterceptor.doIntercept(MockMethodInterceptor.java:49)
> at
> org.mockito.internal.creation.bytebuddy.MockMethodInterceptor$DispatcherDefaultingToRealMethod.interceptSuperCallable(MockMethodInterceptor.java:110)
> at
> org.apache.flink.runtime.metrics.groups.FrontMetricGroup$MockitoMock$1748504994.getLogicalScope(Unknown
> Source)
> at
> org.apache.flink.metrics.influxdb.MeasurementInfoProvider.getLogicalScope(MeasurementInfoProvider.java:70)
> at
> org.apache.flink.metrics.influxdb.MeasurementInfoProvider.getScopedName(MeasurementInfoProvider.java:65)
> at
> org.apache.flink.metrics.influxdb.MeasurementInfoProvider.getMetricInfo(MeasurementInfoProvider.java:49)
> at
> org.apache.flink.metrics.influxdb.MeasurementInfoProviderTest.testNormalizingTags(MeasurementInfoProviderTest.java:83)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
> at
>
[jira] [Updated] (FLINK-23618) Compilation error for MeasurementInfoProviderTest
[
https://issues.apache.org/jira/browse/FLINK-23618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler updated FLINK-23618:
-
Issue Type: Technical Debt (was: Bug)
> Compilation error for MeasurementInfoProviderTest
> -
>
> Key: FLINK-23618
> URL: https://issues.apache.org/jira/browse/FLINK-23618
> Project: Flink
> Issue Type: Technical Debt
> Components: Runtime / Metrics
>Affects Versions: 1.12.6, 1.13.3
>Reporter: Dawid Wysakowicz
>Assignee: Chesnay Schepler
>Priority: Blocker
> Fix For: 1.12.6, 1.13.3
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=21471=logs=52b61abe-a3cc-5bde-cc35-1bbe89bb7df5=54421a62-0c80-5aad-3319-094ff69180bb=13881
> {code}
> [ERROR] COMPILATION ERROR :
> [INFO] -
> [ERROR]
> /__w/2/s/flink-metrics/flink-metrics-influxdb/src/test/java/org/apache/flink/metrics/influxdb/MeasurementInfoProviderTest.java:[75,15]
> cannot find symbol
> symbol: class MetricGroup
> location: class
> org.apache.flink.metrics.influxdb.MeasurementInfoProviderTest
> [ERROR]
> /__w/2/s/flink-metrics/flink-metrics-influxdb/src/test/java/org/apache/flink/metrics/influxdb/MeasurementInfoProviderTest.java:[76,17]
> cannot find symbol
> symbol: variable TestMetricGroup
> location: class
> org.apache.flink.metrics.influxdb.MeasurementInfoProviderTest
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-24213) Deadlock in QueryableState Client
[
https://issues.apache.org/jira/browse/FLINK-24213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler updated FLINK-24213:
-
Summary: Deadlock in QueryableState Client (was: Java deadlock in
QueryableState Client)
> Deadlock in QueryableState Client
> -
>
> Key: FLINK-24213
> URL: https://issues.apache.org/jira/browse/FLINK-24213
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Queryable State
>Affects Versions: 1.14.0, 1.12.6, 1.13.3
>Reporter: Dawid Wysakowicz
>Assignee: Chesnay Schepler
>Priority: Major
> Labels: pull-request-available, test-stability
> Fix For: 1.14.0, 1.12.6, 1.13.3
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=23750=logs=d44f43ce-542c-597d-bf94-b0718c71e5e8=ed165f3f-d0f6-524b-5279-86f8ee7d0e2d=15476
> {code}
> Found one Java-level deadlock:
> Sep 08 11:12:50 =
> Sep 08 11:12:50 "Flink Test Client Event Loop Thread 0":
> Sep 08 11:12:50 waiting to lock monitor 0x7f4e380309c8 (object
> 0x86b2cd50, a java.lang.Object),
> Sep 08 11:12:50 which is held by "main"
> Sep 08 11:12:50 "main":
> Sep 08 11:12:50 waiting to lock monitor 0x7f4ea4004068 (object
> 0x86b2cf50, a java.lang.Object),
> Sep 08 11:12:50 which is held by "Flink Test Client Event Loop Thread 0"
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-24213) Java deadlock in QueryableState Client
[
https://issues.apache.org/jira/browse/FLINK-24213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler updated FLINK-24213:
-
Summary: Java deadlock in QueryableState Client (was: Java deadlock in
QueryableState ClientTest)
> Java deadlock in QueryableState Client
> --
>
> Key: FLINK-24213
> URL: https://issues.apache.org/jira/browse/FLINK-24213
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Queryable State
>Affects Versions: 1.14.0, 1.12.6, 1.13.3
>Reporter: Dawid Wysakowicz
>Assignee: Chesnay Schepler
>Priority: Major
> Labels: pull-request-available, test-stability
> Fix For: 1.14.0, 1.12.6, 1.13.3
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=23750=logs=d44f43ce-542c-597d-bf94-b0718c71e5e8=ed165f3f-d0f6-524b-5279-86f8ee7d0e2d=15476
> {code}
> Found one Java-level deadlock:
> Sep 08 11:12:50 =
> Sep 08 11:12:50 "Flink Test Client Event Loop Thread 0":
> Sep 08 11:12:50 waiting to lock monitor 0x7f4e380309c8 (object
> 0x86b2cd50, a java.lang.Object),
> Sep 08 11:12:50 which is held by "main"
> Sep 08 11:12:50 "main":
> Sep 08 11:12:50 waiting to lock monitor 0x7f4ea4004068 (object
> 0x86b2cf50, a java.lang.Object),
> Sep 08 11:12:50 which is held by "Flink Test Client Event Loop Thread 0"
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-23954) Queryable state (rocksdb) with TM restart end-to-end test fails on azure
[
https://issues.apache.org/jira/browse/FLINK-23954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler updated FLINK-23954:
-
Issue Type: Technical Debt (was: Bug)
> Queryable state (rocksdb) with TM restart end-to-end test fails on azure
>
>
> Key: FLINK-23954
> URL: https://issues.apache.org/jira/browse/FLINK-23954
> Project: Flink
> Issue Type: Technical Debt
> Components: Runtime / Queryable State
>Affects Versions: 1.14.0
>Reporter: Xintong Song
>Assignee: Till Rohrmann
>Priority: Critical
> Labels: pull-request-available, test-stability
> Fix For: 1.14.0, 1.12.6, 1.13.3
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=22714=logs=c88eea3b-64a0-564d-0031-9fdcd7b8abee=070ff179-953e-5bda-71fa-d6599415701c=11090
> {code}
> Aug 24 09:54:17
> ==
> Aug 24 09:54:17 Running 'Queryable state (rocksdb) with TM restart end-to-end
> test'
> Aug 24 09:54:17
> ==
> Aug 24 09:54:17 TEST_DATA_DIR:
> /home/vsts/work/1/s/flink-end-to-end-tests/test-scripts/temp-test-directory-17362481511
> Aug 24 09:54:17 Flink dist directory:
> /home/vsts/work/1/s/flink-dist/target/flink-1.14-SNAPSHOT-bin/flink-1.14-SNAPSHOT
> Aug 24 09:54:17 Adding flink-queryable-state-runtime to lib/
> Aug 24 09:54:17 Starting cluster.
> Aug 24 09:54:18 Starting standalonesession daemon on host fv-az123-794.
> Aug 24 09:54:19 Starting taskexecutor daemon on host fv-az123-794.
> Aug 24 09:54:19 Waiting for Dispatcher REST endpoint to come up...
> Aug 24 09:54:20 Waiting for Dispatcher REST endpoint to come up...
> Aug 24 09:54:21 Waiting for Dispatcher REST endpoint to come up...
> Aug 24 09:54:22 Waiting for Dispatcher REST endpoint to come up...
> Aug 24 09:54:23 Waiting for Dispatcher REST endpoint to come up...
> Aug 24 09:54:24 Dispatcher REST endpoint is up.
> Aug 24 09:54:31 Job (f230364470a082b78dc34147f8da38f1) is running.
> Aug 24 09:54:31 Starting to wait for completion of 10 checkpoints
> Aug 24 09:54:31 2/10 completed checkpoints
> Aug 24 09:54:33 6/10 completed checkpoints
> Aug 24 09:54:35 6/10 completed checkpoints
> Aug 24 09:54:37 SERVER: 127.0.0.1
> Aug 24 09:54:37 PORT: 9069
> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> SLF4J: Defaulting to no-operation (NOP) logger implementation
> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
> details.
> Aug 24 09:54:38 MapState has 17 entries
> Aug 24 09:54:38 TaskManager 412715 killed.
> Aug 24 09:54:38 Number of running task managers 1 is not yet 0.
> Aug 24 09:54:42 Number of running task managers has reached 0.
> Aug 24 09:54:42 Latest snapshot count was 22
> Aug 24 09:54:43 Starting taskexecutor daemon on host fv-az123-794.
> Aug 24 09:54:43 Number of running task managers 0 is not yet 1.
> Aug 24 09:54:47 Number of running task managers has reached 1.
> Aug 24 09:54:49 Job (f230364470a082b78dc34147f8da38f1) is running.
> Aug 24 09:54:49 Starting to wait for completion of 16 checkpoints
> Aug 24 09:54:49 11/16 completed checkpoints
> Aug 24 09:54:51 11/16 completed checkpoints
> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> SLF4J: Defaulting to no-operation (NOP) logger implementation
> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
> details.
> Aug 24 09:54:54 after: 21
> Aug 24 09:54:54 An error occurred
> Aug 24 09:54:54 [FAIL] Test script contains errors.
> Aug 24 09:54:54 Checking of logs skipped.
> Aug 24 09:54:54
> Aug 24 09:54:54 [FAIL] 'Queryable state (rocksdb) with TM restart end-to-end
> test' failed after 0 minutes and 37 seconds! Test exited with exit code 1
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-24036) SSL cannot be installed on CI
[
https://issues.apache.org/jira/browse/FLINK-24036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler updated FLINK-24036:
-
Issue Type: Technical Debt (was: Bug)
> SSL cannot be installed on CI
> -
>
> Key: FLINK-24036
> URL: https://issues.apache.org/jira/browse/FLINK-24036
> Project: Flink
> Issue Type: Technical Debt
> Components: Build System / CI
>Affects Versions: 1.14.0, 1.12.5, 1.13.2, 1.15.0
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Blocker
> Fix For: 1.14.0, 1.12.6, 1.13.3
>
>
> {code}
> # install libssl1.0.0 for netty tcnative
> wget
> http://security.ubuntu.com/ubuntu/pool/main/o/openssl1.0/libssl1.0.0_1.0.2n-1ubuntu5.6_amd64.deb
> sudo apt install ./libssl1.0.0_1.0.2n-1ubuntu5.6_amd64.deb
> {code}
> {code}
> --2021-08-27 20:48:49--
> http://security.ubuntu.com/ubuntu/pool/main/o/openssl1.0/libssl1.0.0_1.0.2n-1ubuntu5.6_amd64.deb
> Resolving security.ubuntu.com (security.ubuntu.com)... 91.189.91.39,
> 91.189.91.38, 2001:67c:1562::15, ...
> Connecting to security.ubuntu.com (security.ubuntu.com)|91.189.91.39|:80...
> connected.
> HTTP request sent, awaiting response... 404 Not Found
> 2021-08-27 20:48:49 ERROR 404: Not Found.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-23965) E2E do not execute locally on MacOS
[
https://issues.apache.org/jira/browse/FLINK-23965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler updated FLINK-23965:
-
Issue Type: Technical Debt (was: Bug)
> E2E do not execute locally on MacOS
> ---
>
> Key: FLINK-23965
> URL: https://issues.apache.org/jira/browse/FLINK-23965
> Project: Flink
> Issue Type: Technical Debt
> Components: Tests
>Affects Versions: 1.14.0, 1.13.2
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.14.0, 1.13.3
>
>
> After FLINK-21346, the e2e tests are no longer executing locally on MacOS.
> The problem seems to be that the e2e configure a log directory that does not
> exist and this fails starting a Flink cluster.
> I suggest to change the directory to the old directory {{FLINK_DIR/log}}
> instead of {{FLINK_DIR/logs}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-23336) Use the same log4j version as in Flink 1.13
[ https://issues.apache.org/jira/browse/FLINK-23336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chesnay Schepler updated FLINK-23336: - Fix Version/s: (was: 1.14.0) > Use the same log4j version as in Flink 1.13 > --- > > Key: FLINK-23336 > URL: https://issues.apache.org/jira/browse/FLINK-23336 > Project: Flink > Issue Type: Improvement > Components: Documentation / Training / Exercises >Reporter: Nico Kruber >Assignee: Nico Kruber >Priority: Major > Labels: pull-request-available > Fix For: 1.13.3 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-22382) ProcessFailureCancelingITCase.testCancelingOnProcessFailure
[
https://issues.apache.org/jira/browse/FLINK-22382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler updated FLINK-22382:
-
Issue Type: Technical Debt (was: Bug)
> ProcessFailureCancelingITCase.testCancelingOnProcessFailure
> ---
>
> Key: FLINK-22382
> URL: https://issues.apache.org/jira/browse/FLINK-22382
> Project: Flink
> Issue Type: Technical Debt
> Components: Runtime / Coordination
>Affects Versions: 1.13.0, 1.14.0
>Reporter: Dawid Wysakowicz
>Assignee: Till Rohrmann
>Priority: Critical
> Labels: pull-request-available, test-stability
> Fix For: 1.14.0, 1.12.6, 1.13.3
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16896=logs=a57e0635-3fad-5b08-57c7-a4142d7d6fa9=5360d54c-8d94-5d85-304e-a89267eb785a=9756
> {code}
> Apr 20 18:05:14 Suppressed: java.util.concurrent.TimeoutException
> Apr 20 18:05:14 at
> org.apache.flink.core.testutils.CommonTestUtils.waitUtil(CommonTestUtils.java:210)
> Apr 20 18:05:14 at
> org.apache.flink.test.recovery.ProcessFailureCancelingITCase.waitUntilAtLeastOneTaskHasBeenDeployed(ProcessFailureCancelingITCase.java:236)
> Apr 20 18:05:14 at
> org.apache.flink.test.recovery.ProcessFailureCancelingITCase.testCancelingOnProcessFailure(ProcessFailureCancelingITCase.java:193)
> Apr 20 18:05:14 at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> Apr 20 18:05:14 at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> Apr 20 18:05:14 at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Apr 20 18:05:14 at
> java.lang.reflect.Method.invoke(Method.java:498)
> Apr 20 18:05:14 at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> Apr 20 18:05:14 at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> Apr 20 18:05:14 at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> Apr 20 18:05:14 at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> Apr 20 18:05:14 at
> org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
> Apr 20 18:05:14 at
> org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
> Apr 20 18:05:14 at
> org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
> Apr 20 18:05:14 at
> org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45)
> Apr 20 18:05:14 at
> org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> Apr 20 18:05:14 at
> org.junit.rules.RunRules.evaluate(RunRules.java:20)
> Apr 20 18:05:14 at
> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> Apr 20 18:05:14 at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> Apr 20 18:05:14 at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> Apr 20 18:05:14 at
> org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> Apr 20 18:05:14 at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> Apr 20 18:05:14 at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> Apr 20 18:05:14 at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> Apr 20 18:05:14 at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> Apr 20 18:05:14 at
> org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> Apr 20 18:05:14 at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> Apr 20 18:05:14 at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> Apr 20 18:05:14 at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> Apr 20 18:05:14 at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> Apr 20 18:05:14 at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
> Apr 20 18:05:14 at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
> Apr 20 18:05:14 at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
> Apr 20 18:05:14 at
>
[jira] [Updated] (FLINK-22367) syncSavepointId should only be reset if it is equal to the checkpoint id
[
https://issues.apache.org/jira/browse/FLINK-22367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler updated FLINK-22367:
-
Summary: syncSavepointId should only be reset if it is equal to the
checkpoint id (was:
JobMasterStopWithSavepointITCase.terminateWithSavepointWithoutComplicationsShouldSucceedAndLeadJobToFinished
times out)
> syncSavepointId should only be reset if it is equal to the checkpoint id
>
>
> Key: FLINK-22367
> URL: https://issues.apache.org/jira/browse/FLINK-22367
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing
>Affects Versions: 1.13.0
>Reporter: Dawid Wysakowicz
>Assignee: Anton Kalashnikov
>Priority: Critical
> Labels: auto-deprioritized-critical, auto-unassigned,
> pull-request-available, test-stability
> Fix For: 1.12.6, 1.13.3
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16818=logs=2c3cbe13-dee0-5837-cf47-3053da9a8a78=2c7d57b9-7341-5a87-c9af-2cf7cc1a37dc=3844
> {code}
> [ERROR] Tests run: 7, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> 13.135 s <<< FAILURE! - in
> org.apache.flink.runtime.jobmaster.JobMasterStopWithSavepointITCase
> Apr 19 22:28:44 [ERROR]
> terminateWithSavepointWithoutComplicationsShouldSucceedAndLeadJobToFinished(org.apache.flink.runtime.jobmaster.JobMasterStopWithSavepointITCase)
> Time elapsed: 10.237 s <<< ERROR!
> Apr 19 22:28:44 java.util.concurrent.ExecutionException:
> java.util.concurrent.TimeoutException: Invocation of public default
> java.util.concurrent.CompletableFuture
> org.apache.flink.runtime.webmonitor.RestfulGateway.stopWithSavepoint(org.apache.flink.api.common.JobID,java.lang.String,boolean,org.apache.flink.api.common.time.Time)
> timed out.
> Apr 19 22:28:44 at
> java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395)
> Apr 19 22:28:44 at
> java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999)
> Apr 19 22:28:44 at
> org.apache.flink.runtime.jobmaster.JobMasterStopWithSavepointITCase.stopWithSavepointNormalExecutionHelper(JobMasterStopWithSavepointITCase.java:123)
> Apr 19 22:28:44 at
> org.apache.flink.runtime.jobmaster.JobMasterStopWithSavepointITCase.terminateWithSavepointWithoutComplicationsShouldSucceedAndLeadJobToFinished(JobMasterStopWithSavepointITCase.java:111)
> Apr 19 22:28:44 at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> Apr 19 22:28:44 at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> Apr 19 22:28:44 at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Apr 19 22:28:44 at
> java.base/java.lang.reflect.Method.invoke(Method.java:566)
> Apr 19 22:28:44 at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> Apr 19 22:28:44 at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> Apr 19 22:28:44 at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> Apr 19 22:28:44 at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> Apr 19 22:28:44 at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> Apr 19 22:28:44 at
> org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
> Apr 19 22:28:44 at
> org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45)
> Apr 19 22:28:44 at
> org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> Apr 19 22:28:44 at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> Apr 19 22:28:44 at
> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> Apr 19 22:28:44 at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> Apr 19 22:28:44 at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> Apr 19 22:28:44 at
> org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> Apr 19 22:28:44 at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> Apr 19 22:28:44 at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> Apr 19 22:28:44 at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> Apr 19 22:28:44 at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> Apr 19 22:28:44 at
> org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
> Apr 19 22:28:44 at
>
[jira] [Updated] (FLINK-22333) Elasticsearch7DynamicSinkITCase.testWritingDocuments failed due to deploy task timeout.
[
https://issues.apache.org/jira/browse/FLINK-22333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler updated FLINK-22333:
-
Issue Type: Technical Debt (was: Bug)
> Elasticsearch7DynamicSinkITCase.testWritingDocuments failed due to deploy
> task timeout.
> ---
>
> Key: FLINK-22333
> URL: https://issues.apache.org/jira/browse/FLINK-22333
> Project: Flink
> Issue Type: Technical Debt
> Components: Runtime / Coordination
>Affects Versions: 1.13.0
>Reporter: Guowei Ma
>Assignee: Till Rohrmann
>Priority: Major
> Labels: pull-request-available, test-stability
> Fix For: 1.14.0, 1.12.6, 1.13.3
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16694=logs=d44f43ce-542c-597d-bf94-b0718c71e5e8=03dca39c-73e8-5aaf-601d-328ae5c35f20=12329
> {code:java}
> 2021-04-16T23:37:23.5719280Z Apr 16 23:37:23
> org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
> 2021-04-16T23:37:23.5739250Z Apr 16 23:37:23 at
> org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:144)
> 2021-04-16T23:37:23.5759329Z Apr 16 23:37:23 at
> org.apache.flink.runtime.minicluster.MiniClusterJobClient.lambda$getJobExecutionResult$3(MiniClusterJobClient.java:137)
> 2021-04-16T23:37:23.5779145Z Apr 16 23:37:23 at
> java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616)
> 2021-04-16T23:37:23.5799204Z Apr 16 23:37:23 at
> java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591)
> 2021-04-16T23:37:23.5819302Z Apr 16 23:37:23 at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> 2021-04-16T23:37:23.5839106Z Apr 16 23:37:23 at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
> 2021-04-16T23:37:23.5859276Z Apr 16 23:37:23 at
> org.apache.flink.runtime.rpc.akka.AkkaInvocationHandler.lambda$invokeRpc$0(AkkaInvocationHandler.java:237)
> 2021-04-16T23:37:23.5868964Z Apr 16 23:37:23 at
> java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
> 2021-04-16T23:37:23.5869925Z Apr 16 23:37:23 at
> java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
> 2021-04-16T23:37:23.5919839Z Apr 16 23:37:23 at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> 2021-04-16T23:37:23.5959562Z Apr 16 23:37:23 at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
> 2021-04-16T23:37:23.5989732Z Apr 16 23:37:23 at
> org.apache.flink.runtime.concurrent.FutureUtils$1.onComplete(FutureUtils.java:1081)
> 2021-04-16T23:37:23.6019422Z Apr 16 23:37:23 at
> akka.dispatch.OnComplete.internal(Future.scala:264)
> 2021-04-16T23:37:23.6039067Z Apr 16 23:37:23 at
> akka.dispatch.OnComplete.internal(Future.scala:261)
> 2021-04-16T23:37:23.6060126Z Apr 16 23:37:23 at
> akka.dispatch.japi$CallbackBridge.apply(Future.scala:191)
> 2021-04-16T23:37:23.6089258Z Apr 16 23:37:23 at
> akka.dispatch.japi$CallbackBridge.apply(Future.scala:188)
> 2021-04-16T23:37:23.6119150Z Apr 16 23:37:23 at
> scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
> 2021-04-16T23:37:23.6139149Z Apr 16 23:37:23 at
> org.apache.flink.runtime.concurrent.Executors$DirectExecutionContext.execute(Executors.java:73)
> 2021-04-16T23:37:23.6159077Z Apr 16 23:37:23 at
> scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)
> 2021-04-16T23:37:23.6189432Z Apr 16 23:37:23 at
> scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)
> 2021-04-16T23:37:23.6215243Z Apr 16 23:37:23 at
> akka.pattern.PromiseActorRef.$bang(AskSupport.scala:572)
> 2021-04-16T23:37:23.6219148Z Apr 16 23:37:23 at
> akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:22)
> 2021-04-16T23:37:23.6220221Z Apr 16 23:37:23 at
> akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:21)
> 2021-04-16T23:37:23.6249411Z Apr 16 23:37:23 at
> scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:436)
> 2021-04-16T23:37:23.6259145Z Apr 16 23:37:23 at
> scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:435)
> 2021-04-16T23:37:23.6289272Z Apr 16 23:37:23 at
> scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
> 2021-04-16T23:37:23.6309243Z Apr 16 23:37:23 at
> akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
> 2021-04-16T23:37:23.6359306Z Apr 16 23:37:23 at
> akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:91)
> 2021-04-16T23:37:23.6369399Z Apr 16 23:37:23 at
>
[jira] [Updated] (FLINK-22198) KafkaTableITCase hang.
[ https://issues.apache.org/jira/browse/FLINK-22198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chesnay Schepler updated FLINK-22198: - Issue Type: Technical Debt (was: Bug) > KafkaTableITCase hang. > -- > > Key: FLINK-22198 > URL: https://issues.apache.org/jira/browse/FLINK-22198 > Project: Flink > Issue Type: Technical Debt > Components: Connectors / Kafka >Affects Versions: 1.14.0, 1.12.4 >Reporter: Guowei Ma >Assignee: Qingsheng Ren >Priority: Critical > Labels: pull-request-available, stale-blocker, test-stability > Fix For: 1.14.0, 1.12.6, 1.13.3 > > > https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16287=logs=c5f0071e-1851-543e-9a45-9ac140befc32=1fb1a56f-e8b5-5a82-00a0-a2db7757b4f5=6625 > There is no any artifacts. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-22312) YARNSessionFIFOSecuredITCase>YARNSessionFIFOITCase.checkForProhibitedLogContents due to the heartbeat exception with Yarn RM
[
https://issues.apache.org/jira/browse/FLINK-22312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler updated FLINK-22312:
-
Issue Type: Technical Debt (was: Bug)
> YARNSessionFIFOSecuredITCase>YARNSessionFIFOITCase.checkForProhibitedLogContents
> due to the heartbeat exception with Yarn RM
>
>
> Key: FLINK-22312
> URL: https://issues.apache.org/jira/browse/FLINK-22312
> Project: Flink
> Issue Type: Technical Debt
> Components: Deployment / YARN
>Affects Versions: 1.14.0, 1.13.1, 1.12.4
>Reporter: Guowei Ma
>Assignee: Xintong Song
>Priority: Major
> Labels: auto-deprioritized-critical, pull-request-available,
> test-stability
> Fix For: 1.14.0, 1.12.5, 1.13.3
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16633=logs=fc5181b0-e452-5c8f-68de-1097947f6483=6b04ca5f-0b52-511d-19c9-52bf0d9fbdfa=26614
> {code:java}
> 2021-04-15T22:11:39.5648550Z java.io.InterruptedIOException: Call interrupted
> 2021-04-15T22:11:39.5649145Z at
> org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1483)
> ~[hadoop-common-2.8.3.jar:?]
> 2021-04-15T22:11:39.5649823Z at
> org.apache.hadoop.ipc.Client.call(Client.java:1435)
> ~[hadoop-common-2.8.3.jar:?]
> 2021-04-15T22:11:39.5650488Z at
> org.apache.hadoop.ipc.Client.call(Client.java:1345)
> ~[hadoop-common-2.8.3.jar:?]
> 2021-04-15T22:11:39.5651387Z at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227)
> ~[hadoop-common-2.8.3.jar:?]
> 2021-04-15T22:11:39.5652193Z at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> ~[hadoop-common-2.8.3.jar:?]
> 2021-04-15T22:11:39.5652675Z at com.sun.proxy.$Proxy32.allocate(Unknown
> Source) ~[?:?]
> 2021-04-15T22:11:39.5653478Z at
> org.apache.hadoop.yarn.api.impl.pb.client.ApplicationMasterProtocolPBClientImpl.allocate(ApplicationMasterProtocolPBClientImpl.java:77)
> ~[hadoop-yarn-common-2.8.3.jar:?]
> 2021-04-15T22:11:39.5654223Z at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_275]
> 2021-04-15T22:11:39.5654742Z at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_275]
> 2021-04-15T22:11:39.5655269Z at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_275]
> 2021-04-15T22:11:39.5655625Z ]
> 2021-04-15T22:11:39.5655853Z at org.junit.Assert.fail(Assert.java:88)
> 2021-04-15T22:11:39.5656281Z at
> org.apache.flink.yarn.YarnTestBase.ensureNoProhibitedStringInLogFiles(YarnTestBase.java:576)
> 2021-04-15T22:11:39.5656831Z at
> org.apache.flink.yarn.YARNSessionFIFOITCase.checkForProhibitedLogContents(YARNSessionFIFOITCase.java:86)
> 2021-04-15T22:11:39.5657360Z at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 2021-04-15T22:11:39.565Z at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 2021-04-15T22:11:39.5658252Z at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 2021-04-15T22:11:39.5658723Z at
> java.lang.reflect.Method.invoke(Method.java:498)
> 2021-04-15T22:11:39.5659311Z at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> 2021-04-15T22:11:39.5659780Z at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 2021-04-15T22:11:39.5660248Z at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> 2021-04-15T22:11:39.5660829Z at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
> 2021-04-15T22:11:39.5661247Z at
> org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
> 2021-04-15T22:11:39.5661652Z at
> org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> 2021-04-15T22:11:39.5662006Z at
> org.junit.rules.RunRules.evaluate(RunRules.java:20)
> 2021-04-15T22:11:39.5662379Z at
> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> 2021-04-15T22:11:39.5662812Z at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> 2021-04-15T22:11:39.5663260Z at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> 2021-04-15T22:11:39.5663935Z at
> org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> 2021-04-15T22:11:39.5664384Z at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> 2021-04-15T22:11:39.5664849Z at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> 2021-04-15T22:11:39.5665283Z at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>
[jira] [Updated] (FLINK-22002) AggregateReduceGroupingITCase.testSingleAggOnTable_HashAgg_WithLocalAgg fail because of submitting task time-out.
[
https://issues.apache.org/jira/browse/FLINK-22002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler updated FLINK-22002:
-
Issue Type: Improvement (was: Bug)
> AggregateReduceGroupingITCase.testSingleAggOnTable_HashAgg_WithLocalAgg fail
> because of submitting task time-out.
> -
>
> Key: FLINK-22002
> URL: https://issues.apache.org/jira/browse/FLINK-22002
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.14.0, 1.13.2
>Reporter: Guowei Ma
>Assignee: Till Rohrmann
>Priority: Major
> Labels: pull-request-available, test-stability
> Fix For: 1.14.0, 1.13.3
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=15634=logs=955770d3-1fed-5a0a-3db6-0c7554c910cb=14447d61-56b4-5000-80c1-daa459247f6a=6424
> {code:java}
> org.apache.flink.table.planner.runtime.batch.sql.agg.AggregateReduceGroupingITCase
> 2021-03-29T00:27:25.3406344Z [ERROR]
> testSingleAggOnTable_HashAgg_WithLocalAgg(org.apache.flink.table.planner.runtime.batch.sql.agg.AggregateReduceGroupingITCase)
> Time elapsed: 21.908 s <<< ERROR!
> 2021-03-29T00:27:25.3407190Z java.lang.RuntimeException: Failed to fetch next
> result
> 2021-03-29T00:27:25.3407792Z at
> org.apache.flink.streaming.api.operators.collect.CollectResultIterator.nextResultFromFetcher(CollectResultIterator.java:109)
> 2021-03-29T00:27:25.3408502Z at
> org.apache.flink.streaming.api.operators.collect.CollectResultIterator.hasNext(CollectResultIterator.java:80)
> 2021-03-29T00:27:25.3409188Z at
> org.apache.flink.table.planner.sinks.SelectTableSinkBase$RowIteratorWrapper.hasNext(SelectTableSinkBase.java:117)
> 2021-03-29T00:27:25.3416724Z at
> org.apache.flink.table.api.internal.TableResultImpl$CloseableRowIteratorWrapper.hasNext(TableResultImpl.java:350)
> 2021-03-29T00:27:25.3417510Z at
> java.util.Iterator.forEachRemaining(Iterator.java:115)
> 2021-03-29T00:27:25.3418416Z at
> org.apache.flink.util.CollectionUtil.iteratorToList(CollectionUtil.java:108)
> 2021-03-29T00:27:25.3419031Z at
> org.apache.flink.table.planner.runtime.utils.BatchTestBase.executeQuery(BatchTestBase.scala:298)
> 2021-03-29T00:27:25.3419657Z at
> org.apache.flink.table.planner.runtime.utils.BatchTestBase.check(BatchTestBase.scala:138)
> 2021-03-29T00:27:25.3420638Z at
> org.apache.flink.table.planner.runtime.utils.BatchTestBase.checkResult(BatchTestBase.scala:104)
> 2021-03-29T00:27:25.3421384Z at
> org.apache.flink.table.planner.runtime.batch.sql.agg.AggregateReduceGroupingITCase.testSingleAggOnTable(AggregateReduceGroupingITCase.scala:182)
> 2021-03-29T00:27:25.3422284Z at
> org.apache.flink.table.planner.runtime.batch.sql.agg.AggregateReduceGroupingITCase.testSingleAggOnTable_HashAgg_WithLocalAgg(AggregateReduceGroupingITCase.scala:135)
> 2021-03-29T00:27:25.3422975Z at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 2021-03-29T00:27:25.3423504Z at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 2021-03-29T00:27:25.3424298Z at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 2021-03-29T00:27:25.3425229Z at
> java.lang.reflect.Method.invoke(Method.java:498)
> 2021-03-29T00:27:25.3426107Z at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> 2021-03-29T00:27:25.3426756Z at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 2021-03-29T00:27:25.3427743Z at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> 2021-03-29T00:27:25.3428520Z at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> 2021-03-29T00:27:25.3429128Z at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 2021-03-29T00:27:25.3429715Z at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> 2021-03-29T00:27:25.3433435Z at
> org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> 2021-03-29T00:27:25.3433977Z at
> org.junit.rules.RunRules.evaluate(RunRules.java:20)
> 2021-03-29T00:27:25.3434476Z at
> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> 2021-03-29T00:27:25.3435607Z at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> 2021-03-29T00:27:25.3436460Z at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> 2021-03-29T00:27:25.3437054Z at
> org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> 2021-03-29T00:27:25.3437673Z at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>
[jira] [Updated] (FLINK-22002) taskmanager.slot.timeout should fall back to akka.ask.timeout
[
https://issues.apache.org/jira/browse/FLINK-22002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler updated FLINK-22002:
-
Summary: taskmanager.slot.timeout should fall back to akka.ask.timeout
(was: AggregateReduceGroupingITCase.testSingleAggOnTable_HashAgg_WithLocalAgg
fail because of submitting task time-out.)
> taskmanager.slot.timeout should fall back to akka.ask.timeout
> -
>
> Key: FLINK-22002
> URL: https://issues.apache.org/jira/browse/FLINK-22002
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.14.0, 1.13.2
>Reporter: Guowei Ma
>Assignee: Till Rohrmann
>Priority: Major
> Labels: pull-request-available, test-stability
> Fix For: 1.14.0, 1.13.3
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=15634=logs=955770d3-1fed-5a0a-3db6-0c7554c910cb=14447d61-56b4-5000-80c1-daa459247f6a=6424
> {code:java}
> org.apache.flink.table.planner.runtime.batch.sql.agg.AggregateReduceGroupingITCase
> 2021-03-29T00:27:25.3406344Z [ERROR]
> testSingleAggOnTable_HashAgg_WithLocalAgg(org.apache.flink.table.planner.runtime.batch.sql.agg.AggregateReduceGroupingITCase)
> Time elapsed: 21.908 s <<< ERROR!
> 2021-03-29T00:27:25.3407190Z java.lang.RuntimeException: Failed to fetch next
> result
> 2021-03-29T00:27:25.3407792Z at
> org.apache.flink.streaming.api.operators.collect.CollectResultIterator.nextResultFromFetcher(CollectResultIterator.java:109)
> 2021-03-29T00:27:25.3408502Z at
> org.apache.flink.streaming.api.operators.collect.CollectResultIterator.hasNext(CollectResultIterator.java:80)
> 2021-03-29T00:27:25.3409188Z at
> org.apache.flink.table.planner.sinks.SelectTableSinkBase$RowIteratorWrapper.hasNext(SelectTableSinkBase.java:117)
> 2021-03-29T00:27:25.3416724Z at
> org.apache.flink.table.api.internal.TableResultImpl$CloseableRowIteratorWrapper.hasNext(TableResultImpl.java:350)
> 2021-03-29T00:27:25.3417510Z at
> java.util.Iterator.forEachRemaining(Iterator.java:115)
> 2021-03-29T00:27:25.3418416Z at
> org.apache.flink.util.CollectionUtil.iteratorToList(CollectionUtil.java:108)
> 2021-03-29T00:27:25.3419031Z at
> org.apache.flink.table.planner.runtime.utils.BatchTestBase.executeQuery(BatchTestBase.scala:298)
> 2021-03-29T00:27:25.3419657Z at
> org.apache.flink.table.planner.runtime.utils.BatchTestBase.check(BatchTestBase.scala:138)
> 2021-03-29T00:27:25.3420638Z at
> org.apache.flink.table.planner.runtime.utils.BatchTestBase.checkResult(BatchTestBase.scala:104)
> 2021-03-29T00:27:25.3421384Z at
> org.apache.flink.table.planner.runtime.batch.sql.agg.AggregateReduceGroupingITCase.testSingleAggOnTable(AggregateReduceGroupingITCase.scala:182)
> 2021-03-29T00:27:25.3422284Z at
> org.apache.flink.table.planner.runtime.batch.sql.agg.AggregateReduceGroupingITCase.testSingleAggOnTable_HashAgg_WithLocalAgg(AggregateReduceGroupingITCase.scala:135)
> 2021-03-29T00:27:25.3422975Z at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 2021-03-29T00:27:25.3423504Z at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 2021-03-29T00:27:25.3424298Z at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 2021-03-29T00:27:25.3425229Z at
> java.lang.reflect.Method.invoke(Method.java:498)
> 2021-03-29T00:27:25.3426107Z at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> 2021-03-29T00:27:25.3426756Z at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 2021-03-29T00:27:25.3427743Z at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> 2021-03-29T00:27:25.3428520Z at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> 2021-03-29T00:27:25.3429128Z at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 2021-03-29T00:27:25.3429715Z at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> 2021-03-29T00:27:25.3433435Z at
> org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> 2021-03-29T00:27:25.3433977Z at
> org.junit.rules.RunRules.evaluate(RunRules.java:20)
> 2021-03-29T00:27:25.3434476Z at
> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> 2021-03-29T00:27:25.3435607Z at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> 2021-03-29T00:27:25.3436460Z at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> 2021-03-29T00:27:25.3437054Z at
> org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> 2021-03-29T00:27:25.3437673Z at
>
[jira] [Updated] (FLINK-21538) Elasticsearch6DynamicSinkITCase.testWritingDocuments fails when submitting job
[
https://issues.apache.org/jira/browse/FLINK-21538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler updated FLINK-21538:
-
Issue Type: Technical Debt (was: Bug)
> Elasticsearch6DynamicSinkITCase.testWritingDocuments fails when submitting job
> --
>
> Key: FLINK-21538
> URL: https://issues.apache.org/jira/browse/FLINK-21538
> Project: Flink
> Issue Type: Technical Debt
> Components: Connectors / ElasticSearch, Runtime / Coordination
>Affects Versions: 1.12.1, 1.13.0
>Reporter: Dawid Wysakowicz
>Assignee: Till Rohrmann
>Priority: Minor
> Labels: auto-deprioritized-major, auto-unassigned,
> pull-request-available, test-stability
> Fix For: 1.14.0, 1.12.6, 1.13.3
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=13868=logs=3d12d40f-c62d-5ec4-6acc-0efe94cc3e89=5d6e4255-0ea8-5e2a-f52c-c881b7872361
> {code}
> 2021-02-27T00:16:06.9493539Z
> org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
> 2021-02-27T00:16:06.9494494Z at
> org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:144)
> 2021-02-27T00:16:06.9495733Z at
> org.apache.flink.runtime.minicluster.MiniClusterJobClient.lambda$getJobExecutionResult$2(MiniClusterJobClient.java:117)
> 2021-02-27T00:16:06.9496596Z at
> java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616)
> 2021-02-27T00:16:06.9497354Z at
> java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591)
> 2021-02-27T00:16:06.9525795Z at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> 2021-02-27T00:16:06.9526744Z at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
> 2021-02-27T00:16:06.9527784Z at
> org.apache.flink.runtime.rpc.akka.AkkaInvocationHandler.lambda$invokeRpc$0(AkkaInvocationHandler.java:237)
> 2021-02-27T00:16:06.9528552Z at
> java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
> 2021-02-27T00:16:06.9529271Z at
> java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
> 2021-02-27T00:16:06.9530013Z at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> 2021-02-27T00:16:06.9530482Z at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
> 2021-02-27T00:16:06.9531068Z at
> org.apache.flink.runtime.concurrent.FutureUtils$1.onComplete(FutureUtils.java:1046)
> 2021-02-27T00:16:06.9531544Z at
> akka.dispatch.OnComplete.internal(Future.scala:264)
> 2021-02-27T00:16:06.9531908Z at
> akka.dispatch.OnComplete.internal(Future.scala:261)
> 2021-02-27T00:16:06.9532449Z at
> akka.dispatch.japi$CallbackBridge.apply(Future.scala:191)
> 2021-02-27T00:16:06.9532860Z at
> akka.dispatch.japi$CallbackBridge.apply(Future.scala:188)
> 2021-02-27T00:16:06.9533245Z at
> scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
> 2021-02-27T00:16:06.9533721Z at
> org.apache.flink.runtime.concurrent.Executors$DirectExecutionContext.execute(Executors.java:73)
> 2021-02-27T00:16:06.9534225Z at
> scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:68)
> 2021-02-27T00:16:06.9534697Z at
> scala.concurrent.impl.Promise$DefaultPromise.$anonfun$tryComplete$1(Promise.scala:284)
> 2021-02-27T00:16:06.9535217Z at
> scala.concurrent.impl.Promise$DefaultPromise.$anonfun$tryComplete$1$adapted(Promise.scala:284)
> 2021-02-27T00:16:06.9535718Z at
> scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:284)
> 2021-02-27T00:16:06.9536127Z at
> akka.pattern.PromiseActorRef.$bang(AskSupport.scala:573)
> 2021-02-27T00:16:06.9536861Z at
> akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:22)
> 2021-02-27T00:16:06.9537394Z at
> akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:21)
> 2021-02-27T00:16:06.9537916Z at
> scala.concurrent.Future.$anonfun$andThen$1(Future.scala:532)
> 2021-02-27T00:16:06.9605804Z at
> scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:29)
> 2021-02-27T00:16:06.9606794Z at
> scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:29)
> 2021-02-27T00:16:06.9607642Z at
> scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
> 2021-02-27T00:16:06.9608419Z at
> akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
> 2021-02-27T00:16:06.9609252Z at
> akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:91)
> 2021-02-27T00:16:06.9610024Z at
> scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
> 2021-02-27T00:16:06.9613676Z at
>
[jira] [Updated] (FLINK-21853) Running HA per-job cluster (rocks, non-incremental) end-to-end test could not finished in 900 seconds
[
https://issues.apache.org/jira/browse/FLINK-21853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler updated FLINK-21853:
-
Issue Type: Technical Debt (was: Bug)
> Running HA per-job cluster (rocks, non-incremental) end-to-end test could not
> finished in 900 seconds
> -
>
> Key: FLINK-21853
> URL: https://issues.apache.org/jira/browse/FLINK-21853
> Project: Flink
> Issue Type: Technical Debt
> Components: Runtime / Coordination, Runtime / State Backends
>Affects Versions: 1.11.3, 1.13.0
>Reporter: Guowei Ma
>Assignee: Till Rohrmann
>Priority: Minor
> Labels: auto-deprioritized-major, pull-request-available,
> test-stability
> Fix For: 1.13.3, 1.15.0, 1.14.1
>
> Attachments: flink-vsts-standalonejob-3-fv-az227-245.log.zip
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=14921=logs=91bf6583-3fb2-592f-e4d4-d79d79c3230a=03dbd840-5430-533d-d1a7-05d0ebe03873=7318
> {code:java}
> Waiting for text Completed checkpoint [1-9]* for job
> to appear 2 of times in logs...
> grep:
> /home/vsts/work/1/s/flink-dist/target/flink-1.11-SNAPSHOT-bin/flink-1.11-SNAPSHOT/log/*standalonejob-2*.log:
> No such file or directory
> grep:
> /home/vsts/work/1/s/flink-dist/target/flink-1.11-SNAPSHOT-bin/flink-1.11-SNAPSHOT/log/*standalonejob-2*.log:
> No such file or directory
> grep:
> /home/vsts/work/1/s/flink-dist/target/flink-1.11-SNAPSHOT-bin/flink-1.11-SNAPSHOT/log/*standalonejob-2*.log:
> No such file or directory
> Starting standalonejob daemon on host fv-az232-135.
> grep:
> /home/vsts/work/1/s/flink-dist/target/flink-1.11-SNAPSHOT-bin/flink-1.11-SNAPSHOT/log/*standalonejob-2*.log:
> No such file or directory
> Killed TM @ 15744
> Killed TM @ 19625
> Test (pid: 9232) did not finish after 900 seconds.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-20461) YARNFileReplicationITCase.testPerJobModeWithDefaultFileReplication
[
https://issues.apache.org/jira/browse/FLINK-20461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler updated FLINK-20461:
-
Issue Type: Technical Debt (was: Bug)
> YARNFileReplicationITCase.testPerJobModeWithDefaultFileReplication
> --
>
> Key: FLINK-20461
> URL: https://issues.apache.org/jira/browse/FLINK-20461
> Project: Flink
> Issue Type: Technical Debt
> Components: Deployment / YARN
>Affects Versions: 1.11.3, 1.12.0, 1.13.0, 1.14.0
>Reporter: Huang Xingbo
>Assignee: Till Rohrmann
>Priority: Critical
> Labels: pull-request-available, test-stability
> Fix For: 1.14.0, 1.12.6, 1.13.3
>
>
> [https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=10450=logs=fc5181b0-e452-5c8f-68de-1097947f6483=62110053-334f-5295-a0ab-80dd7e2babbf]
> {code:java}
> [ERROR]
> testPerJobModeWithDefaultFileReplication(org.apache.flink.yarn.YARNFileReplicationITCase)
> Time elapsed: 32.501 s <<< ERROR! java.io.FileNotFoundException: File does
> not exist:
> hdfs://localhost:46072/user/agent04_azpcontainer/.flink/application_1606950278664_0001/flink-dist_2.11-1.12-SNAPSHOT.jar
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1441)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1434)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1434)
> at
> org.apache.flink.yarn.YARNFileReplicationITCase.extraVerification(YARNFileReplicationITCase.java:148)
> at
> org.apache.flink.yarn.YARNFileReplicationITCase.deployPerJob(YARNFileReplicationITCase.java:113)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-21116) DefaultDispatcherRunnerITCase hangs on azure
[
https://issues.apache.org/jira/browse/FLINK-21116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler updated FLINK-21116:
-
Issue Type: Technical Debt (was: Bug)
> DefaultDispatcherRunnerITCase hangs on azure
>
>
> Key: FLINK-21116
> URL: https://issues.apache.org/jira/browse/FLINK-21116
> Project: Flink
> Issue Type: Technical Debt
> Components: Runtime / Coordination, Tests
>Affects Versions: 1.11.4
>Reporter: Dawid Wysakowicz
>Assignee: David Morávek
>Priority: Minor
> Labels: auto-deprioritized-major, pull-request-available,
> test-stability
> Fix For: 1.14.0, 1.12.6, 1.13.3
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=12430=logs=3b6ec2fd-a816-5e75-c775-06fb87cb6670=b33fdd4f-3de5-542e-2624-5d53167bb672
> {code}
> "main" #1 prio=5 os_prio=0 tid=0x7f35fc00b800 nid=0x710a waiting on
> condition [0x7f360451e000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x86330750> (a
> java.util.concurrent.CompletableFuture$Signaller)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1707)
> at
> java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
> at
> java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1742)
> at
> java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
> at
> org.apache.flink.util.AutoCloseableAsync.close(AutoCloseableAsync.java:36)
> at
> org.apache.flink.runtime.dispatcher.runner.DefaultDispatcherRunnerITCase.leaderChange_withBlockingJobManagerTermination_doesNotAffectNewLeader(DefaultDispatcherRunnerITCase.java:211)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-23405) FlinkKafkaProducerMigrationOperatorTest.testRestoreProducer fails due to BindException
[
https://issues.apache.org/jira/browse/FLINK-23405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler updated FLINK-23405:
-
Fix Version/s: (was: 1.14.1)
(was: 1.15.0)
> FlinkKafkaProducerMigrationOperatorTest.testRestoreProducer fails due to
> BindException
> --
>
> Key: FLINK-23405
> URL: https://issues.apache.org/jira/browse/FLINK-23405
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
>Affects Versions: 1.14.0
>Reporter: Xintong Song
>Priority: Major
> Labels: pull-request-available, test-stability
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=20523=logs=72d4811f-9f0d-5fd0-014a-0bc26b72b642=c1d93a6a-ba91-515d-3196-2ee8019fbda7=6851
> {code}
> Jul 15 21:23:44 [ERROR] Tests run: 4, Failures: 0, Errors: 1, Skipped: 0,
> Time elapsed: 102.561 s <<< FAILURE! - in
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerMigrationOperatorTest
> Jul 15 21:23:44 [ERROR] testRestoreProducer[Migration Savepoint:
> 1.10](org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerMigrationOperatorTest)
> Time elapsed: 2.015 s <<< ERROR!
> Jul 15 21:23:44 java.net.BindException: Address already in use
> Jul 15 21:23:44 at sun.nio.ch.Net.bind0(Native Method)
> Jul 15 21:23:44 at sun.nio.ch.Net.bind(Net.java:461)
> Jul 15 21:23:44 at sun.nio.ch.Net.bind(Net.java:453)
> Jul 15 21:23:44 at
> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:222)
> Jul 15 21:23:44 at
> sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:85)
> Jul 15 21:23:44 at
> sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:78)
> Jul 15 21:23:44 at
> org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:90)
> Jul 15 21:23:44 at
> org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:120)
> Jul 15 21:23:44 at
> org.apache.curator.test.TestingZooKeeperMain.runFromConfig(TestingZooKeeperMain.java:93)
> Jul 15 21:23:44 at
> org.apache.curator.test.TestingZooKeeperServer$1.run(TestingZooKeeperServer.java:148)
> Jul 15 21:23:44 at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-24058) TaskSlotTableImplTest.testMarkSlotActiveDeactivatesSlotTimeout fails on azure
[
https://issues.apache.org/jira/browse/FLINK-24058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler updated FLINK-24058:
-
Fix Version/s: 1.14.1
> TaskSlotTableImplTest.testMarkSlotActiveDeactivatesSlotTimeout fails on azure
> -
>
> Key: FLINK-24058
> URL: https://issues.apache.org/jira/browse/FLINK-24058
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
>Affects Versions: 1.15.0
>Reporter: Xintong Song
>Assignee: Chesnay Schepler
>Priority: Critical
> Labels: pull-request-available, test-stability
> Fix For: 1.15.0, 1.14.1
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=23083=logs=0da23115-68bb-5dcd-192c-bd4c8adebde1=24c3384f-1bcb-57b3-224f-51bf973bbee8=8583
> {code}
> Aug 30 10:55:16 [ERROR] Tests run: 20, Failures: 1, Errors: 0, Skipped: 0,
> Time elapsed: 1.243 s <<< FAILURE! - in
> org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImplTest
> Aug 30 10:55:16 [ERROR] testMarkSlotActiveDeactivatesSlotTimeout Time
> elapsed: 0.334 s <<< FAILURE!
> Aug 30 10:55:16 java.lang.AssertionError: The slot timeout should have been
> deactivated.
> Aug 30 10:55:16 at org.junit.Assert.fail(Assert.java:89)
> Aug 30 10:55:16 at
> org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImplTest.runDeactivateSlotTimeoutTest(TaskSlotTableImplTest.java:509)
> Aug 30 10:55:16 at
> org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImplTest.testMarkSlotActiveDeactivatesSlotTimeout(TaskSlotTableImplTest.java:472)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #17437: [FLINK-24467][streaming] Announce the min and max buffer size despite last diff less than threshold
flinkbot edited a comment on pull request #17437: URL: https://github.com/apache/flink/pull/17437#issuecomment-938641904 ## CI report: * 99519029ca897f2c0e6cc40e5e8b3ff1557c1cc4 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=24962) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17345: [FLINK-24227][connectors] FLIP-171: Added Kinesis Data Streams Sink i…
flinkbot edited a comment on pull request #17345: URL: https://github.com/apache/flink/pull/17345#issuecomment-926109717 ## CI report: * af963d1fa3477f9578fa34398c8b4603bb4631e1 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=24959) * 3e47c0e0650872e5435c03e7b1e0c44da1eb0587 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=24964) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17345: [FLINK-24227][connectors] FLIP-171: Added Kinesis Data Streams Sink i…
flinkbot edited a comment on pull request #17345: URL: https://github.com/apache/flink/pull/17345#issuecomment-926109717 ## CI report: * af963d1fa3477f9578fa34398c8b4603bb4631e1 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=24959) * 3e47c0e0650872e5435c03e7b1e0c44da1eb0587 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] CrynetLogistics commented on pull request #17345: [FLINK-24227][connectors] FLIP-171: Added Kinesis Data Streams Sink i…
CrynetLogistics commented on pull request #17345: URL: https://github.com/apache/flink/pull/17345#issuecomment-940378397 Tests now passing. Will deal with the rest of the comments tomorrow! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-24229) [FLIP-171] DynamoDB implementation of Async Sink
[
https://issues.apache.org/jira/browse/FLINK-24229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17427308#comment-17427308
]
Yuri Gusev commented on FLINK-24229:
Yes, I think it should not be hard to adopt it to the new async sink base.
Thanks [~CrynetLogistics], then will try to do it and come back for
input/reviews.
Do you know when is the deadline for 1.15 release?
Best regards,
Yuri Gusev
> [FLIP-171] DynamoDB implementation of Async Sink
>
>
> Key: FLINK-24229
> URL: https://issues.apache.org/jira/browse/FLINK-24229
> Project: Flink
> Issue Type: New Feature
> Components: Connectors / Common
>Reporter: Zichen Liu
>Assignee: Zichen Liu
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.15.0
>
>
> h2. Motivation
> *User stories:*
> As a Flink user, I’d like to use DynamoDB as sink for my data pipeline.
> *Scope:*
> * Implement an asynchronous sink for DynamoDB by inheriting the
> AsyncSinkBase class. The implementation can for now reside in its own module
> in flink-connectors.
> * Implement an asynchornous sink writer for DynamoDB by extending the
> AsyncSinkWriter. The implementation must deal with failed requests and retry
> them using the {{requeueFailedRequestEntry}} method. If possible, the
> implementation should batch multiple requests (PutRecordsRequestEntry
> objects) to Firehose for increased throughput. The implemented Sink Writer
> will be used by the Sink class that will be created as part of this story.
> * Java / code-level docs.
> * End to end testing: add tests that hits a real AWS instance. (How to best
> donate resources to the Flink project to allow this to happen?)
> h2. References
> More details to be found
> [https://cwiki.apache.org/confluence/display/FLINK/FLIP-171%3A+Async+Sink]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #17451: [FLINK-24476][docs] Rename Elasticsearch to remove CamelCase
flinkbot edited a comment on pull request #17451: URL: https://github.com/apache/flink/pull/17451#issuecomment-940126969 ## CI report: * 55bbb7e7f747599b91a2926e47af4ea299a728e4 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=24960) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17345: [FLINK-24227][connectors] FLIP-171: Added Kinesis Data Streams Sink i…
flinkbot edited a comment on pull request #17345: URL: https://github.com/apache/flink/pull/17345#issuecomment-926109717 ## CI report: * af963d1fa3477f9578fa34398c8b4603bb4631e1 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=24959) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17441: [FLINK-24461][table] PrintUtils common now accepts only internal data types
flinkbot edited a comment on pull request #17441: URL: https://github.com/apache/flink/pull/17441#issuecomment-938905036 ## CI report: * 0040bfe2d73e9b07cf348fd91a6d93149689bebb Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=24958) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17452: [FLINK-20732][connector/pulsar] Introduction of Pulsar Sink
flinkbot edited a comment on pull request #17452: URL: https://github.com/apache/flink/pull/17452#issuecomment-940136217 ## CI report: * 7a74987d989f699bc68c2da8e0cd9bf3fa8065a6 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=24963) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-24471) Add microbenchmark for throughput with debloating enabled
[ https://issues.apache.org/jira/browse/FLINK-24471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-24471: --- Labels: pull-request-available (was: ) > Add microbenchmark for throughput with debloating enabled > - > > Key: FLINK-24471 > URL: https://issues.apache.org/jira/browse/FLINK-24471 > Project: Flink > Issue Type: Sub-task > Components: Benchmarks, Runtime / Network >Reporter: Piotr Nowojski >Assignee: Piotr Nowojski >Priority: Major > Labels: pull-request-available > Fix For: 1.15.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink-benchmarks] pnowojski merged pull request #36: [FLINK-24471] Add microbenchmark for throughput with debloating enabled
pnowojski merged pull request #36: URL: https://github.com/apache/flink-benchmarks/pull/36 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17452: [FLINK-20732][connector/pulsar] Introduction of Pulsar Sink
flinkbot edited a comment on pull request #17452: URL: https://github.com/apache/flink/pull/17452#issuecomment-940136217 ## CI report: * Unknown: [CANCELED](TBD) * 7a74987d989f699bc68c2da8e0cd9bf3fa8065a6 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=24963) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17452: [FLINK-20732][connector/pulsar] Introduction of Pulsar Sink
flinkbot edited a comment on pull request #17452: URL: https://github.com/apache/flink/pull/17452#issuecomment-940136217 ## CI report: * Unknown: [CANCELED](TBD) * 7a74987d989f699bc68c2da8e0cd9bf3fa8065a6 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] akalash commented on pull request #17440: [FLINK-24468][runtime] Wait for the channel activation before creating partition request client
akalash commented on pull request #17440: URL: https://github.com/apache/flink/pull/17440#issuecomment-940200558 > Do we really just stop debloating or do we trigger a failover? As I can see we just stopping debloating and no failover would be triggered > Shouldn't the exception from the mailbox processor fail the task? I was also surprised but it looks like we just ignore this error(it is exactly what should be figured out later). > Why does it happen only for announcing the new buffer size? if we have input data already then the this NPE is impossible, it is exactly what happens when we call other methods except for the new buffer size. So everything except the announcement of the buffer size is a reaction to the input data. > Shouldn't we have the same issue for all other methods of CreditBasedPartitionRequestClientHandler such as e.g. notifyCreditAvailable? As you can see I made this NPE impossible by waiting for channel initialization on the start. So everything should work well now > Is it an error if we continue processing? IMO, at best it should be warn, I am event thinking if it shouldn't be just a debug. What are users supposed to do with the line in logs? I don't really sure that we need this try/catch block at all. My main fix is about waiting for the initialization but these changes I made just in case, but perhaps it is much better to understand the flink behavior when we have exceptions inside of the action.( But in general, I agree that `debug` suits here more) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17437: [FLINK-24467][streaming] Announce the min and max buffer size despite last diff less than threshold
flinkbot edited a comment on pull request #17437: URL: https://github.com/apache/flink/pull/17437#issuecomment-938641904 ## CI report: * fbfc14527fed6e285c2d65460cc317b10e5e2e94 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=24858) * 99519029ca897f2c0e6cc40e5e8b3ff1557c1cc4 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=24962) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17437: [FLINK-24467][streaming] Announce the min and max buffer size despite last diff less than threshold
flinkbot edited a comment on pull request #17437: URL: https://github.com/apache/flink/pull/17437#issuecomment-938641904 ## CI report: * fbfc14527fed6e285c2d65460cc317b10e5e2e94 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=24858) * 99519029ca897f2c0e6cc40e5e8b3ff1557c1cc4 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] akalash commented on a change in pull request #17437: [FLINK-24467][streaming] Announce the min and max buffer size despite last diff less than threshold
akalash commented on a change in pull request #17437:
URL: https://github.com/apache/flink/pull/17437#discussion_r726336587
##
File path:
flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/tasks/bufferdebloat/BufferDebloaterTest.java
##
@@ -105,14 +107,50 @@ public void testNegativeConsumptionTime() {
testBufferSizeCalculation(3, asList(3, 5, 8), , 50, 1100, -1, 248);
}
-private void testBufferSizeCalculation(
+@Test
+public void
testAnnouncedMaxBufferSizeEvenDespiteLastDiffLessThanThreshold() {
+// Calculate the buffer size a little lower than the max buffer size.
+BufferDebloater bufferDebloater =
+testBufferSizeCalculation(1, singletonList(1), 500, 50, 1100,
1000, 500);
+bufferDebloater.recalculateBufferSize(1000);
+assertThat(bufferDebloater.getLastBufferSize(), is(1000));
+
+// Recalculate the buffer size to max value.
+bufferDebloater.recalculateBufferSize(2000);
+
+// The max value should be announced despite it differ from the
previous one by less than
+// threshold value.
+assertThat(bufferDebloater.getLastBufferSize(), is(1100));
+
+// Make sure that there is no repeated announcement of max buffer size.
+bufferDebloater.recalculateBufferSize(2000);
+}
+
+@Test
+public void
testAnnouncedMinBufferSizeEvenDespiteLastDiffLessThanThreshold() {
+// Calculate the buffer size a little greater than the min buffer size.
+BufferDebloater bufferDebloater =
+testBufferSizeCalculation(1, singletonList(1), 60, 50, 1100,
1000, 60);
Review comment:
The initial value is always max value but after this method, it will be
60(because throughput is 60). But you are right and I will add this information
explicitly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (FLINK-24331) PartiallyFinishedSourcesITCase fails with "No downstream received 0 from xxx;"
[
https://issues.apache.org/jira/browse/FLINK-24331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Piotr Nowojski updated FLINK-24331:
---
Component/s: Runtime / Task
> PartiallyFinishedSourcesITCase fails with "No downstream received 0 from xxx;"
> --
>
> Key: FLINK-24331
> URL: https://issues.apache.org/jira/browse/FLINK-24331
> Project: Flink
> Issue Type: Bug
> Components: API / DataStream, Runtime / Task
>Affects Versions: 1.14.0, 1.15.0
>Reporter: Xintong Song
>Priority: Critical
> Labels: test-stability
> Fix For: 1.15.0
>
> Attachments:
> logs-ci_build-test_ci_build_finegrained_resource_management-1633890853.zip
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=24287=logs=4d4a0d10-fca2-5507-8eed-c07f0bdf4887=7b25afdf-cc6c-566f-5459-359dc2585798=10945
> {code}
> Sep 18 02:21:08 [ERROR] Tests run: 12, Failures: 1, Errors: 0, Skipped: 0,
> Time elapsed: 224.44 s <<< FAILURE! - in
> org.apache.flink.runtime.operators.lifecycle.PartiallyFinishedSourcesITCase
> Sep 18 02:21:08 [ERROR] test[complex graph SINGLE_SUBTASK, failover: true,
> strategy: region] Time elapsed: 28.807 s <<< FAILURE!
> Sep 18 02:21:08 java.lang.AssertionError: No downstream received 0 from
> 0003[0]; received: {0=OperatorFinished
> 0007/0, 1=OperatorFinished
> 0007/1, 2=OperatorFinished
> 0007/2, 3=OperatorFinished
> 0007/3}
> Sep 18 02:21:08 at org.junit.Assert.fail(Assert.java:89)
> Sep 18 02:21:08 at org.junit.Assert.assertTrue(Assert.java:42)
> Sep 18 02:21:08 at
> org.apache.flink.runtime.operators.lifecycle.validation.TestJobDataFlowValidator.lambda$checkDataFlow$1(TestJobDataFlowValidator.java:96)
> Sep 18 02:21:08 at java.util.HashMap.forEach(HashMap.java:1289)
> Sep 18 02:21:08 at
> org.apache.flink.runtime.operators.lifecycle.validation.TestJobDataFlowValidator.checkDataFlow(TestJobDataFlowValidator.java:94)
> Sep 18 02:21:08 at
> org.apache.flink.runtime.operators.lifecycle.validation.TestJobDataFlowValidator.checkDataFlow(TestJobDataFlowValidator.java:62)
> Sep 18 02:21:08 at
> org.apache.flink.runtime.operators.lifecycle.PartiallyFinishedSourcesITCase.test(PartiallyFinishedSourcesITCase.java:139)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #17452: [FLINK-20732][connector/pulsar] Introduction of Pulsar Sink
flinkbot edited a comment on pull request #17452: URL: https://github.com/apache/flink/pull/17452#issuecomment-940136217 ## CI report: * Unknown: [CANCELED](TBD) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] jianyun8023 edited a comment on pull request #17452: [FLINK-20732][connector/pulsar] Introduction of Pulsar Sink
jianyun8023 edited a comment on pull request #17452: URL: https://github.com/apache/flink/pull/17452#issuecomment-940139594 @AHeise @syhily @nlu90 This is the pr of Pulsar's new Sink, if you're have time to help me review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17452: [FLINK-20732][connector/pulsar] Introduction of Pulsar Sink
flinkbot edited a comment on pull request #17452: URL: https://github.com/apache/flink/pull/17452#issuecomment-940136217 ## CI report: * Unknown: [CANCELED](TBD) * 7a74987d989f699bc68c2da8e0cd9bf3fa8065a6 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17452: [FLINK-20732][connector/pulsar] Introduction of Pulsar Sink
flinkbot edited a comment on pull request #17452: URL: https://github.com/apache/flink/pull/17452#issuecomment-940136217 ## CI report: * 43ffad7e2cbbb990ea6c1008a804986b844aa639 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=24961) * Unknown: [CANCELED](TBD) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17452: [FLINK-20732][connector/pulsar] Introduction of Pulsar Sink
flinkbot edited a comment on pull request #17452: URL: https://github.com/apache/flink/pull/17452#issuecomment-940136217 ## CI report: * 43ffad7e2cbbb990ea6c1008a804986b844aa639 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=24961) * Unknown: [CANCELED](TBD) * 7a74987d989f699bc68c2da8e0cd9bf3fa8065a6 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-24331) PartiallyFinishedSourcesITCase fails with "No downstream received 0 from xxx;"
[
https://issues.apache.org/jira/browse/FLINK-24331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Piotr Nowojski updated FLINK-24331:
---
Priority: Critical (was: Major)
> PartiallyFinishedSourcesITCase fails with "No downstream received 0 from xxx;"
> --
>
> Key: FLINK-24331
> URL: https://issues.apache.org/jira/browse/FLINK-24331
> Project: Flink
> Issue Type: Bug
> Components: API / DataStream
>Affects Versions: 1.14.0, 1.15.0
>Reporter: Xintong Song
>Priority: Critical
> Labels: test-stability
> Fix For: 1.15.0
>
> Attachments:
> logs-ci_build-test_ci_build_finegrained_resource_management-1633890853.zip
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=24287=logs=4d4a0d10-fca2-5507-8eed-c07f0bdf4887=7b25afdf-cc6c-566f-5459-359dc2585798=10945
> {code}
> Sep 18 02:21:08 [ERROR] Tests run: 12, Failures: 1, Errors: 0, Skipped: 0,
> Time elapsed: 224.44 s <<< FAILURE! - in
> org.apache.flink.runtime.operators.lifecycle.PartiallyFinishedSourcesITCase
> Sep 18 02:21:08 [ERROR] test[complex graph SINGLE_SUBTASK, failover: true,
> strategy: region] Time elapsed: 28.807 s <<< FAILURE!
> Sep 18 02:21:08 java.lang.AssertionError: No downstream received 0 from
> 0003[0]; received: {0=OperatorFinished
> 0007/0, 1=OperatorFinished
> 0007/1, 2=OperatorFinished
> 0007/2, 3=OperatorFinished
> 0007/3}
> Sep 18 02:21:08 at org.junit.Assert.fail(Assert.java:89)
> Sep 18 02:21:08 at org.junit.Assert.assertTrue(Assert.java:42)
> Sep 18 02:21:08 at
> org.apache.flink.runtime.operators.lifecycle.validation.TestJobDataFlowValidator.lambda$checkDataFlow$1(TestJobDataFlowValidator.java:96)
> Sep 18 02:21:08 at java.util.HashMap.forEach(HashMap.java:1289)
> Sep 18 02:21:08 at
> org.apache.flink.runtime.operators.lifecycle.validation.TestJobDataFlowValidator.checkDataFlow(TestJobDataFlowValidator.java:94)
> Sep 18 02:21:08 at
> org.apache.flink.runtime.operators.lifecycle.validation.TestJobDataFlowValidator.checkDataFlow(TestJobDataFlowValidator.java:62)
> Sep 18 02:21:08 at
> org.apache.flink.runtime.operators.lifecycle.PartiallyFinishedSourcesITCase.test(PartiallyFinishedSourcesITCase.java:139)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] jianyun8023 commented on pull request #17452: [FLINK-20732][connector/pulsar] Introduction of Pulsar Sink
jianyun8023 commented on pull request #17452: URL: https://github.com/apache/flink/pull/17452#issuecomment-940143459 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17452: [FLINK-20732][connector/pulsar] Introduction of Pulsar Sink
flinkbot edited a comment on pull request #17452: URL: https://github.com/apache/flink/pull/17452#issuecomment-940136217 ## CI report: * 43ffad7e2cbbb990ea6c1008a804986b844aa639 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=24961) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] jianyun8023 commented on pull request #17452: [FLINK-20732][connector/pulsar] Introduction of Pulsar Sink
jianyun8023 commented on pull request #17452: URL: https://github.com/apache/flink/pull/17452#issuecomment-940140071 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] jianyun8023 commented on pull request #17452: [FLINK-20732][connector/pulsar] Introduction of Pulsar Sink
jianyun8023 commented on pull request #17452: URL: https://github.com/apache/flink/pull/17452#issuecomment-940139594 @syhily @nlu90 This is the pr of Pulsar's new Sink, if you have time to help review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17452: [FLINK-20732][connector/pulsar] Introduction of Pulsar Sink
flinkbot edited a comment on pull request #17452: URL: https://github.com/apache/flink/pull/17452#issuecomment-940136217 ## CI report: * 43ffad7e2cbbb990ea6c1008a804986b844aa639 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=24961) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] jianyun8023 commented on pull request #17452: [FLINK-20732][connector/pulsar] Introduction of Pulsar Sink
jianyun8023 commented on pull request #17452: URL: https://github.com/apache/flink/pull/17452#issuecomment-940137342 There is still a lot of room for improvement in this part of PulsarSink's state saving and recovery, so please help! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #17452: [FLINK-20732][connector/pulsar] Introduction of Pulsar Sink
flinkbot commented on pull request #17452: URL: https://github.com/apache/flink/pull/17452#issuecomment-940136862 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 43ffad7e2cbbb990ea6c1008a804986b844aa639 (Mon Oct 11 15:34:58 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-24448) Tumbling Window Not working with EPOOCH Time converted using TO_TIMESTAMP_LTZ
[
https://issues.apache.org/jira/browse/FLINK-24448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17427204#comment-17427204
]
Mehul Batra commented on FLINK-24448:
-
Hi, [~twalthr] can you please help me understand this as we really want to move
to flink 1.14 due to the job name inconsistency in 1.13.1
> Tumbling Window Not working with EPOOCH Time converted using TO_TIMESTAMP_LTZ
> -
>
> Key: FLINK-24448
> URL: https://issues.apache.org/jira/browse/FLINK-24448
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Runtime
>Reporter: Mehul Batra
>Priority: Major
>
> *When I am running my code to test the connector = 'print' to see my window
> aggregated data it is not printing anything in version 1.14.0 and when I am
> excluding the tumbling window it is printing data, but the same code is
> working with the tumble window in FLINK 1.13.1.*
> SQL API CONNECTORS TO REFER :
> tableEnv.executeSql("CREATE TABLE IF NOT EXISTS Teamtopic (\n"
> + " eventName String,\n"
> + " ingestion_time BIGINT,\n"
> + " t_ltz as TO_TIMESTAMP_LTZ(ingestion_time,3) ,
> + " WATERMARK FOR t_ltz AS t_ltz - INTERVAL '5' SECOND
> + " as event-time attribute\n"
> + ") WITH (\n"
> + " 'connector' = 'kafka'
> tableEnv.executeSql("CREATE TABLE minutess (\n"
> + " `minute` TIMESTAMP(3),\n"
> + " hits BIGINT ,\n"
> + " type STRING\n"
> + ") WITH (\n"
> + " 'connector' = 'print' "
> + ")");
> tableEnv.createStatementSet()
> .addInsertSql("INSERT INTO minutess \n"
> + " SELECT "
> + "TUMBLE_END(t_ltz,INTERVAL '1' MINUTE) AS windowmin ,"
> + "COUNT(eventName) as hits, "
> + "'team_save_failed_minute_error_types' as type\n"
> + " FROM TeamSaveFailed\n"
> +" GROUP BY TUMBLE(t_ltz, INTERVAL '1' MINUTE ),eventName")
> .execute();
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot commented on pull request #17452: [FLINK-20732][connector/pulsar] Introduction of Pulsar Sink
flinkbot commented on pull request #17452: URL: https://github.com/apache/flink/pull/17452#issuecomment-940136217 ## CI report: * 43ffad7e2cbbb990ea6c1008a804986b844aa639 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-20732) Pulsar Sink
[ https://issues.apache.org/jira/browse/FLINK-20732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-20732: --- Labels: pull-request-available (was: ) > Pulsar Sink > --- > > Key: FLINK-20732 > URL: https://issues.apache.org/jira/browse/FLINK-20732 > Project: Flink > Issue Type: Sub-task > Components: Connectors / Common >Affects Versions: 1.13.0 >Reporter: Jianyun Zhao >Assignee: Jianyun Zhao >Priority: Major > Labels: pull-request-available > > Based on the FLIP-143 implementation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] jianyun8023 opened a new pull request #17452: [FLINK-20732][connector/pulsar] Introduction of Pulsar Sink
jianyun8023 opened a new pull request #17452: URL: https://github.com/apache/flink/pull/17452 ## What is the purpose of the change This pull request completes the development of Pulsar based on the new Sink, which will enable smoother data interaction between Flink and Pulsar. ## Brief change log Introduction Pulsar Sink Connector : - support exactly-once or at-latest-once feature. - `Sink`, `SinkWriter`, `Committer`, and other interfaces are implemented. ## Verifying this change This change added tests and can be verified as follows: - *Added a simple pulsar sink e2e test* - *CommitterState ser test* - *Unit testing of some support classes* ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes) - The serializers: (don't know) - The runtime per-record code paths (performance sensitive): (don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no) - The S3 file system connector: (no) ## Documentation - Does this pull request introduce a new feature? (yes) - If yes, how is the feature documented? (docs / JavaDocs) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-20732) Pulsar Sink
[ https://issues.apache.org/jira/browse/FLINK-20732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17427197#comment-17427197 ] Jianyun Zhao commented on FLINK-20732: -- [~MartijnVisser] Thank you~ > Pulsar Sink > --- > > Key: FLINK-20732 > URL: https://issues.apache.org/jira/browse/FLINK-20732 > Project: Flink > Issue Type: Sub-task > Components: Connectors / Common >Affects Versions: 1.13.0 >Reporter: Jianyun Zhao >Assignee: Jianyun Zhao >Priority: Major > > Based on the FLIP-143 implementation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
