[GitHub] [flink] KarmaGYZ commented on a change in pull request #16405: [FLINK-23165][python] Add StreamExecutionEnvironment#registerSlotSharingGroup to PyFlink and Scala
KarmaGYZ commented on a change in pull request #16405:
URL: https://github.com/apache/flink/pull/16405#discussion_r665065834
##
File path:
flink-streaming-scala/src/main/scala/org/apache/flink/streaming/api/scala/DataStream.scala
##
@@ -327,6 +327,29 @@ class DataStream[T](stream: JavaStream[T]) {
this
}
+ /**
Review comment:
It's better to be split into another commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16215: [FLINK-23023][table-planner-blink] Support offset in window TVF.
flinkbot edited a comment on pull request #16215: URL: https://github.com/apache/flink/pull/16215#issuecomment-864821802 ## CI report: * ffc2f22cf926fd97e18b6d9e527d4e6f5556b91f Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=19512) * 1161b3f3dcf37ec14edee3ce83cae6eea71af7ac UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] hehuiyuan commented on pull request #16394: [FLINK-20321][formats] Fix NPE when using AvroDeserializationSchema to deserialize null input in 1.12 version
hehuiyuan commented on pull request #16394: URL: https://github.com/apache/flink/pull/16394#issuecomment-875305823 @wuchong -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-23219) temproary join ttl configruation does not take effect
[
https://issues.apache.org/jira/browse/FLINK-23219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
waywtdcc updated FLINK-23219:
-
Description:
* version: flink 1.12.2
* problem: I run the job of table A temproary left join table B, and set the
table.exec.state.ttl configuration
to 3 hour or 2 sencond for test. But the task status keeps growing for more
than 7 days.
* code
{code:java}
tableEnv.getConfig().setIdleStateRetention(Duration.ofSeconds(2));
tableEnv.executeSql("drop table if exists persons_table_kafka2");
String kafka_source_sql = "CREATE TABLE persons_table_kafka2 (\n" +
" `id` BIGINT,\n" +
" `name` STRING,\n" +
" `age` INT,\n" +
" proctime as PROCTIME(),\n" +
" `ts` TIMESTAMP(3),\n" +
" WATERMARK FOR ts AS ts\n" +
") WITH (\n" +
" 'connector' = 'kafka',\n" +
" 'topic' = 'persons_test_auto',\n" +
" 'properties.bootstrap.servers' = 'node2:6667',\n" +
" 'properties.group.id' = 'testGrodsu1765',\n" +
" 'scan.startup.mode' = 'group-offsets',\n" +
" 'format' = 'json'\n" +
")";
tableEnv.executeSql(kafka_source_sql);
tableEnv.executeSql("drop table if exists persons_message_table_kafka2");
String kafka_source_sql2 = "CREATE TABLE persons_message_table_kafka2 (\n" +
" `id` BIGINT,\n" +
" `name` STRING,\n" +
" `message` STRING,\n" +
" `ts` TIMESTAMP(3) ," +
// " WATERMARK FOR ts AS ts - INTERVAL '5' SECOND\n" +
" WATERMARK FOR ts AS ts\n" +
") WITH (\n" +
" 'connector' = 'kafka',\n" +
" 'topic' = 'persons_extra_message_auto',\n" +
" 'properties.bootstrap.servers' = 'node2:6667',\n" +
" 'properties.group.id' = 'testGroud125313',\n" +
" 'scan.startup.mode' = 'group-offsets',\n" +
" 'format' = 'json'\n" +
")";
tableEnv.executeSql(kafka_source_sql2);
tableEnv.executeSql(
"CREATE TEMPORARY VIEW persons_message_table22 AS \n" +
"SELECT id, name, message,ts \n" +
" FROM (\n" +
" SELECT *,\n" +
" ROW_NUMBER() OVER (PARTITION BY name \n" +
" ORDER BY ts DESC) AS rowNum \n" +
" FROM persons_message_table_kafka2 " +
" )\n" +
"WHERE rowNum = 1");
tableEnv.executeSql("" +
"CREATE TEMPORARY VIEW result_data_view " +
" as " +
" select " +
" t1.name, t1.age,t1.ts as ts1,t2.message, cast (t2.ts as string) as ts2 " +
" from persons_table_kafka2 t1 " +
" left join persons_message_table22 FOR SYSTEM_TIME AS OF t1.ts AS t2 on
t1.name = t2.name "
);
Table resultTable = tableEnv.from("result_data_view");
DataStream rowDataDataStream = tableEnv.toAppendStream(resultTable,
RowData.class);
rowDataDataStream.print();
env.execute("test_it");
{code}
* the result like !image-2021-07-02-16-29-40-310.png!
was:
* version: flink 1.12.2
* problem: I run the job of table A temproary left join table B, and set the
table.exec.state.ttl configuration
to 3 hour or 2 sencond for test. But the task status keeps growing for more
than 7 days.
* code
{code:java}
tableEnv.getConfig().setIdleStateRetention(Duration.ofSeconds(2));
tableEnv.executeSql("drop table if exists persons_table_kafka2");
String kafka_source_sql = "CREATE TABLE persons_table_kafka2 (\n" +
" `id` BIGINT,\n" +
" `name` STRING,\n" +
" `age` INT,\n" +
" proctime as PROCTIME(),\n" +
" `ts` TIMESTAMP(3),\n" +
" WATERMARK FOR ts AS ts\n" +
") WITH (\n" +
" 'connector' = 'kafka',\n" +
" 'topic' = 'persons_test_auto',\n" +
" 'properties.bootstrap.servers' = 'node2:6667',\n" +
" 'properties.group.id' = 'testGrodsu1765',\n" +
" 'scan.startup.mode' = 'group-offsets',\n" +
" 'format' = 'json'\n" +
")";
tableEnv.executeSql(kafka_source_sql);
tableEnv.executeSql("drop table if exists persons_message_table_kafka2");
String kafka_source_sql2 = "CREATE TABLE persons_message_table_kafka2 (\n" +
" `id` BIGINT,\n" +
" `name` STRING,\n" +
" `message` STRING,\n" +
" `ts` TIMESTAMP(3) ," +
// " WATERMARK FOR ts AS ts - INTERVAL '5' SECOND\n" +
" WATERMARK FOR ts AS ts\n" +
") WITH (\n" +
" 'connector' = 'kafka',\n" +
" 'topic' = 'persons_extra_message_auto',\n" +
" 'properties.bootstrap.servers' = 'node2:6667',\n" +
" 'properties.group.id' = 'testGroud125313',\n" +
" 'scan.startup.mode' = 'group-offsets',\n" +
" 'format' = 'json'\n" +
")";
tableEnv.executeSql(kafka_source_sql2);
tableEnv.executeSql(
"CREATE TEMPORARY VIEW persons_message_table22 AS \n" +
"SELECT id, name, message,ts \n" +
" FROM (\n" +
" SELECT *,\n" +
" ROW_NUMBER() OVER (PARTITION BY name \n" +
" ORDER BY ts DESC) AS rowNum \n" +
" FROM persons_message_table_kafka2 " +
" )\n" +
"WHERE rowNum = 1");
tableEnv.executeSql("" +
"CREATE TEMPORARY VIEW result_data_view " +
" as " +
" select " +
" t1.name, t1.age,t1.ts as ts1,t2.message, cast (t2.ts as string) as ts2 " +
" from persons_table_kafka2 t1 " +
" left join persons_message_table22 FOR SYSTEM_TIME AS OF t1.ts AS t2 on
t1.name = t2.name "
);
{code}
* the result like !image-2021-07-02-16-29-40-310.png!
> temproary join ttl configruation does not take effect
>
[jira] [Commented] (FLINK-23289) BinarySection should null check in constructor method
[
https://issues.apache.org/jira/browse/FLINK-23289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17376282#comment-17376282
]
Jingsong Lee commented on FLINK-23289:
--
+1
> BinarySection should null check in constructor method
> -
>
> Key: FLINK-23289
> URL: https://issues.apache.org/jira/browse/FLINK-23289
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Runtime
>Reporter: Terry Wang
>Priority: Major
>
> {code:java}
> Caused by: java.lang.NullPointerException
> at
> org.apache.flink.table.data.binary.BinarySegmentUtils.inFirstSegment(BinarySegmentUtils.java:411)
> at
> org.apache.flink.table.data.binary.BinarySegmentUtils.copyToBytes(BinarySegmentUtils.java:132)
> at
> org.apache.flink.table.data.binary.BinarySegmentUtils.copyToBytes(BinarySegmentUtils.java:118)
> at
> org.apache.flink.table.data.binary.BinaryStringData.copy(BinaryStringData.java:360)
> at
> org.apache.flink.table.runtime.typeutils.StringDataSerializer.copy(StringDataSerializer.java:59)
> at
> org.apache.flink.table.runtime.typeutils.StringDataSerializer.copy(StringDataSerializer.java:37)
> at
> org.apache.flink.table.runtime.typeutils.ArrayDataSerializer.copyGenericArray(ArrayDataSerializer.java:128)
> at
> org.apache.flink.table.runtime.typeutils.ArrayDataSerializer.copy(ArrayDataSerializer.java:86)
> at
> org.apache.flink.table.runtime.typeutils.ArrayDataSerializer.copy(ArrayDataSerializer.java:47)
> at
> org.apache.flink.table.runtime.typeutils.RowDataSerializer.copyRowData(RowDataSerializer.java:170)
> at
> org.apache.flink.table.runtime.typeutils.RowDataSerializer.copy(RowDataSerializer.java:131)
> at
> org.apache.flink.table.runtime.typeutils.RowDataSerializer.copy(RowDataSerializer.java:48)
> at
> org.apache.flink.table.runtime.operators.join.lookup.AsyncLookupJoinWithCalcRunner$CalcCollectionCollector.collect(AsyncLookupJoinWithCalcRunner.java:152)
> at
> org.apache.flink.table.runtime.operators.join.lookup.AsyncLookupJoinWithCalcRunner$CalcCollectionCollector.collect(AsyncLookupJoinWithCalcRunner.java:142)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] wsry commented on a change in pull request #11877: [FLINK-16641][network] Announce sender's backlog to solve the deadlock issue without exclusive buffers
wsry commented on a change in pull request #11877:
URL: https://github.com/apache/flink/pull/11877#discussion_r665059027
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/BoundedBlockingSubpartitionDirectTransferReader.java
##
@@ -91,10 +91,14 @@ public BufferAndBacklog getNextBuffer() throws IOException {
updateStatistics(current);
-// We simply assume all the data are non-events for batch jobs to
avoid pre-fetching the
-// next header
-Buffer.DataType nextDataType =
-numDataAndEventBuffers > 0 ? Buffer.DataType.DATA_BUFFER :
Buffer.DataType.NONE;
+// We simply assume all the data except for the last one
(EndOfPartitionEvent)
+// are non-events for batch jobs to avoid pre-fetching the next header
+Buffer.DataType nextDataType = Buffer.DataType.NONE;
+if (numDataBuffers > 0) {
+nextDataType = Buffer.DataType.DATA_BUFFER;
+} else if (numDataAndEventBuffers > 0) {
+nextDataType = Buffer.DataType.EVENT_BUFFER;
+}
Review comment:
I think this is just an optimization. Without this change, the backlog
announced to the downstream does not include the event and EVENT_BUFFER and
DATA_BUFFER are not distinguished, this means the event buffer also need a
credit to send. If there is no exclusive buffer, no enough buffer will be
allocated for the event, because we only announce the data buffer backlog to
the downstream. As a result, some tests will hang for there is no credit for
the event buffer. To solve this dead lock, there are two simple ways:
1. Announce both data buffer and event buffer backlog to the downstream,
this lead to allocate more buffers than needed, these buffers will be released
when the EOF is received at the downstream task.
2. Distinguish EVENT_BUFFER and DATA_BUFFER just like what is doing now.
These choices are both acceptable to me. I chose the second one because
EVENT_BUFFER and DATA_BUFFER are distinguished at the downstream task and we
can allocate one less buffer.
Both of the choices need to do some change to
BoundedBlockingSubpartitionDirectTransferReader. Which one do you prefer? I
think both are acceptable for me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16405: [FLINK-23165][python] Add StreamExecutionEnvironment#registerSlotSharingGroup to PyFlink and Scala
flinkbot edited a comment on pull request #16405: URL: https://github.com/apache/flink/pull/16405#issuecomment-875224279 ## CI report: * b7fc24e07e9fb4eb0ac2b9ac6e16e89cd9041aa7 UNKNOWN * e9eb7e2451023def24169564053f8a0eeb746176 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20046) * a4456c35f860b3fff7ef3e74a7e77df77d4c3786 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20056) * 9b55685f88d05d0d608e8c3864bfb5fbfa192889 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20057) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16404: [FLINK-23277][state/changelog] Store and recover TTL metadata using changelog
flinkbot edited a comment on pull request #16404: URL: https://github.com/apache/flink/pull/16404#issuecomment-875114816 ## CI report: * 8eb9d49182bddde07fc1abd731347206c820471f Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20042) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] Airblader commented on pull request #16287: [FLINK-23066] Introduce TableEnvironment#from(TableDescriptor)
Airblader commented on pull request #16287: URL: https://github.com/apache/flink/pull/16287#issuecomment-875297336 @twalthr Rebased -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16402: [FLINK-23278][state/changelog] Allow recovery without non-materialized state
flinkbot edited a comment on pull request #16402: URL: https://github.com/apache/flink/pull/16402#issuecomment-875102555 ## CI report: * 676141bc40216f55c169f6202e1f0b7edf289508 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20040) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16369: [FLINK-22443][streaming-java] Fix overflow in MultipleInputSelectionHandler
flinkbot edited a comment on pull request #16369: URL: https://github.com/apache/flink/pull/16369#issuecomment-873893547 ## CI report: * 3bb34b002d5a8952f045088b16f6a4c6a0532527 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=19917) * 11105a90383651025185cdc1804733c19e59b8ca Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20049) Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20059) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16349: [FLINK-23267][table-planner-blink] Enable Java code splitting for all generated classes
flinkbot edited a comment on pull request #16349: URL: https://github.com/apache/flink/pull/16349#issuecomment-872790415 ## CI report: * 9747a5d6ae327f3078c97fb95020c1cffccf5e20 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=19976) * 37f9fc51058e1572a1e2d29d616c5f5db9dc46db Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20054) * 4897910e2eae3cde8806cb08c75bc7d008f00402 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16348: [FLINK-16093][docs-zh]Translate "System Functions" page of "Functions" into Chinese
flinkbot edited a comment on pull request #16348: URL: https://github.com/apache/flink/pull/16348#issuecomment-872690776 ## CI report: * 81f69ba757726f2aaa4a0fce16e6cf412fd05f2a UNKNOWN * 25af18e59b680ad66d9b0d98c2176ffbdc2e0214 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20052) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16163: [FLINK-22994][Table SQL / Planner] improve the performace of invoking…
flinkbot edited a comment on pull request #16163: URL: https://github.com/apache/flink/pull/16163#issuecomment-861284972 ## CI report: * 00f7cd34fc503db8cc116dd201bcc7f54a4b6f20 UNKNOWN * da303c24e513b0678794526c6351003b6e76cb6d UNKNOWN * 4133ee287c23af5da472db1098c9d7ab51b81cb9 UNKNOWN * 5ba22e8ce8dcf237d3e7a6626fafe33d203f8b31 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=19986) * 9af8cef01d169ef25c7f7ffb1e31d111b1fb6b3c UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] TsReaper commented on pull request #16369: [FLINK-22443][streaming-java] Fix overflow in MultipleInputSelectionHandler
TsReaper commented on pull request #16369: URL: https://github.com/apache/flink/pull/16369#issuecomment-875282299 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16403: [FLINK-23276][state/changelog] Fix missing delegation in getPartitionedState
flinkbot edited a comment on pull request #16403: URL: https://github.com/apache/flink/pull/16403#issuecomment-875102625 ## CI report: * 1d1368d281acdb14f47adc7ec19078d7e21dda5f Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20041) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16359: [FLINK-23222]Translate page 'Application Profiling & Debugging' into Chinese
flinkbot edited a comment on pull request #16359: URL: https://github.com/apache/flink/pull/16359#issuecomment-872999229 ## CI report: * f38e1fff477d7520a34e21ec20d9ede7b621f522 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20048) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] zicat commented on pull request #16163: [FLINK-22994][Table SQL / Planner] improve the performace of invoking…
zicat commented on pull request #16163: URL: https://github.com/apache/flink/pull/16163#issuecomment-875277764 > @zicat Can you create ITCase instead of UT `BridgingFunctionGenUtilTest`? You can create case in `CalcITCase`. Hi @JingsongLi I add the test function testOptimizeNestingInvokeScalarFunction in `CalcITCase` and delete the UT `BridgingFunctionGenUtilTest`. Please help to review it again, thx. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (FLINK-22657) HiveParserDDLSemanticAnalyzer can directly return operations
[ https://issues.apache.org/jira/browse/FLINK-22657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li reassigned FLINK-22657: -- Assignee: luoyuxia > HiveParserDDLSemanticAnalyzer can directly return operations > > > Key: FLINK-22657 > URL: https://issues.apache.org/jira/browse/FLINK-22657 > Project: Flink > Issue Type: Sub-task > Components: Connectors / Hive >Reporter: Rui Li >Assignee: luoyuxia >Priority: Major > > There's no need to first generate some "desc" and later convert to operation > with DDLOperationConverter -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (FLINK-23283) GroupWindowITCase.testWindowAggregateOnUpsertSource fails on azure
[
https://issues.apache.org/jira/browse/FLINK-23283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song reassigned FLINK-23283:
Assignee: JING ZHANG
> GroupWindowITCase.testWindowAggregateOnUpsertSource fails on azure
> --
>
> Key: FLINK-23283
> URL: https://issues.apache.org/jira/browse/FLINK-23283
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.14.0
>Reporter: Roman Khachatryan
>Assignee: JING ZHANG
>Priority: Blocker
> Fix For: 1.14.0
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=19978=logs=e25d5e7e-2a9c-5589-4940-0b638d75a414=a6e0f756-5bb9-5ea8-a468-5f60db442a29=7086
> {code}
> 2021-07-06T10:13:10.6091443Z Jul 06 10:13:10 [ERROR] Tests run: 48, Failures:
> 1, Errors: 0, Skipped: 0, Time elapsed: 14.945 s <<< FAILURE! - in
> org.apache.flink.table.planner.runtime.stream.sql.GroupWindowITCase
> 2021-07-06T10:13:10.6093667Z Jul 06 10:13:10 [ERROR]
> testWindowAggregateOnUpsertSource[StateBackend=HEAP, UseTimestampLtz =
> false](org.apache.flin*k.table.planner.runtime.stream.sql.GroupWindowITCase)
> Time elapsed: 0.329 s <<< FAILURE!
> 2021-07-06T10:13:10.6096965Z Jul 06 10:13:10 java.lang.AssertionError:
> expected: r,1,102,1970-01-01T00:00,1970-01-01T00:00:05,
> Yen,1,1,1970-01-01T00:00,1970-01-01T00:00:05)> but
> was: Dollar,1,104,1970-01-01T00:00,1970-01-01T00:00:05,
> Yen,1,1,1970-01-01T00:00,1970-01-01T00:00:05)>
> 2021-07-06T10:13:10.6098985Z Jul 06 10:13:10at
> org.junit.Assert.fail(Assert.java:89)
> 2021-07-06T10:13:10.6099695Z Jul 06 10:13:10at
> org.junit.Assert.failNotEquals(Assert.java:835)
> 2021-07-06T10:13:10.6100489Z Jul 06 10:13:10at
> org.junit.Assert.assertEquals(Assert.java:120)
> 2021-07-06T10:13:10.6101292Z Jul 06 10:13:10at
> org.junit.Assert.assertEquals(Assert.java:146)
> 2021-07-06T10:13:10.6102395Z Jul 06 10:13:10at
> org.apache.flink.table.planner.runtime.stream.sql.GroupWindowITCase.testWindowAggregateOnUpsert*Source(GroupWindowITCase.scala:421)
> {code}
> Fails locally 3 times out of 1000
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-23283) GroupWindowITCase.testWindowAggregateOnUpsertSource fails on azure
[
https://issues.apache.org/jira/browse/FLINK-23283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17376252#comment-17376252
]
Xintong Song commented on FLINK-23283:
--
Thank you [~qingru zhang], you are assigned.
> GroupWindowITCase.testWindowAggregateOnUpsertSource fails on azure
> --
>
> Key: FLINK-23283
> URL: https://issues.apache.org/jira/browse/FLINK-23283
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.14.0
>Reporter: Roman Khachatryan
>Assignee: JING ZHANG
>Priority: Blocker
> Fix For: 1.14.0
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=19978=logs=e25d5e7e-2a9c-5589-4940-0b638d75a414=a6e0f756-5bb9-5ea8-a468-5f60db442a29=7086
> {code}
> 2021-07-06T10:13:10.6091443Z Jul 06 10:13:10 [ERROR] Tests run: 48, Failures:
> 1, Errors: 0, Skipped: 0, Time elapsed: 14.945 s <<< FAILURE! - in
> org.apache.flink.table.planner.runtime.stream.sql.GroupWindowITCase
> 2021-07-06T10:13:10.6093667Z Jul 06 10:13:10 [ERROR]
> testWindowAggregateOnUpsertSource[StateBackend=HEAP, UseTimestampLtz =
> false](org.apache.flin*k.table.planner.runtime.stream.sql.GroupWindowITCase)
> Time elapsed: 0.329 s <<< FAILURE!
> 2021-07-06T10:13:10.6096965Z Jul 06 10:13:10 java.lang.AssertionError:
> expected: r,1,102,1970-01-01T00:00,1970-01-01T00:00:05,
> Yen,1,1,1970-01-01T00:00,1970-01-01T00:00:05)> but
> was: Dollar,1,104,1970-01-01T00:00,1970-01-01T00:00:05,
> Yen,1,1,1970-01-01T00:00,1970-01-01T00:00:05)>
> 2021-07-06T10:13:10.6098985Z Jul 06 10:13:10at
> org.junit.Assert.fail(Assert.java:89)
> 2021-07-06T10:13:10.6099695Z Jul 06 10:13:10at
> org.junit.Assert.failNotEquals(Assert.java:835)
> 2021-07-06T10:13:10.6100489Z Jul 06 10:13:10at
> org.junit.Assert.assertEquals(Assert.java:120)
> 2021-07-06T10:13:10.6101292Z Jul 06 10:13:10at
> org.junit.Assert.assertEquals(Assert.java:146)
> 2021-07-06T10:13:10.6102395Z Jul 06 10:13:10at
> org.apache.flink.table.planner.runtime.stream.sql.GroupWindowITCase.testWindowAggregateOnUpsert*Source(GroupWindowITCase.scala:421)
> {code}
> Fails locally 3 times out of 1000
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #16401: [FLINK-22889] Add debug statements to JdbcExactlyOnceSinkE2eTest
flinkbot edited a comment on pull request #16401: URL: https://github.com/apache/flink/pull/16401#issuecomment-875082819 ## CI report: * f01334a7976d820cd0308b3994c9a7c696aa653c Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20039) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16405: [FLINK-23165][python] Add StreamExecutionEnvironment#registerSlotSharingGroup to PyFlink and Scala
flinkbot edited a comment on pull request #16405: URL: https://github.com/apache/flink/pull/16405#issuecomment-875224279 ## CI report: * b7fc24e07e9fb4eb0ac2b9ac6e16e89cd9041aa7 UNKNOWN * e9eb7e2451023def24169564053f8a0eeb746176 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20046) * a4456c35f860b3fff7ef3e74a7e77df77d4c3786 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20056) * 9b55685f88d05d0d608e8c3864bfb5fbfa192889 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16379: [BP-1.13][FLINK-22722 / FLINK-22766][connector/kafka] Add docs and metrics for Kafka new source
flinkbot edited a comment on pull request #16379: URL: https://github.com/apache/flink/pull/16379#issuecomment-874137469 ## CI report: * ca0408634212c0c0ea1b4e7da66095fa10af2a04 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=19932) * 59cbbb4c0fd63984705c33dbf60d0efe0f39b6dc Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20055) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16368: ignore
flinkbot edited a comment on pull request #16368: URL: https://github.com/apache/flink/pull/16368#issuecomment-873854539 ## CI report: * 4ef1427b1f0b5882ade6f3fc776763ab6ade1674 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20038) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16349: [FLINK-23267][table-planner-blink] Enable Java code splitting for all generated classes
flinkbot edited a comment on pull request #16349: URL: https://github.com/apache/flink/pull/16349#issuecomment-872790415 ## CI report: * 9747a5d6ae327f3078c97fb95020c1cffccf5e20 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=19976) * 37f9fc51058e1572a1e2d29d616c5f5db9dc46db Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20054) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16341: [FLINK-21804][state/changelog] Create and wire changelog storage with state backend
flinkbot edited a comment on pull request #16341: URL: https://github.com/apache/flink/pull/16341#issuecomment-872173720 ## CI report: * 864fd641dbe6d5101e8f9ed4ef87c7d6c2b009b1 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20053) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] RocMarshal commented on a change in pull request #16359: [FLINK-23222]Translate page 'Application Profiling & Debugging' into Chinese
RocMarshal commented on a change in pull request #16359:
URL: https://github.com/apache/flink/pull/16359#discussion_r665031149
##
File path: docs/content.zh/docs/ops/debugging/application_profiling.md
##
@@ -25,60 +25,48 @@ specific language governing permissions and limitations
under the License.
-->
-# Application Profiling & Debugging
+# 应用程序分析与调试
-## Overview of Custom Logging with Apache Flink
+## Apache Flink 自定义日志概述
-Each standalone JobManager, TaskManager, HistoryServer, and ZooKeeper daemon
redirects `stdout` and `stderr` to a file
-with a `.out` filename suffix and writes internal logging to a file with a
`.log` suffix. Java options configured by the
-user in `env.java.opts`, `env.java.opts.jobmanager`,
`env.java.opts.taskmanager`, `env.java.opts.historyserver` and
-`env.java.opts.client` can likewise define log files with
-use of the script variable `FLINK_LOG_PREFIX` and by enclosing the options in
double quotes for late evaluation. Log files
-using `FLINK_LOG_PREFIX` are rotated along with the default `.out` and `.log`
files.
+每个独立的 JobManager，TaskManager，HistoryServer，ZooKeeper 守护进程都将 `stdout` 和
`stderr` 重定向到名称后缀为 `.out` 的文件，并将其内部的日志记录写入到 `.log` 后缀的文件。用户可以在
`env.java.opts`，`env.java.opts.jobmanager`，`env.java.opts.taskmanager`，`env.java.opts.historyserver`
和 `env.java.opts.client` 配置项中配置 Java 选项（包括 log 相关的选项），同样也可以使用脚本变量
`FLINK_LOG_PREFIX` 定义日志文件，并将选项括在双引号中以供后期使用。日志文件将使用 `FLINK_LOG_PREFIX` 与默认的
`.out` 和 `.log` 后缀一起滚动。
-## Profiling with Java Flight Recorder
+## 使用 Java Flight Recorder 分析
-Java Flight Recorder is a profiling and event collection framework built into
the Oracle JDK.
-[Java Mission
Control](http://www.oracle.com/technetwork/java/javaseproducts/mission-control/java-mission-control-1998576.html)
-is an advanced set of tools that enables efficient and detailed analysis of
the extensive of data collected by Java
-Flight Recorder. Example configuration:
+Java Flight Recorder 是 Oracle JDK 内置的分析和事件收集框架。[Java Mission
Control](http://www.oracle.com/technetwork/java/javaseproducts/mission-control/java-mission-control-1998576.html)
是一套先进的工具，可以对 Java Flight Recorder 收集的大量数据进行高效和详细的分析。配置示例：
```yaml
env.java.opts: "-XX:+UnlockCommercialFeatures -XX:+UnlockDiagnosticVMOptions
-XX:+FlightRecorder -XX:+DebugNonSafepoints
-XX:FlightRecorderOptions=defaultrecording=true,dumponexit=true,dumponexitpath=${FLINK_LOG_PREFIX}.jfr"
```
-## Profiling with JITWatch
+## 使用 JITWatch 分析
-[JITWatch](https://github.com/AdoptOpenJDK/jitwatch/wiki) is a log analyser
and visualizer for the Java HotSpot JIT
-compiler used to inspect inlining decisions, hot methods, bytecode, and
assembly. Example configuration:
+[JITWatch](https://github.com/AdoptOpenJDK/jitwatch/wiki) Java HotSpot JIT
编译器的日志分析器和可视化工具，用于检查内联决策，热方法，字节码和汇编。配置示例：
Review comment:
What about `热方法` -> `热点方法`？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (FLINK-14055) Add advanced function DDL syntax "USING JAR/FILE/ACHIVE"
[
https://issues.apache.org/jira/browse/FLINK-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17376238#comment-17376238
]
Zhenqiu Huang commented on FLINK-14055:
---
[~frank wang]
Sure. The feature covers many components in the Flink. I created a draft of
FLIP for discussion in detail. Please feel free to add comments and opinions.
For each cluster management system, we probably need more input from other
people.
https://docs.google.com/document/d/1ru6gn-SRhWUmhHtv-k2zj7B8h8RYoOUCfnxg120Gg4Q/edit#
> Add advanced function DDL syntax "USING JAR/FILE/ACHIVE"
>
>
> Key: FLINK-14055
> URL: https://issues.apache.org/jira/browse/FLINK-14055
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / API
>Reporter: Bowen Li
>Priority: Major
> Labels: auto-unassigned, sprint
>
> As FLINK-7151 adds basic function DDL to Flink, this ticket is to support
> dynamically loading functions from external source in function DDL with
> advanced syntax like
>
> {code:java}
> CREATE FUNCTION func_name as class_name USING JAR/FILE/ACHIEVE 'xxx' [,
> JAR/FILE/ACHIEVE 'yyy'] ;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-23032) Refactor HiveSource to make it usable in data stream job
[ https://issues.apache.org/jira/browse/FLINK-23032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17376233#comment-17376233 ] frank wang commented on FLINK-23032: [~lirui] can you assign to me about this feature, mayby i can help do it > Refactor HiveSource to make it usable in data stream job > > > Key: FLINK-23032 > URL: https://issues.apache.org/jira/browse/FLINK-23032 > Project: Flink > Issue Type: Improvement > Components: Connectors / Hive >Reporter: Rui Li >Assignee: Rui Li >Priority: Major > Fix For: 1.14.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-10851) sqlUpdate support complex insert grammar
[
https://issues.apache.org/jira/browse/FLINK-10851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17376230#comment-17376230
]
frank wang commented on FLINK-10851:
[~dwysakowicz] yes, we can close this ticket
> sqlUpdate support complex insert grammar
>
>
> Key: FLINK-10851
> URL: https://issues.apache.org/jira/browse/FLINK-10851
> Project: Flink
> Issue Type: Bug
>Reporter: frank wang
>Priority: Major
> Labels: pull-request-available
>
> my code is
> {{tableEnv.sqlUpdate("insert into kafka.sdkafka.product_4 select filedName1,
> filedName2 from kafka.sdkafka.order_4");}}
> but flink give me error info, said kafka "No table was registered under the
> name kafka"
> i modify the code ,that is ok now
> TableEnvironment.scala
> {code:java}
> def sqlUpdate(stmt: String, config: QueryConfig): Unit = {
> val planner = new FlinkPlannerImpl(getFrameworkConfig, getPlanner,
> getTypeFactory)
> // parse the sql query
> val parsed = planner.parse(stmt)
> parsed match {
> case insert: SqlInsert =>
> // validate the SQL query
> val query = insert.getSource
> val validatedQuery = planner.validate(query)
> // get query result as Table
> val queryResult = new Table(this,
> LogicalRelNode(planner.rel(validatedQuery).rel))
> // get name of sink table
> val targetTableName =
> insert.getTargetTable.asInstanceOf[SqlIdentifier].names.get(0)
> // insert query result into sink table
> insertInto(queryResult, targetTableName, config)
> case _ =>
> throw new TableException(
> "Unsupported SQL query! sqlUpdate() only accepts SQL statements of
> type INSERT.")
> }
> }
> {code}
> should modify to this
> {code:java}
> def sqlUpdate(stmt: String, config: QueryConfig): Unit = {
> val planner = new FlinkPlannerImpl(getFrameworkConfig, getPlanner,
> getTypeFactory)
> // parse the sql query
> val parsed = planner.parse(stmt)
> parsed match {
> case insert: SqlInsert =>
> // validate the SQL query
> val query = insert.getSource
> val validatedQuery = planner.validate(query)
> // get query result as Table
> val queryResult = new Table(this,
> LogicalRelNode(planner.rel(validatedQuery).rel))
> // get name of sink table
> //val targetTableName =
> insert.getTargetTable.asInstanceOf[SqlIdentifier].names.get(0)
> val targetTableName = insert.getTargetTable.toString
> // insert query result into sink table
> insertInto(queryResult, targetTableName, config)
> case _ =>
> throw new TableException(
> "Unsupported SQL query! sqlUpdate() only accepts SQL statements of
> type INSERT.")
> }
> }
> {code}
>
> i hope this can be acceptted， thx
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-12031) the registerFactory method of TypeExtractor Should not be private
[
https://issues.apache.org/jira/browse/FLINK-12031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17376231#comment-17376231
]
frank wang commented on FLINK-12031:
[~twalthr] i can help to solve this problem, can you assign to me ?
> the registerFactory method of TypeExtractor Should not be private
> --
>
> Key: FLINK-12031
> URL: https://issues.apache.org/jira/browse/FLINK-12031
> Project: Flink
> Issue Type: Bug
> Components: API / Type Serialization System
>Reporter: frank wang
>Priority: Minor
>
> [https://github.com/apache/flink/blob/master/flink-core/src/main/java/org/apache/flink/api/java/typeutils/TypeExtractor.java]
> {code:java}
> /**
> * Registers a type information factory globally for a certain type. Every
> following type extraction
> * operation will use the provided factory for this type. The factory will
> have highest precedence
> * for this type. In a hierarchy of types the registered factory has higher
> precedence than annotations
> * at the same level but lower precedence than factories defined down the
> hierarchy.
> *
> * @param t type for which a new factory is registered
> * @param factory type information factory that will produce {@link
> TypeInformation}
> */
> private static void registerFactory(Type t, Class
> factory) {
>Preconditions.checkNotNull(t, "Type parameter must not be null.");
>Preconditions.checkNotNull(factory, "Factory parameter must not be null.");
>if (!TypeInfoFactory.class.isAssignableFrom(factory)) {
> throw new IllegalArgumentException("Class is not a TypeInfoFactory.");
>}
>if (registeredTypeInfoFactories.containsKey(t)) {
> throw new InvalidTypesException("A TypeInfoFactory for type '" + t + "'
> is already registered.");
>}
>registeredTypeInfoFactories.put(t, factory);
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-23283) GroupWindowITCase.testWindowAggregateOnUpsertSource fails on azure
[
https://issues.apache.org/jira/browse/FLINK-23283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17376219#comment-17376219
]
JING ZHANG commented on FLINK-23283:
[~xintongsong] Sure, please assign to me.
> GroupWindowITCase.testWindowAggregateOnUpsertSource fails on azure
> --
>
> Key: FLINK-23283
> URL: https://issues.apache.org/jira/browse/FLINK-23283
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.14.0
>Reporter: Roman Khachatryan
>Priority: Blocker
> Fix For: 1.14.0
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=19978=logs=e25d5e7e-2a9c-5589-4940-0b638d75a414=a6e0f756-5bb9-5ea8-a468-5f60db442a29=7086
> {code}
> 2021-07-06T10:13:10.6091443Z Jul 06 10:13:10 [ERROR] Tests run: 48, Failures:
> 1, Errors: 0, Skipped: 0, Time elapsed: 14.945 s <<< FAILURE! - in
> org.apache.flink.table.planner.runtime.stream.sql.GroupWindowITCase
> 2021-07-06T10:13:10.6093667Z Jul 06 10:13:10 [ERROR]
> testWindowAggregateOnUpsertSource[StateBackend=HEAP, UseTimestampLtz =
> false](org.apache.flin*k.table.planner.runtime.stream.sql.GroupWindowITCase)
> Time elapsed: 0.329 s <<< FAILURE!
> 2021-07-06T10:13:10.6096965Z Jul 06 10:13:10 java.lang.AssertionError:
> expected: r,1,102,1970-01-01T00:00,1970-01-01T00:00:05,
> Yen,1,1,1970-01-01T00:00,1970-01-01T00:00:05)> but
> was: Dollar,1,104,1970-01-01T00:00,1970-01-01T00:00:05,
> Yen,1,1,1970-01-01T00:00,1970-01-01T00:00:05)>
> 2021-07-06T10:13:10.6098985Z Jul 06 10:13:10at
> org.junit.Assert.fail(Assert.java:89)
> 2021-07-06T10:13:10.6099695Z Jul 06 10:13:10at
> org.junit.Assert.failNotEquals(Assert.java:835)
> 2021-07-06T10:13:10.6100489Z Jul 06 10:13:10at
> org.junit.Assert.assertEquals(Assert.java:120)
> 2021-07-06T10:13:10.6101292Z Jul 06 10:13:10at
> org.junit.Assert.assertEquals(Assert.java:146)
> 2021-07-06T10:13:10.6102395Z Jul 06 10:13:10at
> org.apache.flink.table.planner.runtime.stream.sql.GroupWindowITCase.testWindowAggregateOnUpsert*Source(GroupWindowITCase.scala:421)
> {code}
> Fails locally 3 times out of 1000
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #16405: [FLINK-23165][python] Add StreamExecutionEnvironment#registerSlotSharingGroup to PyFlink
flinkbot edited a comment on pull request #16405: URL: https://github.com/apache/flink/pull/16405#issuecomment-875224279 ## CI report: * b7fc24e07e9fb4eb0ac2b9ac6e16e89cd9041aa7 UNKNOWN * e9eb7e2451023def24169564053f8a0eeb746176 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20046) * a4456c35f860b3fff7ef3e74a7e77df77d4c3786 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16379: [BP-1.13][FLINK-22722 / FLINK-22766][connector/kafka] Add docs and metrics for Kafka new source
flinkbot edited a comment on pull request #16379: URL: https://github.com/apache/flink/pull/16379#issuecomment-874137469 ## CI report: * ca0408634212c0c0ea1b4e7da66095fa10af2a04 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=19932) * 59cbbb4c0fd63984705c33dbf60d0efe0f39b6dc UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16348: [FLINK-16093][docs-zh]Translate "System Functions" page of "Functions" into Chinese
flinkbot edited a comment on pull request #16348: URL: https://github.com/apache/flink/pull/16348#issuecomment-872690776 ## CI report: * 81f69ba757726f2aaa4a0fce16e6cf412fd05f2a UNKNOWN * c07588d1c85496bdaa60c3cb6663ad25e443e8fa Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=19958) * 25af18e59b680ad66d9b0d98c2176ffbdc2e0214 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20052) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16341: [FLINK-21804][state/changelog] Create and wire changelog storage with state backend
flinkbot edited a comment on pull request #16341: URL: https://github.com/apache/flink/pull/16341#issuecomment-872173720 ## CI report: * 7f12f5e1bfd2866475d936cb7e54ad133b1517a9 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=19991) * 864fd641dbe6d5101e8f9ed4ef87c7d6c2b009b1 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16349: [FLINK-23267][table-planner-blink] Enable Java code splitting for all generated classes
flinkbot edited a comment on pull request #16349: URL: https://github.com/apache/flink/pull/16349#issuecomment-872790415 ## CI report: * 9747a5d6ae327f3078c97fb95020c1cffccf5e20 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=19976) * 37f9fc51058e1572a1e2d29d616c5f5db9dc46db UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16192: [FLINK-22954][table-planner-blink] Rewrite Join on constant TableFunctionScan to Correlate
flinkbot edited a comment on pull request #16192: URL: https://github.com/apache/flink/pull/16192#issuecomment-863860019 ## CI report: * a99d7b5b998336ed4ddf177ac4430461c9c78465 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=19951) * a54eb5522501dc04e800914ec5b2a3a77f430724 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20051) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (FLINK-14055) Add advanced function DDL syntax "USING JAR/FILE/ACHIVE"
[
https://issues.apache.org/jira/browse/FLINK-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17376208#comment-17376208
]
frank wang edited comment on FLINK-14055 at 7/7/21, 3:55 AM:
-
before, in our team, we implement function custom class loader, support from
hdfs load, and local load, maybe i can join this work to help do something
[~zhouqi] [~jark] [~ZhenqiuHuang]
was (Author: frank wang):
before, in our team, we implement function custom class loader, support from
hdfs load, and local load, maybe i can join this work to help do something
[~zhouqi] [~jark]
> Add advanced function DDL syntax "USING JAR/FILE/ACHIVE"
>
>
> Key: FLINK-14055
> URL: https://issues.apache.org/jira/browse/FLINK-14055
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / API
>Reporter: Bowen Li
>Priority: Major
> Labels: auto-unassigned, sprint
>
> As FLINK-7151 adds basic function DDL to Flink, this ticket is to support
> dynamically loading functions from external source in function DDL with
> advanced syntax like
>
> {code:java}
> CREATE FUNCTION func_name as class_name USING JAR/FILE/ACHIEVE 'xxx' [,
> JAR/FILE/ACHIEVE 'yyy'] ;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] SteNicholas commented on pull request #16405: [FLINK-23165][python] Add StreamExecutionEnvironment#registerSlotSharingGroup to PyFlink
SteNicholas commented on pull request #16405: URL: https://github.com/apache/flink/pull/16405#issuecomment-875253967 > Maybe we can complete the missing scala interface as well in this PR, with another commit. WDYT? @KarmaGYZ IMO, I would like to add scala interface into another commit in this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (FLINK-14055) Add advanced function DDL syntax "USING JAR/FILE/ACHIVE"
[
https://issues.apache.org/jira/browse/FLINK-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17376208#comment-17376208
]
frank wang edited comment on FLINK-14055 at 7/7/21, 3:53 AM:
-
before, in our team, we implement function custom class loader, support from
hdfs load, and local load, maybe i can join this work to help do something
[~zhouqi] [~jark]
was (Author: frank wang):
before, in our team, we implement function custom class loader, support from
hdfs load, and local load, maybe i can join this work to help do something
> Add advanced function DDL syntax "USING JAR/FILE/ACHIVE"
>
>
> Key: FLINK-14055
> URL: https://issues.apache.org/jira/browse/FLINK-14055
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / API
>Reporter: Bowen Li
>Priority: Major
> Labels: auto-unassigned, sprint
>
> As FLINK-7151 adds basic function DDL to Flink, this ticket is to support
> dynamically loading functions from external source in function DDL with
> advanced syntax like
>
> {code:java}
> CREATE FUNCTION func_name as class_name USING JAR/FILE/ACHIEVE 'xxx' [,
> JAR/FILE/ACHIEVE 'yyy'] ;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-14055) Add advanced function DDL syntax "USING JAR/FILE/ACHIVE"
[
https://issues.apache.org/jira/browse/FLINK-14055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17376208#comment-17376208
]
frank wang commented on FLINK-14055:
before, in our team, we implement function custom class loader, support from
hdfs load, and local load, maybe i can join this work to help do something
> Add advanced function DDL syntax "USING JAR/FILE/ACHIVE"
>
>
> Key: FLINK-14055
> URL: https://issues.apache.org/jira/browse/FLINK-14055
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / API
>Reporter: Bowen Li
>Priority: Major
> Labels: auto-unassigned, sprint
>
> As FLINK-7151 adds basic function DDL to Flink, this ticket is to support
> dynamically loading functions from external source in function DDL with
> advanced syntax like
>
> {code:java}
> CREATE FUNCTION func_name as class_name USING JAR/FILE/ACHIEVE 'xxx' [,
> JAR/FILE/ACHIEVE 'yyy'] ;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] KarmaGYZ commented on pull request #16405: [FLINK-23165][python] Add StreamExecutionEnvironment#registerSlotSharingGroup to PyFlink
KarmaGYZ commented on pull request #16405: URL: https://github.com/apache/flink/pull/16405#issuecomment-875251511 Maybe we can complete the missing scala interface as well in this PR, with another commit. WDYT? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-22317) Support DROP column/constraint/watermark for ALTER TABLE statement
[ https://issues.apache.org/jira/browse/FLINK-22317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17376204#comment-17376204 ] Jark Wu commented on FLINK-22317: - The sanity check sounds good to me. > Support DROP column/constraint/watermark for ALTER TABLE statement > -- > > Key: FLINK-22317 > URL: https://issues.apache.org/jira/browse/FLINK-22317 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API >Reporter: Jark Wu >Assignee: Aiden Gong >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] SteNicholas edited a comment on pull request #16405: [FLINK-23165][python] Add StreamExecutionEnvironment#registerSlotSharingGroup to PyFlink
SteNicholas edited a comment on pull request #16405: URL: https://github.com/apache/flink/pull/16405#issuecomment-875249416 > The testing fails in format checking. Would you like to fix it? @KarmaGYZ I have fixed the checkstyle of the `slot_sharing_group.py`. @HuangXingBo please help to review the changes of this pull request. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] SteNicholas edited a comment on pull request #16405: [FLINK-23165][python] Add StreamExecutionEnvironment#registerSlotSharingGroup to PyFlink
SteNicholas edited a comment on pull request #16405: URL: https://github.com/apache/flink/pull/16405#issuecomment-875250917 > Just out of curiosity, is there any tool that can automatically reformat the python code? @KarmaGYZ No automatically reformat tool for PyFlink like Maven Spotless. Only use `lint-python.sh` to validate the checkstyle and the tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] SteNicholas commented on pull request #16405: [FLINK-23165][python] Add StreamExecutionEnvironment#registerSlotSharingGroup to PyFlink
SteNicholas commented on pull request #16405: URL: https://github.com/apache/flink/pull/16405#issuecomment-875250917 > Just out of curiosity, is there any tool that can automatically reformat the python code? No automatically reformat tool for PyFlink like Maven Spotless. Only use `lint-python.sh` to validate the checkstyle and the tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] KarmaGYZ commented on pull request #16405: [FLINK-23165][python] Add StreamExecutionEnvironment#registerSlotSharingGroup to PyFlink
KarmaGYZ commented on pull request #16405: URL: https://github.com/apache/flink/pull/16405#issuecomment-875250217 Just out of curiosity, is there any tool that can automatically reformat the python code? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] SteNicholas commented on pull request #16405: [FLINK-23165][python] Add StreamExecutionEnvironment#registerSlotSharingGroup to PyFlink
SteNicholas commented on pull request #16405: URL: https://github.com/apache/flink/pull/16405#issuecomment-875249416 > The testing fails in format checking. Would you like to fix it? I have fixed the checkstyle of the `slot_sharing_group.py`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-23289) BinarySection should null check in constructor method
[
https://issues.apache.org/jira/browse/FLINK-23289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Terry Wang updated FLINK-23289:
---
Component/s: Table SQL / Runtime
> BinarySection should null check in constructor method
> -
>
> Key: FLINK-23289
> URL: https://issues.apache.org/jira/browse/FLINK-23289
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Runtime
>Reporter: Terry Wang
>Priority: Major
>
> {code:java}
> Caused by: java.lang.NullPointerException
> at
> org.apache.flink.table.data.binary.BinarySegmentUtils.inFirstSegment(BinarySegmentUtils.java:411)
> at
> org.apache.flink.table.data.binary.BinarySegmentUtils.copyToBytes(BinarySegmentUtils.java:132)
> at
> org.apache.flink.table.data.binary.BinarySegmentUtils.copyToBytes(BinarySegmentUtils.java:118)
> at
> org.apache.flink.table.data.binary.BinaryStringData.copy(BinaryStringData.java:360)
> at
> org.apache.flink.table.runtime.typeutils.StringDataSerializer.copy(StringDataSerializer.java:59)
> at
> org.apache.flink.table.runtime.typeutils.StringDataSerializer.copy(StringDataSerializer.java:37)
> at
> org.apache.flink.table.runtime.typeutils.ArrayDataSerializer.copyGenericArray(ArrayDataSerializer.java:128)
> at
> org.apache.flink.table.runtime.typeutils.ArrayDataSerializer.copy(ArrayDataSerializer.java:86)
> at
> org.apache.flink.table.runtime.typeutils.ArrayDataSerializer.copy(ArrayDataSerializer.java:47)
> at
> org.apache.flink.table.runtime.typeutils.RowDataSerializer.copyRowData(RowDataSerializer.java:170)
> at
> org.apache.flink.table.runtime.typeutils.RowDataSerializer.copy(RowDataSerializer.java:131)
> at
> org.apache.flink.table.runtime.typeutils.RowDataSerializer.copy(RowDataSerializer.java:48)
> at
> org.apache.flink.table.runtime.operators.join.lookup.AsyncLookupJoinWithCalcRunner$CalcCollectionCollector.collect(AsyncLookupJoinWithCalcRunner.java:152)
> at
> org.apache.flink.table.runtime.operators.join.lookup.AsyncLookupJoinWithCalcRunner$CalcCollectionCollector.collect(AsyncLookupJoinWithCalcRunner.java:142)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-23289) BinarySection should null check in contusctor method
Terry Wang created FLINK-23289:
--
Summary: BinarySection should null check in contusctor method
Key: FLINK-23289
URL: https://issues.apache.org/jira/browse/FLINK-23289
Project: Flink
Issue Type: Improvement
Reporter: Terry Wang
{code:java}
Caused by: java.lang.NullPointerException
at
org.apache.flink.table.data.binary.BinarySegmentUtils.inFirstSegment(BinarySegmentUtils.java:411)
at
org.apache.flink.table.data.binary.BinarySegmentUtils.copyToBytes(BinarySegmentUtils.java:132)
at
org.apache.flink.table.data.binary.BinarySegmentUtils.copyToBytes(BinarySegmentUtils.java:118)
at
org.apache.flink.table.data.binary.BinaryStringData.copy(BinaryStringData.java:360)
at
org.apache.flink.table.runtime.typeutils.StringDataSerializer.copy(StringDataSerializer.java:59)
at
org.apache.flink.table.runtime.typeutils.StringDataSerializer.copy(StringDataSerializer.java:37)
at
org.apache.flink.table.runtime.typeutils.ArrayDataSerializer.copyGenericArray(ArrayDataSerializer.java:128)
at
org.apache.flink.table.runtime.typeutils.ArrayDataSerializer.copy(ArrayDataSerializer.java:86)
at
org.apache.flink.table.runtime.typeutils.ArrayDataSerializer.copy(ArrayDataSerializer.java:47)
at
org.apache.flink.table.runtime.typeutils.RowDataSerializer.copyRowData(RowDataSerializer.java:170)
at
org.apache.flink.table.runtime.typeutils.RowDataSerializer.copy(RowDataSerializer.java:131)
at
org.apache.flink.table.runtime.typeutils.RowDataSerializer.copy(RowDataSerializer.java:48)
at
org.apache.flink.table.runtime.operators.join.lookup.AsyncLookupJoinWithCalcRunner$CalcCollectionCollector.collect(AsyncLookupJoinWithCalcRunner.java:152)
at
org.apache.flink.table.runtime.operators.join.lookup.AsyncLookupJoinWithCalcRunner$CalcCollectionCollector.collect(AsyncLookupJoinWithCalcRunner.java:142)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-23289) BinarySection should null check in constructor method
[
https://issues.apache.org/jira/browse/FLINK-23289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Terry Wang updated FLINK-23289:
---
Summary: BinarySection should null check in constructor method (was:
BinarySection should null check in contusctor method)
> BinarySection should null check in constructor method
> -
>
> Key: FLINK-23289
> URL: https://issues.apache.org/jira/browse/FLINK-23289
> Project: Flink
> Issue Type: Improvement
>Reporter: Terry Wang
>Priority: Major
>
> {code:java}
> Caused by: java.lang.NullPointerException
> at
> org.apache.flink.table.data.binary.BinarySegmentUtils.inFirstSegment(BinarySegmentUtils.java:411)
> at
> org.apache.flink.table.data.binary.BinarySegmentUtils.copyToBytes(BinarySegmentUtils.java:132)
> at
> org.apache.flink.table.data.binary.BinarySegmentUtils.copyToBytes(BinarySegmentUtils.java:118)
> at
> org.apache.flink.table.data.binary.BinaryStringData.copy(BinaryStringData.java:360)
> at
> org.apache.flink.table.runtime.typeutils.StringDataSerializer.copy(StringDataSerializer.java:59)
> at
> org.apache.flink.table.runtime.typeutils.StringDataSerializer.copy(StringDataSerializer.java:37)
> at
> org.apache.flink.table.runtime.typeutils.ArrayDataSerializer.copyGenericArray(ArrayDataSerializer.java:128)
> at
> org.apache.flink.table.runtime.typeutils.ArrayDataSerializer.copy(ArrayDataSerializer.java:86)
> at
> org.apache.flink.table.runtime.typeutils.ArrayDataSerializer.copy(ArrayDataSerializer.java:47)
> at
> org.apache.flink.table.runtime.typeutils.RowDataSerializer.copyRowData(RowDataSerializer.java:170)
> at
> org.apache.flink.table.runtime.typeutils.RowDataSerializer.copy(RowDataSerializer.java:131)
> at
> org.apache.flink.table.runtime.typeutils.RowDataSerializer.copy(RowDataSerializer.java:48)
> at
> org.apache.flink.table.runtime.operators.join.lookup.AsyncLookupJoinWithCalcRunner$CalcCollectionCollector.collect(AsyncLookupJoinWithCalcRunner.java:152)
> at
> org.apache.flink.table.runtime.operators.join.lookup.AsyncLookupJoinWithCalcRunner$CalcCollectionCollector.collect(AsyncLookupJoinWithCalcRunner.java:142)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] KarmaGYZ commented on pull request #16405: [FLINK-23165][python] Add StreamExecutionEnvironment#registerSlotSharingGroup to PyFlink
KarmaGYZ commented on pull request #16405: URL: https://github.com/apache/flink/pull/16405#issuecomment-875248419 The testing fails in format checking. Would you like to fix it? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] KarmaGYZ commented on a change in pull request #16405: [FLINK-23165][python] Add StreamExecutionEnvironment#registerSlotSharingGroup to PyFlink
KarmaGYZ commented on a change in pull request #16405: URL: https://github.com/apache/flink/pull/16405#discussion_r665000412 ## File path: flink-python/pyflink/datastream/slot_sharing_group.py ## @@ -0,0 +1,282 @@ + +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +__all__ = ['MemorySize', 'SlotSharingGroup'] + +from typing import Optional + +from pyflink.java_gateway import get_gateway + + +class MemorySize(object): +""" +MemorySize is a representation of a number of bytes, viewable in different units. +""" + +def __init__(self, j_memory_size=None, bytes_size: int = None): +self._j_memory_size = get_gateway().jvm \ +.org.apache.flink.configuration.MemorySize(bytes_size) \ +if j_memory_size is None else j_memory_size + +@staticmethod +def of_mebi_bytes(mebi_bytes: int) -> 'MemorySize': +return MemorySize( + get_gateway().jvm.org.apache.flink.configuration.MemorySize.ofMebiBytes(mebi_bytes)) + +def get_bytes(self) -> int: +""" +Gets the memory size in bytes. + +:return: The memory size in bytes. +""" +return self._j_memory_size.getBytes() + +def get_kibi_bytes(self) -> int: +""" +Gets the memory size in Kibibytes (= 1024 bytes). + +:return: The memory size in Kibibytes. +""" +return self._j_memory_size.getKibiBytes() + +def get_mebi_bytes(self) -> int: +""" +Gets the memory size in Mebibytes (= 1024 Kibibytes). + +:return: The memory size in Mebibytes. +""" +return self._j_memory_size.getMebiBytes() + +def get_gibi_bytes(self) -> int: +""" +Gets the memory size in Gibibytes (= 1024 Mebibytes). + +:return: The memory size in Gibibytes. +""" +return self._j_memory_size.getGibiBytes() + +def get_tebi_bytes(self) -> int: +""" +Gets the memory size in Tebibytes (= 1024 Gibibytes). + +:return: The memory size in Tebibytes. +""" +return self._j_memory_size.getTebiBytes() + +def get_java_memory_size(self): +""" +Get the Java MemorySize object. + +:return: The Java MemorySize object. +""" +return self._j_memory_size + +def __eq__(self, other): +return isinstance(other, self.__class__) and \ + self._j_memory_size == other._j_memory_size + +def __hash__(self): +return self._j_memory_size.hashCode() + + +class SlotSharingGroup(object): +""" +Describe the name and the the different resource components of a slot sharing group. +""" + +def __init__(self, j_slot_sharing_group): +self._j_slot_sharing_group = j_slot_sharing_group + +def get_name(self) -> str: +""" +Get the name of this SlotSharingGroup. + +:return: The name of the SlotSharingGroup. +""" +return self._j_slot_sharing_group.getName() + +def get_managed_memory(self) -> Optional[MemorySize]: +""" +Get the task managed memory for this SlotSharingGroup. + +:return: The task managed memory of the SlotSharingGroup. +""" +managed_memory = self._j_slot_sharing_group.getManagedMemory() +return MemorySize(managed_memory.get()) if managed_memory.isPresent() else None + +def get_task_heap_memory(self) -> Optional[MemorySize]: +""" +Get the task heap memory for this SlotSharingGroup. + +:return: The task heap memory of the SlotSharingGroup. +""" +task_heap_memory = self._j_slot_sharing_group.getTaskHeapMemory() +return MemorySize(task_heap_memory.get()) if task_heap_memory.isPresent() else None + +def get_task_off_heap_memory(self) -> Optional[MemorySize]: +""" +Get the task off-heap memory for this SlotSharingGroup. + +:return: The task off-heap memory of the SlotSharingGroup. +""" +task_off_heap_memory =
[GitHub] [flink] flinkbot edited a comment on pull request #16405: [FLINK-23165][python] Add StreamExecutionEnvironment#registerSlotSharingGroup to PyFlink
flinkbot edited a comment on pull request #16405: URL: https://github.com/apache/flink/pull/16405#issuecomment-875224279 ## CI report: * b7fc24e07e9fb4eb0ac2b9ac6e16e89cd9041aa7 UNKNOWN * e9eb7e2451023def24169564053f8a0eeb746176 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20046) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16406: [FLINK-23184][table-runtime] Fix compile error in code generation of unary plus and minus
flinkbot edited a comment on pull request #16406: URL: https://github.com/apache/flink/pull/16406#issuecomment-875233094 ## CI report: * b94f97fe8cfc71e1da286bed37b5a684eaed9a96 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20050) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16359: [FLINK-23222]Translate page 'Application Profiling & Debugging' into Chinese
flinkbot edited a comment on pull request #16359: URL: https://github.com/apache/flink/pull/16359#issuecomment-872999229 ## CI report: * 069fb525e2734b4854fc4c829730030c187e21c1 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=19844) * f38e1fff477d7520a34e21ec20d9ede7b621f522 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20048) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16369: [FLINK-22443][streaming-java] Fix overflow in MultipleInputSelectionHandler
flinkbot edited a comment on pull request #16369: URL: https://github.com/apache/flink/pull/16369#issuecomment-873893547 ## CI report: * 3bb34b002d5a8952f045088b16f6a4c6a0532527 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=19917) * 11105a90383651025185cdc1804733c19e59b8ca Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20049) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16348: [FLINK-16093][docs-zh]Translate "System Functions" page of "Functions" into Chinese
flinkbot edited a comment on pull request #16348: URL: https://github.com/apache/flink/pull/16348#issuecomment-872690776 ## CI report: * 81f69ba757726f2aaa4a0fce16e6cf412fd05f2a UNKNOWN * c07588d1c85496bdaa60c3cb6663ad25e443e8fa Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=19958) * 25af18e59b680ad66d9b0d98c2176ffbdc2e0214 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16192: [FLINK-22954][table-planner-blink] Rewrite Join on constant TableFunctionScan to Correlate
flinkbot edited a comment on pull request #16192: URL: https://github.com/apache/flink/pull/16192#issuecomment-863860019 ## CI report: * a99d7b5b998336ed4ddf177ac4430461c9c78465 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=19951) * a54eb5522501dc04e800914ec5b2a3a77f430724 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] TsReaper commented on a change in pull request #16349: [FLINK-23267][table-planner-blink] Enable Java code splitting for all generated classes
TsReaper commented on a change in pull request #16349: URL: https://github.com/apache/flink/pull/16349#discussion_r665014150 ## File path: flink-table/flink-table-code-splitter/src/main/resources/META-INF/NOTICE ## @@ -7,7 +7,7 @@ The Apache Software Foundation (http://www.apache.org/). This project bundles the following dependencies under the BSD 3-clause license. See bundled license files for details. -- antlr:4.7 +- org.antlr:antlr4-runtime:4.7 Review comment: Grammar files are also mentioned in this NOTICE file. This `org.antlr.grammars:java` does not seem to exist in maven repository, so we have to copy the grammar files into Flink. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-22889) JdbcExactlyOnceSinkE2eTest hangs on azure
[ https://issues.apache.org/jira/browse/FLINK-22889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17376191#comment-17376191 ] Xintong Song commented on FLINK-22889: -- https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=20036=logs=3d12d40f-c62d-5ec4-6acc-0efe94cc3e89=5d6e4255-0ea8-5e2a-f52c-c881b7872361=15741 > JdbcExactlyOnceSinkE2eTest hangs on azure > - > > Key: FLINK-22889 > URL: https://issues.apache.org/jira/browse/FLINK-22889 > Project: Flink > Issue Type: Bug > Components: Connectors / JDBC >Affects Versions: 1.14.0, 1.13.1 >Reporter: Dawid Wysakowicz >Assignee: Roman Khachatryan >Priority: Critical > Labels: pull-request-available, test-stability > > https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=18690=logs=ba53eb01-1462-56a3-8e98-0dd97fbcaab5=bfbc6239-57a0-5db0-63f3-41551b4f7d51=16658 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-22333) Elasticsearch7DynamicSinkITCase.testWritingDocuments failed due to deploy task timeout.
[
https://issues.apache.org/jira/browse/FLINK-22333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17376188#comment-17376188
]
Xintong Song commented on FLINK-22333:
--
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=20036=logs=ba53eb01-1462-56a3-8e98-0dd97fbcaab5=bfbc6239-57a0-5db0-63f3-41551b4f7d51=12401
> Elasticsearch7DynamicSinkITCase.testWritingDocuments failed due to deploy
> task timeout.
> ---
>
> Key: FLINK-22333
> URL: https://issues.apache.org/jira/browse/FLINK-22333
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
>Affects Versions: 1.13.0
>Reporter: Guowei Ma
>Priority: Major
> Labels: test-stability
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16694=logs=d44f43ce-542c-597d-bf94-b0718c71e5e8=03dca39c-73e8-5aaf-601d-328ae5c35f20=12329
> {code:java}
> 2021-04-16T23:37:23.5719280Z Apr 16 23:37:23
> org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
> 2021-04-16T23:37:23.5739250Z Apr 16 23:37:23 at
> org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:144)
> 2021-04-16T23:37:23.5759329Z Apr 16 23:37:23 at
> org.apache.flink.runtime.minicluster.MiniClusterJobClient.lambda$getJobExecutionResult$3(MiniClusterJobClient.java:137)
> 2021-04-16T23:37:23.5779145Z Apr 16 23:37:23 at
> java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616)
> 2021-04-16T23:37:23.5799204Z Apr 16 23:37:23 at
> java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591)
> 2021-04-16T23:37:23.5819302Z Apr 16 23:37:23 at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> 2021-04-16T23:37:23.5839106Z Apr 16 23:37:23 at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
> 2021-04-16T23:37:23.5859276Z Apr 16 23:37:23 at
> org.apache.flink.runtime.rpc.akka.AkkaInvocationHandler.lambda$invokeRpc$0(AkkaInvocationHandler.java:237)
> 2021-04-16T23:37:23.5868964Z Apr 16 23:37:23 at
> java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
> 2021-04-16T23:37:23.5869925Z Apr 16 23:37:23 at
> java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
> 2021-04-16T23:37:23.5919839Z Apr 16 23:37:23 at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> 2021-04-16T23:37:23.5959562Z Apr 16 23:37:23 at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
> 2021-04-16T23:37:23.5989732Z Apr 16 23:37:23 at
> org.apache.flink.runtime.concurrent.FutureUtils$1.onComplete(FutureUtils.java:1081)
> 2021-04-16T23:37:23.6019422Z Apr 16 23:37:23 at
> akka.dispatch.OnComplete.internal(Future.scala:264)
> 2021-04-16T23:37:23.6039067Z Apr 16 23:37:23 at
> akka.dispatch.OnComplete.internal(Future.scala:261)
> 2021-04-16T23:37:23.6060126Z Apr 16 23:37:23 at
> akka.dispatch.japi$CallbackBridge.apply(Future.scala:191)
> 2021-04-16T23:37:23.6089258Z Apr 16 23:37:23 at
> akka.dispatch.japi$CallbackBridge.apply(Future.scala:188)
> 2021-04-16T23:37:23.6119150Z Apr 16 23:37:23 at
> scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
> 2021-04-16T23:37:23.6139149Z Apr 16 23:37:23 at
> org.apache.flink.runtime.concurrent.Executors$DirectExecutionContext.execute(Executors.java:73)
> 2021-04-16T23:37:23.6159077Z Apr 16 23:37:23 at
> scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)
> 2021-04-16T23:37:23.6189432Z Apr 16 23:37:23 at
> scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)
> 2021-04-16T23:37:23.6215243Z Apr 16 23:37:23 at
> akka.pattern.PromiseActorRef.$bang(AskSupport.scala:572)
> 2021-04-16T23:37:23.6219148Z Apr 16 23:37:23 at
> akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:22)
> 2021-04-16T23:37:23.6220221Z Apr 16 23:37:23 at
> akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:21)
> 2021-04-16T23:37:23.6249411Z Apr 16 23:37:23 at
> scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:436)
> 2021-04-16T23:37:23.6259145Z Apr 16 23:37:23 at
> scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:435)
> 2021-04-16T23:37:23.6289272Z Apr 16 23:37:23 at
> scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
> 2021-04-16T23:37:23.6309243Z Apr 16 23:37:23 at
> akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
> 2021-04-16T23:37:23.6359306Z Apr 16 23:37:23 at
> akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:91)
> 2021-04-16T23:37:23.6369399Z Apr 16 23:37:23 at
[jira] [Commented] (FLINK-20329) Elasticsearch7DynamicSinkITCase hangs
[
https://issues.apache.org/jira/browse/FLINK-20329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17376187#comment-17376187
]
Xintong Song commented on FLINK-20329:
--
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=20036=logs=e1276d0f-df12-55ec-86b5-c0ad597d83c9=906e9244-f3be-5604-1979-e767c8a6f6d9=11897
> Elasticsearch7DynamicSinkITCase hangs
> -
>
> Key: FLINK-20329
> URL: https://issues.apache.org/jira/browse/FLINK-20329
> Project: Flink
> Issue Type: Bug
> Components: Connectors / ElasticSearch
>Affects Versions: 1.12.0, 1.13.0
>Reporter: Dian Fu
>Assignee: Yangze Guo
>Priority: Major
> Labels: pull-request-available, test-stability
> Fix For: 1.14.0
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=10052=logs=d44f43ce-542c-597d-bf94-b0718c71e5e8=03dca39c-73e8-5aaf-601d-328ae5c35f20
> {code}
> 2020-11-24T16:04:05.9260517Z [INFO] Running
> org.apache.flink.streaming.connectors.elasticsearch.table.Elasticsearch7DynamicSinkITCase
> 2020-11-24T16:19:25.5481231Z
> ==
> 2020-11-24T16:19:25.5483549Z Process produced no output for 900 seconds.
> 2020-11-24T16:19:25.5484064Z
> ==
> 2020-11-24T16:19:25.5484498Z
> ==
> 2020-11-24T16:19:25.5484882Z The following Java processes are running (JPS)
> 2020-11-24T16:19:25.5485475Z
> ==
> 2020-11-24T16:19:25.5694497Z Picked up JAVA_TOOL_OPTIONS:
> -XX:+HeapDumpOnOutOfMemoryError
> 2020-11-24T16:19:25.7263048Z 16192 surefirebooter5057948964630155904.jar
> 2020-11-24T16:19:25.7263515Z 18566 Jps
> 2020-11-24T16:19:25.7263709Z 959 Launcher
> 2020-11-24T16:19:25.7411148Z
> ==
> 2020-11-24T16:19:25.7427013Z Printing stack trace of Java process 16192
> 2020-11-24T16:19:25.7427369Z
> ==
> 2020-11-24T16:19:25.7484365Z Picked up JAVA_TOOL_OPTIONS:
> -XX:+HeapDumpOnOutOfMemoryError
> 2020-11-24T16:19:26.0848776Z 2020-11-24 16:19:26
> 2020-11-24T16:19:26.0849578Z Full thread dump OpenJDK 64-Bit Server VM
> (25.275-b01 mixed mode):
> 2020-11-24T16:19:26.0849831Z
> 2020-11-24T16:19:26.0850185Z "Attach Listener" #32 daemon prio=9 os_prio=0
> tid=0x7fc148001000 nid=0x48e7 waiting on condition [0x]
> 2020-11-24T16:19:26.0850595Zjava.lang.Thread.State: RUNNABLE
> 2020-11-24T16:19:26.0850814Z
> 2020-11-24T16:19:26.0851375Z "testcontainers-ryuk" #31 daemon prio=5
> os_prio=0 tid=0x7fc251232000 nid=0x3fb0 in Object.wait()
> [0x7fc1012c4000]
> 2020-11-24T16:19:26.0854688Zjava.lang.Thread.State: TIMED_WAITING (on
> object monitor)
> 2020-11-24T16:19:26.0855379Z at java.lang.Object.wait(Native Method)
> 2020-11-24T16:19:26.0855844Z at
> org.testcontainers.utility.ResourceReaper.lambda$null$1(ResourceReaper.java:142)
> 2020-11-24T16:19:26.0857272Z - locked <0x8e2bd2d0> (a
> java.util.ArrayList)
> 2020-11-24T16:19:26.0857977Z at
> org.testcontainers.utility.ResourceReaper$$Lambda$93/1981729428.run(Unknown
> Source)
> 2020-11-24T16:19:26.0858471Z at
> org.rnorth.ducttape.ratelimits.RateLimiter.doWhenReady(RateLimiter.java:27)
> 2020-11-24T16:19:26.0858961Z at
> org.testcontainers.utility.ResourceReaper.lambda$start$2(ResourceReaper.java:133)
> 2020-11-24T16:19:26.0859422Z at
> org.testcontainers.utility.ResourceReaper$$Lambda$92/40191541.run(Unknown
> Source)
> 2020-11-24T16:19:26.0859788Z at java.lang.Thread.run(Thread.java:748)
> 2020-11-24T16:19:26.0860030Z
> 2020-11-24T16:19:26.0860371Z "process reaper" #24 daemon prio=10 os_prio=0
> tid=0x7fc0f803b800 nid=0x3f92 waiting on condition [0x7fc10296e000]
> 2020-11-24T16:19:26.0860913Zjava.lang.Thread.State: TIMED_WAITING
> (parking)
> 2020-11-24T16:19:26.0861387Z at sun.misc.Unsafe.park(Native Method)
> 2020-11-24T16:19:26.0862495Z - parking to wait for <0x8814bf30> (a
> java.util.concurrent.SynchronousQueue$TransferStack)
> 2020-11-24T16:19:26.0863253Z at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> 2020-11-24T16:19:26.0863760Z at
> java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
> 2020-11-24T16:19:26.0864274Z at
> java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
> 2020-11-24T16:19:26.0864762Z at
> java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
> 2020-11-24T16:19:26.0865299Z
[GitHub] [flink] Zakelly commented on a change in pull request #16341: [FLINK-21804][state/changelog] Create and wire changelog storage with state backend
Zakelly commented on a change in pull request #16341:
URL: https://github.com/apache/flink/pull/16341#discussion_r665011824
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/state/TaskExecutorStateChangelogStoragesManager.java
##
@@ -0,0 +1,143 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.state;
+
+import org.apache.flink.api.common.JobID;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.runtime.state.changelog.StateChangelogStorage;
+import org.apache.flink.runtime.state.changelog.StateChangelogStorageLoader;
+import org.apache.flink.util.ShutdownHookUtil;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import javax.annotation.Nonnull;
+
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * This class holds the all {@link StateChangelogStorage} objects for a task
executor (manager). No
+ * thread-safe.
+ */
+public class TaskExecutorStateChangelogStoragesManager {
+
+/** Logger for this class. */
+private static final Logger LOG =
+
LoggerFactory.getLogger(TaskExecutorStateChangelogStoragesManager.class);
+
+/**
+ * This map holds all state changelog storages for tasks running on the
task manager / executor
+ * that own the instance of this. Maps from job id to all the subtask's
state changelog
+ * storages.
+ */
+private final Map>
changelogStoragesByJobId;
+
+private boolean closed;
+
+/** shutdown hook for this manager. */
+private final Thread shutdownHook;
+
+public TaskExecutorStateChangelogStoragesManager() {
+this.changelogStoragesByJobId = new HashMap<>();
+this.closed = false;
+
+// register a shutdown hook
+this.shutdownHook =
+ShutdownHookUtil.addShutdownHook(this::shutdown,
getClass().getSimpleName(), LOG);
+}
+
+public StateChangelogStorage stateChangelogStorageForJob(
+@Nonnull JobID jobId, Configuration configuration) {
+if (closed) {
+throw new IllegalStateException(
+"TaskExecutorStateChangelogStoragesManager is already
closed and cannot "
++ "register a new StateChangelogStorage.");
+}
+
+StateChangelogStorage stateChangelogStorage =
changelogStoragesByJobId.get(jobId);
+
+if (stateChangelogStorage == null) {
+stateChangelogStorage =
StateChangelogStorageLoader.load(configuration);
Review comment:
Yeah. I think it's better to change the value type for map
```changelogStoragesByJobId``` as
```AtomicReference>```, then we could record the
previous loaded ```null``` value. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] wsry commented on a change in pull request #11877: [FLINK-16641][network] Announce sender's backlog to solve the deadlock issue without exclusive buffers
wsry commented on a change in pull request #11877:
URL: https://github.com/apache/flink/pull/11877#discussion_r665012443
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/PartitionRequestQueue.java
##
@@ -164,6 +168,27 @@ void addCreditOrResumeConsumption(
}
}
+/**
+ * Announces remaining backlog to the consumer after the available data
notification or data
+ * consumption resumption.
+ */
+private void announceBacklog(NetworkSequenceViewReader reader) {
+int backlog = reader.getRemainingBacklog();
+if (backlog > 0) {
Review comment:
You are right, we can do some optimization here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (FLINK-23233) OperatorEventSendingCheckpointITCase.testOperatorEventLostWithReaderFailure fails on azure
[
https://issues.apache.org/jira/browse/FLINK-23233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-23233:
-
Fix Version/s: 1.14.0
> OperatorEventSendingCheckpointITCase.testOperatorEventLostWithReaderFailure
> fails on azure
> --
>
> Key: FLINK-23233
> URL: https://issues.apache.org/jira/browse/FLINK-23233
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing
>Affects Versions: 1.14.0, 1.13.1
>Reporter: Xintong Song
>Assignee: Yun Gao
>Priority: Blocker
> Fix For: 1.14.0, 1.13.2
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=19857=logs=4d4a0d10-fca2-5507-8eed-c07f0bdf4887=c2734c79-73b6-521c-e85a-67c7ecae9107=9382
> {code}
> Jul 03 01:37:31 [ERROR] Tests run: 4, Failures: 1, Errors: 0, Skipped: 0,
> Time elapsed: 21.415 s <<< FAILURE! - in
> org.apache.flink.runtime.operators.coordination.OperatorEventSendingCheckpointITCase
> Jul 03 01:37:31 [ERROR]
> testOperatorEventLostWithReaderFailure(org.apache.flink.runtime.operators.coordination.OperatorEventSendingCheckpointITCase)
> Time elapsed: 3.623 s <<< FAILURE!
> Jul 03 01:37:31 java.lang.AssertionError: expected:<[1, 2, 3, 4, 5, 6, 7, 8,
> 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27,
> 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46,
> 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65,
> 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84,
> 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100]> but
> was:<[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20,
> 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39,
> 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58,
> 59, 60, 61, 62, 63, 64, 65, 66, 67]>
> Jul 03 01:37:31 at org.junit.Assert.fail(Assert.java:88)
> Jul 03 01:37:31 at org.junit.Assert.failNotEquals(Assert.java:834)
> Jul 03 01:37:31 at org.junit.Assert.assertEquals(Assert.java:118)
> Jul 03 01:37:31 at org.junit.Assert.assertEquals(Assert.java:144)
> Jul 03 01:37:31 at
> org.apache.flink.runtime.operators.coordination.OperatorEventSendingCheckpointITCase.runTest(OperatorEventSendingCheckpointITCase.java:254)
> Jul 03 01:37:31 at
> org.apache.flink.runtime.operators.coordination.OperatorEventSendingCheckpointITCase.testOperatorEventLostWithReaderFailure(OperatorEventSendingCheckpointITCase.java:143)
> Jul 03 01:37:31 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> Jul 03 01:37:31 at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> Jul 03 01:37:31 at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Jul 03 01:37:31 at java.lang.reflect.Method.invoke(Method.java:498)
> Jul 03 01:37:31 at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> Jul 03 01:37:31 at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> Jul 03 01:37:31 at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> Jul 03 01:37:31 at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> Jul 03 01:37:31 at
> org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45)
> Jul 03 01:37:31 at
> org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> Jul 03 01:37:31 at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> Jul 03 01:37:31 at
> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> Jul 03 01:37:31 at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> Jul 03 01:37:31 at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-23233) OperatorEventSendingCheckpointITCase.testOperatorEventLostWithReaderFailure fails on azure
[
https://issues.apache.org/jira/browse/FLINK-23233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-23233:
-
Priority: Blocker (was: Major)
> OperatorEventSendingCheckpointITCase.testOperatorEventLostWithReaderFailure
> fails on azure
> --
>
> Key: FLINK-23233
> URL: https://issues.apache.org/jira/browse/FLINK-23233
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing
>Affects Versions: 1.13.1
>Reporter: Xintong Song
>Assignee: Yun Gao
>Priority: Blocker
> Fix For: 1.13.2
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=19857=logs=4d4a0d10-fca2-5507-8eed-c07f0bdf4887=c2734c79-73b6-521c-e85a-67c7ecae9107=9382
> {code}
> Jul 03 01:37:31 [ERROR] Tests run: 4, Failures: 1, Errors: 0, Skipped: 0,
> Time elapsed: 21.415 s <<< FAILURE! - in
> org.apache.flink.runtime.operators.coordination.OperatorEventSendingCheckpointITCase
> Jul 03 01:37:31 [ERROR]
> testOperatorEventLostWithReaderFailure(org.apache.flink.runtime.operators.coordination.OperatorEventSendingCheckpointITCase)
> Time elapsed: 3.623 s <<< FAILURE!
> Jul 03 01:37:31 java.lang.AssertionError: expected:<[1, 2, 3, 4, 5, 6, 7, 8,
> 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27,
> 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46,
> 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65,
> 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84,
> 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100]> but
> was:<[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20,
> 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39,
> 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58,
> 59, 60, 61, 62, 63, 64, 65, 66, 67]>
> Jul 03 01:37:31 at org.junit.Assert.fail(Assert.java:88)
> Jul 03 01:37:31 at org.junit.Assert.failNotEquals(Assert.java:834)
> Jul 03 01:37:31 at org.junit.Assert.assertEquals(Assert.java:118)
> Jul 03 01:37:31 at org.junit.Assert.assertEquals(Assert.java:144)
> Jul 03 01:37:31 at
> org.apache.flink.runtime.operators.coordination.OperatorEventSendingCheckpointITCase.runTest(OperatorEventSendingCheckpointITCase.java:254)
> Jul 03 01:37:31 at
> org.apache.flink.runtime.operators.coordination.OperatorEventSendingCheckpointITCase.testOperatorEventLostWithReaderFailure(OperatorEventSendingCheckpointITCase.java:143)
> Jul 03 01:37:31 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> Jul 03 01:37:31 at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> Jul 03 01:37:31 at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Jul 03 01:37:31 at java.lang.reflect.Method.invoke(Method.java:498)
> Jul 03 01:37:31 at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> Jul 03 01:37:31 at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> Jul 03 01:37:31 at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> Jul 03 01:37:31 at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> Jul 03 01:37:31 at
> org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45)
> Jul 03 01:37:31 at
> org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> Jul 03 01:37:31 at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> Jul 03 01:37:31 at
> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> Jul 03 01:37:31 at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> Jul 03 01:37:31 at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-23233) OperatorEventSendingCheckpointITCase.testOperatorEventLostWithReaderFailure fails on azure
[
https://issues.apache.org/jira/browse/FLINK-23233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xintong Song updated FLINK-23233:
-
Affects Version/s: 1.14.0
> OperatorEventSendingCheckpointITCase.testOperatorEventLostWithReaderFailure
> fails on azure
> --
>
> Key: FLINK-23233
> URL: https://issues.apache.org/jira/browse/FLINK-23233
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing
>Affects Versions: 1.14.0, 1.13.1
>Reporter: Xintong Song
>Assignee: Yun Gao
>Priority: Blocker
> Fix For: 1.13.2
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=19857=logs=4d4a0d10-fca2-5507-8eed-c07f0bdf4887=c2734c79-73b6-521c-e85a-67c7ecae9107=9382
> {code}
> Jul 03 01:37:31 [ERROR] Tests run: 4, Failures: 1, Errors: 0, Skipped: 0,
> Time elapsed: 21.415 s <<< FAILURE! - in
> org.apache.flink.runtime.operators.coordination.OperatorEventSendingCheckpointITCase
> Jul 03 01:37:31 [ERROR]
> testOperatorEventLostWithReaderFailure(org.apache.flink.runtime.operators.coordination.OperatorEventSendingCheckpointITCase)
> Time elapsed: 3.623 s <<< FAILURE!
> Jul 03 01:37:31 java.lang.AssertionError: expected:<[1, 2, 3, 4, 5, 6, 7, 8,
> 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27,
> 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46,
> 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65,
> 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84,
> 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100]> but
> was:<[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20,
> 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39,
> 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58,
> 59, 60, 61, 62, 63, 64, 65, 66, 67]>
> Jul 03 01:37:31 at org.junit.Assert.fail(Assert.java:88)
> Jul 03 01:37:31 at org.junit.Assert.failNotEquals(Assert.java:834)
> Jul 03 01:37:31 at org.junit.Assert.assertEquals(Assert.java:118)
> Jul 03 01:37:31 at org.junit.Assert.assertEquals(Assert.java:144)
> Jul 03 01:37:31 at
> org.apache.flink.runtime.operators.coordination.OperatorEventSendingCheckpointITCase.runTest(OperatorEventSendingCheckpointITCase.java:254)
> Jul 03 01:37:31 at
> org.apache.flink.runtime.operators.coordination.OperatorEventSendingCheckpointITCase.testOperatorEventLostWithReaderFailure(OperatorEventSendingCheckpointITCase.java:143)
> Jul 03 01:37:31 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> Jul 03 01:37:31 at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> Jul 03 01:37:31 at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Jul 03 01:37:31 at java.lang.reflect.Method.invoke(Method.java:498)
> Jul 03 01:37:31 at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> Jul 03 01:37:31 at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> Jul 03 01:37:31 at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> Jul 03 01:37:31 at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> Jul 03 01:37:31 at
> org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45)
> Jul 03 01:37:31 at
> org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> Jul 03 01:37:31 at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> Jul 03 01:37:31 at
> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> Jul 03 01:37:31 at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> Jul 03 01:37:31 at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] Zakelly commented on a change in pull request #16341: [FLINK-21804][state/changelog] Create and wire changelog storage with state backend
Zakelly commented on a change in pull request #16341:
URL: https://github.com/apache/flink/pull/16341#discussion_r665011824
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/state/TaskExecutorStateChangelogStoragesManager.java
##
@@ -0,0 +1,143 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.state;
+
+import org.apache.flink.api.common.JobID;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.runtime.state.changelog.StateChangelogStorage;
+import org.apache.flink.runtime.state.changelog.StateChangelogStorageLoader;
+import org.apache.flink.util.ShutdownHookUtil;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import javax.annotation.Nonnull;
+
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * This class holds the all {@link StateChangelogStorage} objects for a task
executor (manager). No
+ * thread-safe.
+ */
+public class TaskExecutorStateChangelogStoragesManager {
+
+/** Logger for this class. */
+private static final Logger LOG =
+
LoggerFactory.getLogger(TaskExecutorStateChangelogStoragesManager.class);
+
+/**
+ * This map holds all state changelog storages for tasks running on the
task manager / executor
+ * that own the instance of this. Maps from job id to all the subtask's
state changelog
+ * storages.
+ */
+private final Map>
changelogStoragesByJobId;
+
+private boolean closed;
+
+/** shutdown hook for this manager. */
+private final Thread shutdownHook;
+
+public TaskExecutorStateChangelogStoragesManager() {
+this.changelogStoragesByJobId = new HashMap<>();
+this.closed = false;
+
+// register a shutdown hook
+this.shutdownHook =
+ShutdownHookUtil.addShutdownHook(this::shutdown,
getClass().getSimpleName(), LOG);
+}
+
+public StateChangelogStorage stateChangelogStorageForJob(
+@Nonnull JobID jobId, Configuration configuration) {
+if (closed) {
+throw new IllegalStateException(
+"TaskExecutorStateChangelogStoragesManager is already
closed and cannot "
++ "register a new StateChangelogStorage.");
+}
+
+StateChangelogStorage stateChangelogStorage =
changelogStoragesByJobId.get(jobId);
+
+if (stateChangelogStorage == null) {
+stateChangelogStorage =
StateChangelogStorageLoader.load(configuration);
Review comment:
Yeah. I think it's better to change the value type for map
```changelogStoragesByJobId``` as ```Reference>```,
then we could record the previous loaded ```null``` value. WDYT?
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/state/TaskExecutorStateChangelogStoragesManager.java
##
@@ -0,0 +1,143 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.state;
+
+import org.apache.flink.api.common.JobID;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.runtime.state.changelog.StateChangelogStorage;
+import org.apache.flink.runtime.state.changelog.StateChangelogStorageLoader;
+import org.apache.flink.util.ShutdownHookUtil;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import javax.annotation.Nonnull;
+
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * This class holds the all {@link
[GitHub] [flink] Zakelly commented on a change in pull request #16341: [FLINK-21804][state/changelog] Create and wire changelog storage with state backend
Zakelly commented on a change in pull request #16341:
URL: https://github.com/apache/flink/pull/16341#discussion_r665011729
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/state/changelog/StateChangelogStorageLoader.java
##
@@ -28,16 +34,63 @@
/** A thin wrapper around {@link PluginManager} to load {@link
StateChangelogStorage}. */
@Internal
public class StateChangelogStorageLoader {
-private final PluginManager pluginManager;
-public StateChangelogStorageLoader(PluginManager pluginManager) {
-this.pluginManager = pluginManager;
+private static final Logger LOG =
LoggerFactory.getLogger(StateChangelogStorageLoader.class);
+
+/**
+ * Mapping of state changelog storage identifier to the corresponding
storage factories,
+ * populated in {@link
StateChangelogStorageLoader#initialize(PluginManager)}.
+ */
+private static final HashMap
+STATE_CHANGELOG_STORAGE_FACTORIES = new HashMap<>();
+
+static {
+// Guarantee to trigger once.
+initialize(null);
}
-@SuppressWarnings({"rawtypes"})
-public Iterator load() {
-return concat(
-pluginManager.load(StateChangelogStorage.class),
-ServiceLoader.load(StateChangelogStorage.class).iterator());
+public static void initialize(PluginManager pluginManager) {
+STATE_CHANGELOG_STORAGE_FACTORIES.clear();
+Iterator iterator =
+pluginManager == null
+?
ServiceLoader.load(StateChangelogStorageFactory.class).iterator()
+: concat(
+
pluginManager.load(StateChangelogStorageFactory.class),
+
ServiceLoader.load(StateChangelogStorageFactory.class).iterator());
+iterator.forEachRemaining(
+factory -> {
+String identifier = factory.getIdentifier().toLowerCase();
+StateChangelogStorageFactory prev =
+STATE_CHANGELOG_STORAGE_FACTORIES.get(identifier);
+if (prev == null) {
+STATE_CHANGELOG_STORAGE_FACTORIES.put(identifier,
factory);
+} else {
+LOG.warn(
+"StateChangelogStorageLoader found duplicated
factory,"
++ " using {} instead of {} for name
{}.",
+prev.getClass().getName(),
+factory.getClass().getName(),
+identifier);
+}
+});
+LOG.info(
+"StateChangelogStorageLoader initialized with shortcut names
{{}}.",
+String.join(",", STATE_CHANGELOG_STORAGE_FACTORIES.keySet()));
+}
+
+public static StateChangelogStorage load(Configuration configuration) {
+final String identifier =
+configuration
+
.getString(CheckpointingOptions.STATE_CHANGE_LOG_STORAGE)
+.toLowerCase();
+
+StateChangelogStorageFactory factory =
STATE_CHANGELOG_STORAGE_FACTORIES.get(identifier);
+if (factory == null) {
+LOG.warn("Cannot find a factory for changelog storage with name
'{}'.", identifier);
+return null;
Review comment:
1. Currently there is a default value for ```STATE_CHANGE_LOG_STORAGE
``` so we won't run into this case. However, a ```Nullable``` mark here makes
sense. Will do.
2. I'd rather keep it return null here. User may disable changelog and
config a wrong value for ```STATE_CHANGE_LOG_STORAGE ``` (by some copy
operation or missing jars for factory). Since they have disable the changelog,
the changelog-related configuration should better not make the job FAIL. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (FLINK-23288) Inserting (1.378593404E9) (0.6047707965147558) to a double type, it will generate CodeGenException
[
https://issues.apache.org/jira/browse/FLINK-23288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caizhi Weng updated FLINK-23288:
Description:
{code:sql}
CREATE TABLE database5_t0(
`c0` DOUBLE , `c1` INTEGER , `c2` STRING
) WITH (
'connector' = 'filesystem',
'path' = 'hdfs:///tmp/database5_t0.csv',
'format' = 'csv'
)
INSERT OVERWRITE database5_t0(c0, c1, c2) VALUES(1.378593404E9, 1336919677,
'1969-12-31 20:29:41'), (0.6047707965147558, 1336919677, '1970-01-06 03:36:50')
{code}
*After excuting the sql above, will generate this errors, but mysql,pg,sqlite
won`t have the error:*
{code}
org.apache.flink.table.planner.codegen.CodeGenException: Incompatible types of
expression and result type.
Expression[GeneratedExpression(((org.apache.flink.table.data.DecimalData)
decimal$3),false,,DECIMAL(17, 16) NOT NULL,Some(0.6047707965147558))] type is
[DECIMAL(17, 16) NOT NULL], result type is [DOUBLE NOT NULL]
at
org.apache.flink.table.planner.codegen.ExprCodeGenerator$$anonfun$generateResultExpression$1.apply(ExprCodeGenerator.scala:312)
at
org.apache.flink.table.planner.codegen.ExprCodeGenerator$$anonfun$generateResultExpression$1.apply(ExprCodeGenerator.scala:300)
at
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at
org.apache.flink.table.planner.codegen.ExprCodeGenerator.generateResultExpression(ExprCodeGenerator.scala:300)
at
org.apache.flink.table.planner.codegen.ExprCodeGenerator.generateResultExpression(ExprCodeGenerator.scala:256)
at
org.apache.flink.table.planner.codegen.ValuesCodeGenerator$$anonfun$1.apply(ValuesCodeGenerator.scala:44)
at
org.apache.flink.table.planner.codegen.ValuesCodeGenerator$$anonfun$1.apply(ValuesCodeGenerator.scala:43)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.Iterator$class.foreach(Iterator.scala:891)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.AbstractTraversable.map(Traversable.scala:104)
at
org.apache.flink.table.planner.codegen.ValuesCodeGenerator$.generatorInputFormat(ValuesCodeGenerator.scala:43)
at
org.apache.flink.table.planner.codegen.ValuesCodeGenerator.generatorInputFormat(ValuesCodeGenerator.scala)
at
org.apache.flink.table.planner.plan.nodes.exec.common.CommonExecValues.translateToPlanInternal(CommonExecValues.java:50)
at
org.apache.flink.table.planner.plan.nodes.exec.ExecNodeBase.translateToPlan(ExecNodeBase.java:171)
at
org.apache.flink.table.planner.plan.nodes.exec.ExecEdge.translateToPlan(ExecEdge.java:247)
at
org.apache.flink.table.planner.plan.nodes.exec.batch.BatchExecSink.translateToPlanInternal(BatchExecSink.java:58)
at
org.apache.flink.table.planner.plan.nodes.exec.ExecNodeBase.translateToPlan(ExecNodeBase.java:171)
at
org.apache.flink.table.planner.delegation.BatchPlanner$$anonfun$translateToPlan$1.apply(BatchPlanner.scala:81)
at
org.apache.flink.table.planner.delegation.BatchPlanner$$anonfun$translateToPlan$1.apply(BatchPlanner.scala:80)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.Iterator$class.foreach(Iterator.scala:891)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.AbstractTraversable.map(Traversable.scala:104)
at
org.apache.flink.table.planner.delegation.BatchPlanner.translateToPlan(BatchPlanner.scala:80)
at
org.apache.flink.table.planner.delegation.PlannerBase.translate(PlannerBase.scala:174)
at
org.apache.flink.table.api.internal.TableEnvironmentImpl.translate(TableEnvironmentImpl.java:1658)
at
org.apache.flink.table.api.internal.TableEnvironmentImpl.executeInternal(TableEnvironmentImpl.java:747)
at
org.apache.flink.table.api.internal.StatementSetImpl.execute(StatementSetImpl.java:100)
at
org.apache.flink.table.planner.runtime.batch.sql.TableSourceITCase.testTableXiaojin(TableSourceITCase.scala:483)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
[jira] [Comment Edited] (FLINK-23194) Cache and reuse the ContainerLaunchContext and accelarate the progress of createTaskExecutorLaunchContext on yarn
[ https://issues.apache.org/jira/browse/FLINK-23194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17376184#comment-17376184 ] zlzhang0122 edited comment on FLINK-23194 at 7/7/21, 3:15 AM: -- [~trohrmann] [~pnowojski] [Matthias|https://issues.apache.org/jira/secure/ViewProfile.jspa?name=mapohl] any suggestion is appreciate, thanks was (Author: zlzhang0122): [~trohrmann] [~pnowojski] [Matthias|https://issues.apache.org/jira/secure/ViewProfile.jspa?name=mapohl] any suggestion is appreciate > Cache and reuse the ContainerLaunchContext and accelarate the progress of > createTaskExecutorLaunchContext on yarn > - > > Key: FLINK-23194 > URL: https://issues.apache.org/jira/browse/FLINK-23194 > Project: Flink > Issue Type: Improvement > Components: Deployment / YARN >Affects Versions: 1.13.1, 1.12.4 >Reporter: zlzhang0122 >Priority: Major > Fix For: 1.14.0 > > > When starting the TaskExecutor in container on yarn, this will create > ContainerLaunchContext for n times(n represent the number of the TaskManager). > When I examined the progress of this creation, I found that most of them were > in common and had nothing to do with the particular TaskManager except the > launchCommand. We can create ContainerLaunchContext once and reuse it. Only > the launchCommand need to create separately for every particular TaskManager. > So I propose that we can cache and reuse the ContainerLaunchContext object to > accelerate this creation progress. > I think this can have some benefit like below: > # this can accelerate the creation of ContainerLaunchContext and also the > start of the TaskExecutor, especially under the situation of massive > TaskManager. > # this can decrease the pressure of the HDFS, etc. > # this can also avoid the suddenly failure of the HDFS or yarn, etc. > We have implemented this on our production environment. So far there has no > problem and have a good benefit. Please let me know if there's any point that > I haven't considered. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-23194) Cache and reuse the ContainerLaunchContext and accelarate the progress of createTaskExecutorLaunchContext on yarn
[ https://issues.apache.org/jira/browse/FLINK-23194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17376184#comment-17376184 ] zlzhang0122 commented on FLINK-23194: - [~trohrmann] [~pnowojski] [Matthias|https://issues.apache.org/jira/secure/ViewProfile.jspa?name=mapohl] any suggestion is appreciate > Cache and reuse the ContainerLaunchContext and accelarate the progress of > createTaskExecutorLaunchContext on yarn > - > > Key: FLINK-23194 > URL: https://issues.apache.org/jira/browse/FLINK-23194 > Project: Flink > Issue Type: Improvement > Components: Deployment / YARN >Affects Versions: 1.13.1, 1.12.4 >Reporter: zlzhang0122 >Priority: Major > Fix For: 1.14.0 > > > When starting the TaskExecutor in container on yarn, this will create > ContainerLaunchContext for n times(n represent the number of the TaskManager). > When I examined the progress of this creation, I found that most of them were > in common and had nothing to do with the particular TaskManager except the > launchCommand. We can create ContainerLaunchContext once and reuse it. Only > the launchCommand need to create separately for every particular TaskManager. > So I propose that we can cache and reuse the ContainerLaunchContext object to > accelerate this creation progress. > I think this can have some benefit like below: > # this can accelerate the creation of ContainerLaunchContext and also the > start of the TaskExecutor, especially under the situation of massive > TaskManager. > # this can decrease the pressure of the HDFS, etc. > # this can also avoid the suddenly failure of the HDFS or yarn, etc. > We have implemented this on our production environment. So far there has no > problem and have a good benefit. Please let me know if there's any point that > I haven't considered. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #11877: [FLINK-16641][network] Announce sender's backlog to solve the deadlock issue without exclusive buffers
flinkbot edited a comment on pull request #11877: URL: https://github.com/apache/flink/pull/11877#issuecomment-618273998 ## CI report: * 028f2185c0bb15c542006065550d04f69b0b06bd Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=19955) * fb631b77700e2f05dcfdd50cd50cdea35e7cfb13 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20047) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] ZhijieYang commented on pull request #16355: [hotfix] [docs] Fix document address does not match the actual web page.
ZhijieYang commented on pull request #16355: URL: https://github.com/apache/flink/pull/16355#issuecomment-875237794 @RocMarshal Hi, can you review it? this error can stop build the doc -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-23189) Count and fail the task when the disk is error on JobManager
[ https://issues.apache.org/jira/browse/FLINK-23189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17376181#comment-17376181 ] zlzhang0122 commented on FLINK-23189: - sure, [~pnowojski] I have posted a attachment which record the exception thrown in Flink 1.10. CheckpointCoordinator#triggerCheckpoint() will call the startTriggeringCheckpoint() function, while this function will call the initializeCheckpoint() function, this function may throw an IOException(see [link|https://github.com/zlzhang0122/flink/blob/9e1cc0ac2bbf0a2e8fcf00e6730a10893d651590/flink-runtime/src/main/java/org/apache/flink/runtime/state/CheckpointStorageCoordinatorView.java#L83]). The IOException will produce a CheckpointFailureReason.TRIGGER_CHECKPOINT_FAILURE just like any other Exception, I think that IOException is caused by disk error or any other IO problem that can hardly be resumed, and maybe we should treat it a little more serious and let users know it faster rather than just log it. > Count and fail the task when the disk is error on JobManager > > > Key: FLINK-23189 > URL: https://issues.apache.org/jira/browse/FLINK-23189 > Project: Flink > Issue Type: Improvement > Components: Runtime / Checkpointing >Affects Versions: 1.12.2, 1.13.1 >Reporter: zlzhang0122 >Priority: Major > Attachments: exception.txt > > > When the jobmanager disk is error and the triggerCheckpoint will throw a > IOException and fail, this will cause a TRIGGER_CHECKPOINT_FAILURE, but this > failure won't cause Job failed. Users can hardly find this error if he don't > see the JobManager logs. To avoid this case, I propose that we can figure out > these IOException case and increase the failureCounter which can fail the job > finally. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-23288) Inserting (1.378593404E9) (0.6047707965147558) to a double type, it will generate CodeGenException
[ https://issues.apache.org/jira/browse/FLINK-23288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xiaojin.wy updated FLINK-23288: --- Description: CREATE TABLE database5_t0( `c0` DOUBLE , `c1` INTEGER , `c2` STRING ) WITH ( 'connector' = 'filesystem', 'path' = 'hdfs:///tmp/database5_t0.csv', 'format' = 'csv' ) INSERT OVERWRITE database5_t0(c0, c1, c2) VALUES(1.378593404E9, 1336919677, '1969-12-31 20:29:41'), (0.6047707965147558, 1336919677, '1970-01-06 03:36:50') *After excuting the sql above, will generate this errors, but mysql,pg,sqlite won`t have the error:* org.apache.flink.table.planner.codegen.CodeGenException: Incompatible types of expression and result type. Expression[GeneratedExpression(((org.apache.flink.table.data.DecimalData) decimal$3),false,,DECIMAL(17, 16) NOT NULL,Some(0.6047707965147558))] type is [DECIMAL(17, 16) NOT NULL], result type is [DOUBLE NOT NULL] at org.apache.flink.table.planner.codegen.ExprCodeGenerator$$anonfun$generateResultExpression$1.apply(ExprCodeGenerator.scala:312) at org.apache.flink.table.planner.codegen.ExprCodeGenerator$$anonfun$generateResultExpression$1.apply(ExprCodeGenerator.scala:300) at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48) at org.apache.flink.table.planner.codegen.ExprCodeGenerator.generateResultExpression(ExprCodeGenerator.scala:300) at org.apache.flink.table.planner.codegen.ExprCodeGenerator.generateResultExpression(ExprCodeGenerator.scala:256) at org.apache.flink.table.planner.codegen.ValuesCodeGenerator$$anonfun$1.apply(ValuesCodeGenerator.scala:44) at org.apache.flink.table.planner.codegen.ValuesCodeGenerator$$anonfun$1.apply(ValuesCodeGenerator.scala:43) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) at scala.collection.Iterator$class.foreach(Iterator.scala:891) at scala.collection.AbstractIterator.foreach(Iterator.scala:1334) at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at scala.collection.AbstractIterable.foreach(Iterable.scala:54) at scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at scala.collection.AbstractTraversable.map(Traversable.scala:104) at org.apache.flink.table.planner.codegen.ValuesCodeGenerator$.generatorInputFormat(ValuesCodeGenerator.scala:43) at org.apache.flink.table.planner.codegen.ValuesCodeGenerator.generatorInputFormat(ValuesCodeGenerator.scala) at org.apache.flink.table.planner.plan.nodes.exec.common.CommonExecValues.translateToPlanInternal(CommonExecValues.java:50) at org.apache.flink.table.planner.plan.nodes.exec.ExecNodeBase.translateToPlan(ExecNodeBase.java:171) at org.apache.flink.table.planner.plan.nodes.exec.ExecEdge.translateToPlan(ExecEdge.java:247) at org.apache.flink.table.planner.plan.nodes.exec.batch.BatchExecSink.translateToPlanInternal(BatchExecSink.java:58) at org.apache.flink.table.planner.plan.nodes.exec.ExecNodeBase.translateToPlan(ExecNodeBase.java:171) at org.apache.flink.table.planner.delegation.BatchPlanner$$anonfun$translateToPlan$1.apply(BatchPlanner.scala:81) at org.apache.flink.table.planner.delegation.BatchPlanner$$anonfun$translateToPlan$1.apply(BatchPlanner.scala:80) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) at scala.collection.Iterator$class.foreach(Iterator.scala:891) at scala.collection.AbstractIterator.foreach(Iterator.scala:1334) at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at scala.collection.AbstractIterable.foreach(Iterable.scala:54) at scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at scala.collection.AbstractTraversable.map(Traversable.scala:104) at org.apache.flink.table.planner.delegation.BatchPlanner.translateToPlan(BatchPlanner.scala:80) at org.apache.flink.table.planner.delegation.PlannerBase.translate(PlannerBase.scala:174) at org.apache.flink.table.api.internal.TableEnvironmentImpl.translate(TableEnvironmentImpl.java:1658) at org.apache.flink.table.api.internal.TableEnvironmentImpl.executeInternal(TableEnvironmentImpl.java:747) at org.apache.flink.table.api.internal.StatementSetImpl.execute(StatementSetImpl.java:100) at org.apache.flink.table.planner.runtime.batch.sql.TableSourceITCase.testTableXiaojin(TableSourceITCase.scala:483) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
[jira] [Closed] (FLINK-23286) Create user document for Window Join in SQL
[ https://issues.apache.org/jira/browse/FLINK-23286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JING ZHANG closed FLINK-23286. -- Resolution: Duplicate > Create user document for Window Join in SQL > --- > > Key: FLINK-23286 > URL: https://issues.apache.org/jira/browse/FLINK-23286 > Project: Flink > Issue Type: Improvement > Components: Documentation >Reporter: JING ZHANG >Priority: Major > > Create user document for Window Join in SQL -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-23288) Inserting (1.378593404E9) (0.6047707965147558) to a double type, it will generate CodeGenException
xiaojin.wy created FLINK-23288: -- Summary: Inserting (1.378593404E9) (0.6047707965147558) to a double type, it will generate CodeGenException Key: FLINK-23288 URL: https://issues.apache.org/jira/browse/FLINK-23288 Project: Flink Issue Type: Improvement Components: Table SQL / Runtime Affects Versions: 1.14.0 Reporter: xiaojin.wy CREATE TABLE database5_t0( `c0` DOUBLE , `c1` INTEGER , `c2` STRING ) WITH ( 'connector' = 'filesystem', 'path' = 'hdfs:///tmp/database5_t0.csv', 'format' = 'csv' ) INSERT OVERWRITE database5_t0(c0, c1, c2) VALUES(1.378593404E9, 1336919677, '1969-12-31 20:29:41'), (0.6047707965147558, 1336919677, '1970-01-06 03:36:50') *After excuting that, will generate this errors, but mysql,pg,sqlite won`t have the error:* org.apache.flink.table.planner.codegen.CodeGenException: Incompatible types of expression and result type. Expression[GeneratedExpression(((org.apache.flink.table.data.DecimalData) decimal$3),false,,DECIMAL(17, 16) NOT NULL,Some(0.6047707965147558))] type is [DECIMAL(17, 16) NOT NULL], result type is [DOUBLE NOT NULL] at org.apache.flink.table.planner.codegen.ExprCodeGenerator$$anonfun$generateResultExpression$1.apply(ExprCodeGenerator.scala:312) at org.apache.flink.table.planner.codegen.ExprCodeGenerator$$anonfun$generateResultExpression$1.apply(ExprCodeGenerator.scala:300) at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48) at org.apache.flink.table.planner.codegen.ExprCodeGenerator.generateResultExpression(ExprCodeGenerator.scala:300) at org.apache.flink.table.planner.codegen.ExprCodeGenerator.generateResultExpression(ExprCodeGenerator.scala:256) at org.apache.flink.table.planner.codegen.ValuesCodeGenerator$$anonfun$1.apply(ValuesCodeGenerator.scala:44) at org.apache.flink.table.planner.codegen.ValuesCodeGenerator$$anonfun$1.apply(ValuesCodeGenerator.scala:43) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) at scala.collection.Iterator$class.foreach(Iterator.scala:891) at scala.collection.AbstractIterator.foreach(Iterator.scala:1334) at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at scala.collection.AbstractIterable.foreach(Iterable.scala:54) at scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at scala.collection.AbstractTraversable.map(Traversable.scala:104) at org.apache.flink.table.planner.codegen.ValuesCodeGenerator$.generatorInputFormat(ValuesCodeGenerator.scala:43) at org.apache.flink.table.planner.codegen.ValuesCodeGenerator.generatorInputFormat(ValuesCodeGenerator.scala) at org.apache.flink.table.planner.plan.nodes.exec.common.CommonExecValues.translateToPlanInternal(CommonExecValues.java:50) at org.apache.flink.table.planner.plan.nodes.exec.ExecNodeBase.translateToPlan(ExecNodeBase.java:171) at org.apache.flink.table.planner.plan.nodes.exec.ExecEdge.translateToPlan(ExecEdge.java:247) at org.apache.flink.table.planner.plan.nodes.exec.batch.BatchExecSink.translateToPlanInternal(BatchExecSink.java:58) at org.apache.flink.table.planner.plan.nodes.exec.ExecNodeBase.translateToPlan(ExecNodeBase.java:171) at org.apache.flink.table.planner.delegation.BatchPlanner$$anonfun$translateToPlan$1.apply(BatchPlanner.scala:81) at org.apache.flink.table.planner.delegation.BatchPlanner$$anonfun$translateToPlan$1.apply(BatchPlanner.scala:80) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) at scala.collection.Iterator$class.foreach(Iterator.scala:891) at scala.collection.AbstractIterator.foreach(Iterator.scala:1334) at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at scala.collection.AbstractIterable.foreach(Iterable.scala:54) at scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at scala.collection.AbstractTraversable.map(Traversable.scala:104) at org.apache.flink.table.planner.delegation.BatchPlanner.translateToPlan(BatchPlanner.scala:80) at org.apache.flink.table.planner.delegation.PlannerBase.translate(PlannerBase.scala:174) at org.apache.flink.table.api.internal.TableEnvironmentImpl.translate(TableEnvironmentImpl.java:1658) at org.apache.flink.table.api.internal.TableEnvironmentImpl.executeInternal(TableEnvironmentImpl.java:747) at
[jira] [Created] (FLINK-23287) Create user document for Window Join in SQL
JING ZHANG created FLINK-23287: -- Summary: Create user document for Window Join in SQL Key: FLINK-23287 URL: https://issues.apache.org/jira/browse/FLINK-23287 Project: Flink Issue Type: Sub-task Components: Documentation Reporter: JING ZHANG Create user document for Window Join in SQL -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot commented on pull request #16406: [FLINK-23184][table-runtime] Fix compile error in code generation of unary plus and minus
flinkbot commented on pull request #16406: URL: https://github.com/apache/flink/pull/16406#issuecomment-875233094 ## CI report: * b94f97fe8cfc71e1da286bed37b5a684eaed9a96 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (FLINK-23286) Create user document for Window Join in SQL
JING ZHANG created FLINK-23286: -- Summary: Create user document for Window Join in SQL Key: FLINK-23286 URL: https://issues.apache.org/jira/browse/FLINK-23286 Project: Flink Issue Type: Improvement Components: Documentation Reporter: JING ZHANG Create user document for Window Join in SQL -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #16405: [FLINK-23165][python] Add StreamExecutionEnvironment#registerSlotSharingGroup to PyFlink
flinkbot edited a comment on pull request #16405: URL: https://github.com/apache/flink/pull/16405#issuecomment-875224279 ## CI report: * b7fc24e07e9fb4eb0ac2b9ac6e16e89cd9041aa7 UNKNOWN * e9eb7e2451023def24169564053f8a0eeb746176 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16369: [FLINK-22443][streaming-java] Fix overflow in MultipleInputSelectionHandler
flinkbot edited a comment on pull request #16369: URL: https://github.com/apache/flink/pull/16369#issuecomment-873893547 ## CI report: * 3bb34b002d5a8952f045088b16f6a4c6a0532527 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=19917) * 11105a90383651025185cdc1804733c19e59b8ca UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16359: [FLINK-23222]Translate page 'Application Profiling & Debugging' into Chinese
flinkbot edited a comment on pull request #16359: URL: https://github.com/apache/flink/pull/16359#issuecomment-872999229 ## CI report: * 069fb525e2734b4854fc4c829730030c187e21c1 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=19844) * f38e1fff477d7520a34e21ec20d9ede7b621f522 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-web] leonardBang commented on a change in pull request #455: Add release 1.12.5
leonardBang commented on a change in pull request #455: URL: https://github.com/apache/flink-web/pull/455#discussion_r665002326 ## File path: _posts/2021-07-02-release-1.12.5.md ## @@ -0,0 +1,168 @@ +--- +layout: post +title: "Apache Flink 1.12.5 Released" +date: 2021-07-02 00:00:00 +categories: news +authors: +- arvid: Review comment: ```suggestion - jingsong: ``` Should use RM name -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #11877: [FLINK-16641][network] Announce sender's backlog to solve the deadlock issue without exclusive buffers
flinkbot edited a comment on pull request #11877: URL: https://github.com/apache/flink/pull/11877#issuecomment-618273998 ## CI report: * 028f2185c0bb15c542006065550d04f69b0b06bd Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=19955) * fb631b77700e2f05dcfdd50cd50cdea35e7cfb13 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] wsry commented on a change in pull request #11877: [FLINK-16641][network] Announce sender's backlog to solve the deadlock issue without exclusive buffers
wsry commented on a change in pull request #11877:
URL: https://github.com/apache/flink/pull/11877#discussion_r664999767
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/CreditBasedPartitionRequestClientHandler.java
##
@@ -340,7 +355,15 @@ private void decodeBufferOrEvent(
RemoteInputChannel inputChannel, NettyMessage.BufferResponse
bufferOrEvent)
throws Throwable {
if (bufferOrEvent.isBuffer() && bufferOrEvent.bufferSize == 0) {
-inputChannel.onEmptyBuffer(bufferOrEvent.sequenceNumber,
bufferOrEvent.backlog);
+try {
+inputChannel.onEmptyBuffer(bufferOrEvent.sequenceNumber,
bufferOrEvent.backlog);
+} finally {
+// recycle the empty buffer directly
+Buffer buffer = bufferOrEvent.getBuffer();
+if (buffer != null) {
+buffer.recycleBuffer();
Review comment:
I think I can keep this logic unchanged and release the buffer in decoder
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #16406: [FLINK-23184][table-runtime] Fix compile error in code generation of unary plus and minus
flinkbot commented on pull request #16406: URL: https://github.com/apache/flink/pull/16406#issuecomment-875224832 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit b94f97fe8cfc71e1da286bed37b5a684eaed9a96 (Wed Jul 07 02:38:05 UTC 2021) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #16405: [FLINK-23165][python] Add StreamExecutionEnvironment#registerSlotSharingGroup to PyFlink
flinkbot commented on pull request #16405: URL: https://github.com/apache/flink/pull/16405#issuecomment-875224279 ## CI report: * b7fc24e07e9fb4eb0ac2b9ac6e16e89cd9041aa7 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16395: [FLINK-22662][yarn][test] Stabilize YARNHighAvailabilityITCase
flinkbot edited a comment on pull request #16395: URL: https://github.com/apache/flink/pull/16395#issuecomment-874665804 ## CI report: * ff1821987722e912a4990165ea1ec8aa78d422db Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=19994) * 69f1bc393f0ed01e64fc6dad942bcfe7d60b9e43 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20044) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] wsry commented on a change in pull request #11877: [FLINK-16641][network] Announce sender's backlog to solve the deadlock issue without exclusive buffers
wsry commented on a change in pull request #11877:
URL: https://github.com/apache/flink/pull/11877#discussion_r664997218
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/netty/CreditBasedPartitionRequestClientHandler.java
##
@@ -340,7 +355,15 @@ private void decodeBufferOrEvent(
RemoteInputChannel inputChannel, NettyMessage.BufferResponse
bufferOrEvent)
throws Throwable {
if (bufferOrEvent.isBuffer() && bufferOrEvent.bufferSize == 0) {
-inputChannel.onEmptyBuffer(bufferOrEvent.sequenceNumber,
bufferOrEvent.backlog);
+try {
+inputChannel.onEmptyBuffer(bufferOrEvent.sequenceNumber,
bufferOrEvent.backlog);
+} finally {
+// recycle the empty buffer directly
+Buffer buffer = bufferOrEvent.getBuffer();
+if (buffer != null) {
+buffer.recycleBuffer();
Review comment:
Yes, you are right. Previously, ```bufferOrEevnt.getBuffer()``` was
always ```null```. And currently, it should be never null. So we do not need to
support null buffer now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (FLINK-22198) KafkaTableITCase hang.
[ https://issues.apache.org/jira/browse/FLINK-22198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17376166#comment-17376166 ] Xintong Song commented on FLINK-22198: -- https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=20035=logs=4be4ed2b-549a-533d-aa33-09e28e360cc8=0db94045-2aa0-53fa-f444-0130d6933518=7543 > KafkaTableITCase hang. > -- > > Key: FLINK-22198 > URL: https://issues.apache.org/jira/browse/FLINK-22198 > Project: Flink > Issue Type: Bug > Components: Connectors / Kafka >Affects Versions: 1.12.4 >Reporter: Guowei Ma >Priority: Major > Labels: test-stability > > https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16287=logs=c5f0071e-1851-543e-9a45-9ac140befc32=1fb1a56f-e8b5-5a82-00a0-a2db7757b4f5=6625 > There is no any artifacts. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-23184) CompileException Assignment conversion not possible from type "int" to type "short"
[
https://issues.apache.org/jira/browse/FLINK-23184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caizhi Weng updated FLINK-23184:
Description:
{code:sql}
CREATE TABLE MySink (
`a` SMALLINT
) WITH (
'connector' = 'filesystem',
'format' = 'testcsv',
'path' = '$resultPath'
)
CREATE TABLE database8_t0 (
`c0` SMALLINT
) WITH (
'connector' = 'filesystem',
'format' = 'testcsv',
'path' = '$resultPath11'
)
CREATE TABLE database8_t1 (
`c0` SMALLINT,
`c1` TINYINT
) WITH (
'connector' = 'filesystem',
'format' = 'testcsv',
'path' = '$resultPath22'
)
INSERT OVERWRITE database8_t0(c0) VALUES(cast(22424 as SMALLINT))
INSERT OVERWRITE database8_t1(c0, c1) VALUES(cast(-17443 as SMALLINT), cast(97
as TINYINT))
insert into MySink
SELECT database8_t0.c0 AS ref0 FROM database8_t0, database8_t1 WHERE CAST ((-
(database8_t0.c0)) AS BOOLEAN)
{code}
After running that , you will get the errors:
{code}
2021-06-29 19:39:27
org.apache.flink.runtime.JobException: Recovery is suppressed by
NoRestartBackoffTimeStrategy
at
org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:138)
at
org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getFailureHandlingResult(ExecutionFailureHandler.java:82)
at
org.apache.flink.runtime.scheduler.DefaultScheduler.handleTaskFailure(DefaultScheduler.java:207)
at
org.apache.flink.runtime.scheduler.DefaultScheduler.maybeHandleTaskFailure(DefaultScheduler.java:197)
at
org.apache.flink.runtime.scheduler.DefaultScheduler.updateTaskExecutionStateInternal(DefaultScheduler.java:188)
at
org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:677)
at
org.apache.flink.runtime.scheduler.SchedulerNG.updateTaskExecutionState(SchedulerNG.java:79)
at
org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:440)
at sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:305)
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:212)
at
org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:77)
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:158)
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26)
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21)
at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)
at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
at akka.actor.Actor$class.aroundReceive(Actor.scala:517)
at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592)
at akka.actor.ActorCell.invoke(ActorCell.scala:561)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258)
at akka.dispatch.Mailbox.run(Mailbox.scala:225)
at akka.dispatch.Mailbox.exec(Mailbox.scala:235)
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at
akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at
akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: java.lang.RuntimeException: Could not instantiate generated class
'BatchExecCalc$4536'
at
org.apache.flink.table.runtime.generated.GeneratedClass.newInstance(GeneratedClass.java:66)
at
org.apache.flink.table.runtime.operators.CodeGenOperatorFactory.createStreamOperator(CodeGenOperatorFactory.java:43)
at
org.apache.flink.streaming.api.operators.StreamOperatorFactoryUtil.createOperator(StreamOperatorFactoryUtil.java:80)
at
org.apache.flink.streaming.runtime.tasks.OperatorChain.createOperator(OperatorChain.java:626)
at
org.apache.flink.streaming.runtime.tasks.OperatorChain.createOperatorChain(OperatorChain.java:600)
at
org.apache.flink.streaming.runtime.tasks.OperatorChain.createOutputCollector(OperatorChain.java:540)
at
org.apache.flink.streaming.runtime.tasks.OperatorChain.(OperatorChain.java:171)
at
[GitHub] [flink] YikSanChan commented on pull request #16092: [docs][hotfix] Fix wrong scala syntax in a code snippet
YikSanChan commented on pull request #16092: URL: https://github.com/apache/flink/pull/16092#issuecomment-875222885 @zentol Could you please take another review? Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-23283) GroupWindowITCase.testWindowAggregateOnUpsertSource fails on azure
[
https://issues.apache.org/jira/browse/FLINK-23283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17376165#comment-17376165
]
Xintong Song commented on FLINK-23283:
--
[~qingru zhang], do you mind take a look at this?
> GroupWindowITCase.testWindowAggregateOnUpsertSource fails on azure
> --
>
> Key: FLINK-23283
> URL: https://issues.apache.org/jira/browse/FLINK-23283
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.14.0
>Reporter: Roman Khachatryan
>Priority: Blocker
> Fix For: 1.14.0
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=19978=logs=e25d5e7e-2a9c-5589-4940-0b638d75a414=a6e0f756-5bb9-5ea8-a468-5f60db442a29=7086
> {code}
> 2021-07-06T10:13:10.6091443Z Jul 06 10:13:10 [ERROR] Tests run: 48, Failures:
> 1, Errors: 0, Skipped: 0, Time elapsed: 14.945 s <<< FAILURE! - in
> org.apache.flink.table.planner.runtime.stream.sql.GroupWindowITCase
> 2021-07-06T10:13:10.6093667Z Jul 06 10:13:10 [ERROR]
> testWindowAggregateOnUpsertSource[StateBackend=HEAP, UseTimestampLtz =
> false](org.apache.flin*k.table.planner.runtime.stream.sql.GroupWindowITCase)
> Time elapsed: 0.329 s <<< FAILURE!
> 2021-07-06T10:13:10.6096965Z Jul 06 10:13:10 java.lang.AssertionError:
> expected: r,1,102,1970-01-01T00:00,1970-01-01T00:00:05,
> Yen,1,1,1970-01-01T00:00,1970-01-01T00:00:05)> but
> was: Dollar,1,104,1970-01-01T00:00,1970-01-01T00:00:05,
> Yen,1,1,1970-01-01T00:00,1970-01-01T00:00:05)>
> 2021-07-06T10:13:10.6098985Z Jul 06 10:13:10at
> org.junit.Assert.fail(Assert.java:89)
> 2021-07-06T10:13:10.6099695Z Jul 06 10:13:10at
> org.junit.Assert.failNotEquals(Assert.java:835)
> 2021-07-06T10:13:10.6100489Z Jul 06 10:13:10at
> org.junit.Assert.assertEquals(Assert.java:120)
> 2021-07-06T10:13:10.6101292Z Jul 06 10:13:10at
> org.junit.Assert.assertEquals(Assert.java:146)
> 2021-07-06T10:13:10.6102395Z Jul 06 10:13:10at
> org.apache.flink.table.planner.runtime.stream.sql.GroupWindowITCase.testWindowAggregateOnUpsert*Source(GroupWindowITCase.scala:421)
> {code}
> Fails locally 3 times out of 1000
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] pierre94 commented on pull request #16359: [FLINK-23222]Translate page 'Application Profiling & Debugging' into Chinese
pierre94 commented on pull request #16359: URL: https://github.com/apache/flink/pull/16359#issuecomment-875221887 @RocMarshal tks, i think your suggestions are better. I have updated the PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-23184) CompileException Assignment conversion not possible from type "int" to type "short"
[ https://issues.apache.org/jira/browse/FLINK-23184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-23184: --- Labels: pull-request-available (was: ) > CompileException Assignment conversion not possible from type "int" to type > "short" > --- > > Key: FLINK-23184 > URL: https://issues.apache.org/jira/browse/FLINK-23184 > Project: Flink > Issue Type: Bug > Components: Table SQL / Runtime >Affects Versions: 1.14.0 >Reporter: xiaojin.wy >Assignee: Caizhi Weng >Priority: Major > Labels: pull-request-available > > CREATE TABLE MySink ( > `a` SMALLINT > ) WITH ( > 'connector' = 'filesystem', > 'format' = 'testcsv', > 'path' = '$resultPath' > ) > CREATE TABLE database8_t0 ( > `c0` SMALLINT > ) WITH ( > 'connector' = 'filesystem', > 'format' = 'testcsv', > 'path' = '$resultPath11' > ) > CREATE TABLE database8_t1 ( > `c0` SMALLINT, > `c1` TINYINT > ) WITH ( > 'connector' = 'filesystem', > 'format' = 'testcsv', > 'path' = '$resultPath22' > ) > INSERT OVERWRITE database8_t0(c0) VALUES(cast(22424 as SMALLINT)) > INSERT OVERWRITE database8_t1(c0, c1) VALUES(cast(-17443 as SMALLINT), > cast(97 as TINYINT)) > insert into MySink > SELECT database8_t0.c0 AS ref0 FROM database8_t0, database8_t1 WHERE CAST ((- > (database8_t0.c0)) AS BOOLEAN) > After running that , you will get the errors: > 2021-06-29 19:39:27 > org.apache.flink.runtime.JobException: Recovery is suppressed by > NoRestartBackoffTimeStrategy > at > org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:138) > at > org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getFailureHandlingResult(ExecutionFailureHandler.java:82) > at > org.apache.flink.runtime.scheduler.DefaultScheduler.handleTaskFailure(DefaultScheduler.java:207) > at > org.apache.flink.runtime.scheduler.DefaultScheduler.maybeHandleTaskFailure(DefaultScheduler.java:197) > at > org.apache.flink.runtime.scheduler.DefaultScheduler.updateTaskExecutionStateInternal(DefaultScheduler.java:188) > at > org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:677) > at > org.apache.flink.runtime.scheduler.SchedulerNG.updateTaskExecutionState(SchedulerNG.java:79) > at > org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:440) > at sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:305) > at > org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:212) > at > org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:77) > at > org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:158) > at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) > at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) > at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123) > at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) > at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) > at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) > at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) > at akka.actor.Actor$class.aroundReceive(Actor.scala:517) > at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) > at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) > at akka.actor.ActorCell.invoke(ActorCell.scala:561) > at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) > at akka.dispatch.Mailbox.run(Mailbox.scala:225) > at akka.dispatch.Mailbox.exec(Mailbox.scala:235) > at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) > at > akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) > at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) > at > akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) > Caused by: java.lang.RuntimeException: Could not instantiate generated class > 'BatchExecCalc$4536' > at > org.apache.flink.table.runtime.generated.GeneratedClass.newInstance(GeneratedClass.java:66) > at >
