[jira] [Created] (FLINK-36426) Introduce the strategy for adaptive scheduler to control state locality preferred or minimal task executors preferred when matching resource requests and slots
RocMarshal created FLINK-36426: -- Summary: Introduce the strategy for adaptive scheduler to control state locality preferred or minimal task executors preferred when matching resource requests and slots Key: FLINK-36426 URL: https://issues.apache.org/jira/browse/FLINK-36426 Project: Flink Issue Type: Improvement Components: Runtime / Coordination Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-36039) Support clean historical event handler records in autoscaler jdbc plugin
RocMarshal created FLINK-36039:
--
Summary: Support clean historical event handler records in
autoscaler jdbc plugin
Key: FLINK-36039
URL: https://issues.apache.org/jira/browse/FLINK-36039
Project: Flink
Issue Type: Improvement
Components: Autoscaler
Reporter: RocMarshal
Currently, the autoscaler generates a large amount of historical data for event
handlers. As the system runs for a long time, the volume of historical data
will continue to grow. It is necessary to support automatic cleanup of data
within a fixed period.
Based on the creation time timestamp, the following approach for cleaning up
historical data might be a way:
* Introduce the parameter
{{job.autoscaler.event-handler.historical-ttl-duration}} (of type duration),
which represents the storage time for historical data.
* Introduce the parameter
{{job.autoscaler.event-handler.historical-clean-interval}} (of type duration),
which represents the interval between each data cleanup.
In the constructor of {{{}JdbcAutoScalerEventHandler{}}}, introduce a scheduled
task. Also, add an internal interface method, {{{}close{}}}, to
{{AutoScalerEventHandler}} and {{JobAutoScaler}} for stopping and cleaning up
the relevant logic.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-35966) Introduce the TASKS for TaskManagerLoadBalanceMode enum
RocMarshal created FLINK-35966: -- Summary: Introduce the TASKS for TaskManagerLoadBalanceMode enum Key: FLINK-35966 URL: https://issues.apache.org/jira/browse/FLINK-35966 Project: Flink Issue Type: Sub-task Components: Runtime / Coordination Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-35918) Migrate the Time to Duration for the flink-runtime module.
RocMarshal created FLINK-35918: -- Summary: Migrate the Time to Duration for the flink-runtime module. Key: FLINK-35918 URL: https://issues.apache.org/jira/browse/FLINK-35918 Project: Flink Issue Type: Technical Debt Affects Versions: 2.0.0 Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-35811) Add the documentation sections to describe the usage of users-oriented interfaces
RocMarshal created FLINK-35811: -- Summary: Add the documentation sections to describe the usage of users-oriented interfaces Key: FLINK-35811 URL: https://issues.apache.org/jira/browse/FLINK-35811 Project: Flink Issue Type: Sub-task Components: Connectors / JDBC Reporter: RocMarshal Fix For: jdbc-3.3.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-35765) Support Fine Grained Resource Specifications for Adaptive Scheduler
RocMarshal created FLINK-35765: -- Summary: Support Fine Grained Resource Specifications for Adaptive Scheduler Key: FLINK-35765 URL: https://issues.apache.org/jira/browse/FLINK-35765 Project: Flink Issue Type: Improvement Components: Runtime / Coordination Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-35750) The latency marker metrics aren't updated after failover
RocMarshal created FLINK-35750: -- Summary: The latency marker metrics aren't updated after failover Key: FLINK-35750 URL: https://issues.apache.org/jira/browse/FLINK-35750 Project: Flink Issue Type: Bug Components: Runtime / Metrics Affects Versions: 1.19.1, 1.18.1, 1.20.0 Reporter: RocMarshal Described in https://docs.google.com/document/d/1WbAgdj8NdrSVChUuEywPgWZo3bAc3JKeVsN27AWDiL4/edit?usp=sharing -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-35733) Change org.jetbrains.annotations.NotNull to javax.annotation.Nonnull
RocMarshal created FLINK-35733: -- Summary: Change org.jetbrains.annotations.NotNull to javax.annotation.Nonnull Key: FLINK-35733 URL: https://issues.apache.org/jira/browse/FLINK-35733 Project: Flink Issue Type: Technical Debt Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-35516) Update the Experimental annotation to PublicEvolving for files connector
RocMarshal created FLINK-35516: -- Summary: Update the Experimental annotation to PublicEvolving for files connector Key: FLINK-35516 URL: https://issues.apache.org/jira/browse/FLINK-35516 Project: Flink Issue Type: Technical Debt Components: Connectors / FileSystem Reporter: RocMarshal as described in https://issues.apache.org/jira/browse/FLINK-35496 We should update the annotations for the stable APIs in files connector. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-35496) The annotations of the new JDBC connector should be changed to non-Public/non-PublicEvolving
RocMarshal created FLINK-35496:
--
Summary: The annotations of the new JDBC connector should be
changed to non-Public/non-PublicEvolving
Key: FLINK-35496
URL: https://issues.apache.org/jira/browse/FLINK-35496
Project: Flink
Issue Type: Technical Debt
Components: Connectors / JDBC
Reporter: RocMarshal
In general, we use the Experimental annotation instead of {{PublicEvolving}}
or {{Public}} for new features or new apis. But {{JdbcSink}} and
JdbcSource(merged ) was marked as {{PublicEvolving}} in the first version.
[~fanrui] commented it to the original PR[1].[1]
[https://github.com/apache/flink-connector-jdbc/pull/2#discussion_r1621857589]
CC [~eskabetxe] [~Sergey Nuyanzin] [~fanrui]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-35469) Unstable test case 'testProject' of JdbcDynamicTableSourceITCase
RocMarshal created FLINK-35469: -- Summary: Unstable test case 'testProject' of JdbcDynamicTableSourceITCase Key: FLINK-35469 URL: https://issues.apache.org/jira/browse/FLINK-35469 Project: Flink Issue Type: Bug Reporter: RocMarshal https://github.com/apache/flink-connector-jdbc/actions/runs/9263628064/job/25482376215?pr=119 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-35176) Support property authentication connection for JDBC catalog & dynamic table
RocMarshal created FLINK-35176: -- Summary: Support property authentication connection for JDBC catalog & dynamic table Key: FLINK-35176 URL: https://issues.apache.org/jira/browse/FLINK-35176 Project: Flink Issue Type: Improvement Components: Connectors / JDBC Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34564) Unstable test case TableSourceITCase#testTableHintWithLogicalTableScanReuse
RocMarshal created FLINK-34564: -- Summary: Unstable test case TableSourceITCase#testTableHintWithLogicalTableScanReuse Key: FLINK-34564 URL: https://issues.apache.org/jira/browse/FLINK-34564 Project: Flink Issue Type: Bug Components: Tests Affects Versions: 2.0.0, 1.19.0 Reporter: RocMarshal Attachments: image-2024-03-02-11-01-12-718.png, image-2024-03-02-11-01-44-431.png * branch 1.19 & master * java version 1.8 * how to re-produce ** Add '@RepeatedTest' for TableSourceITCase#testTableHintWithLogicalTableScanReuse ** then run it ** !image-2024-03-02-11-01-12-718.png! ** !image-2024-03-02-11-01-44-431.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34249) Remove DefaultSlotTracker related logic.
RocMarshal created FLINK-34249: -- Summary: Remove DefaultSlotTracker related logic. Key: FLINK-34249 URL: https://issues.apache.org/jira/browse/FLINK-34249 Project: Flink Issue Type: Technical Debt Components: Runtime / Task Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34174) Remove SlotMatchingStrategy related logic
RocMarshal created FLINK-34174: -- Summary: Remove SlotMatchingStrategy related logic Key: FLINK-34174 URL: https://issues.apache.org/jira/browse/FLINK-34174 Project: Flink Issue Type: Technical Debt Components: Runtime / Task Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34102) Invalid configuration when using 'env.log.max' on yarn application mode
RocMarshal created FLINK-34102: -- Summary: Invalid configuration when using 'env.log.max' on yarn application mode Key: FLINK-34102 URL: https://issues.apache.org/jira/browse/FLINK-34102 Project: Flink Issue Type: Bug Components: Runtime / Configuration Affects Versions: 1.19.0 Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33988) Invalid configuration when using initialized root logger level on yarn application mode
RocMarshal created FLINK-33988: -- Summary: Invalid configuration when using initialized root logger level on yarn application mode Key: FLINK-33988 URL: https://issues.apache.org/jira/browse/FLINK-33988 Project: Flink Issue Type: Bug Components: Runtime / Configuration Affects Versions: 1.17.2, 1.18.0 Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33959) Unstable test case
RocMarshal created FLINK-33959: -- Summary: Unstable test case Key: FLINK-33959 URL: https://issues.apache.org/jira/browse/FLINK-33959 Project: Flink Issue Type: Bug Components: Tests Affects Versions: 1.17.2, 1.17.1, 1.17.0 Reporter: RocMarshal [https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=55940&view=logs&j=a57e0635-3fad-5b08-57c7-a4142d7d6fa9&t=2ef0effc-1da1-50e5-c2bd-aab434b1c5b7] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33947) Fix bugs in DelegatingConfiguration missed the prefix mapping
RocMarshal created FLINK-33947: -- Summary: Fix bugs in DelegatingConfiguration missed the prefix mapping Key: FLINK-33947 URL: https://issues.apache.org/jira/browse/FLINK-33947 Project: Flink Issue Type: Bug Components: API / Core Reporter: RocMarshal It was resulted from [https://github.com/apache/flink/pull/23994#issuecomment-1869905090] - Check and confirm other potential bug points - Fix the bugs about prefix key mapping when operating. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33875) Support slots wait mechanism at DeclarativeSlotPoolBridge side for Default Scheduler
RocMarshal created FLINK-33875: -- Summary: Support slots wait mechanism at DeclarativeSlotPoolBridge side for Default Scheduler Key: FLINK-33875 URL: https://issues.apache.org/jira/browse/FLINK-33875 Project: Flink Issue Type: Sub-task Components: Runtime / Task Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33874) Introduce resource request wait mechanism at DefaultDeclarativeSlotPool side for Default Scheduler
RocMarshal created FLINK-33874: -- Summary: Introduce resource request wait mechanism at DefaultDeclarativeSlotPool side for Default Scheduler Key: FLINK-33874 URL: https://issues.apache.org/jira/browse/FLINK-33874 Project: Flink Issue Type: New Feature Components: Runtime / Task Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33853) Migrate Junit5 for DefaultDeclarativeSlotPool test classes
RocMarshal created FLINK-33853: -- Summary: Migrate Junit5 for DefaultDeclarativeSlotPool test classes Key: FLINK-33853 URL: https://issues.apache.org/jira/browse/FLINK-33853 Project: Flink Issue Type: Improvement Components: Tests Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33653) Introduce a benchmark for balanced tasks scheduling
RocMarshal created FLINK-33653: -- Summary: Introduce a benchmark for balanced tasks scheduling Key: FLINK-33653 URL: https://issues.apache.org/jira/browse/FLINK-33653 Project: Flink Issue Type: Sub-task Components: Runtime / Task Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33463) Support the implementation of dynamic source tables based on the new source
RocMarshal created FLINK-33463: -- Summary: Support the implementation of dynamic source tables based on the new source Key: FLINK-33463 URL: https://issues.apache.org/jira/browse/FLINK-33463 Project: Flink Issue Type: Sub-task Components: Table SQL / JDBC Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33462) Sort out the document page about the new Jdbc source.
RocMarshal created FLINK-33462: -- Summary: Sort out the document page about the new Jdbc source. Key: FLINK-33462 URL: https://issues.apache.org/jira/browse/FLINK-33462 Project: Flink Issue Type: Sub-task Components: Connectors / JDBC Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33461) Support stream related semantics for the new jdbc source
RocMarshal created FLINK-33461: -- Summary: Support stream related semantics for the new jdbc source Key: FLINK-33461 URL: https://issues.apache.org/jira/browse/FLINK-33461 Project: Flink Issue Type: Sub-task Components: Connectors / JDBC Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33460) Support more authentication connection types such as the secret.
RocMarshal created FLINK-33460: -- Summary: Support more authentication connection types such as the secret. Key: FLINK-33460 URL: https://issues.apache.org/jira/browse/FLINK-33460 Project: Flink Issue Type: Sub-task Components: Connectors / JDBC Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33459) Introduce the new source that supports the same functionality as the original JDBC input format
RocMarshal created FLINK-33459: -- Summary: Introduce the new source that supports the same functionality as the original JDBC input format Key: FLINK-33459 URL: https://issues.apache.org/jira/browse/FLINK-33459 Project: Flink Issue Type: Sub-task Components: Connectors / JDBC Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33448) Introduce a new configuration item 'taskmanager.load-balance.mode'
RocMarshal created FLINK-33448: -- Summary: Introduce a new configuration item 'taskmanager.load-balance.mode' Key: FLINK-33448 URL: https://issues.apache.org/jira/browse/FLINK-33448 Project: Flink Issue Type: Sub-task Components: Runtime / Task Reporter: RocMarshal Introduce a new configuration item 'taskmanager.load-balance.mode' and make it compatible with "cluster.evenly-spread-out-slots" The ticket is mainly to do three items: - Introduce a new configuration item 'taskmanager.load-balance.mode' - Make it compatible with "cluster.evenly-spread-out-slots" - Marked "cluster.evenly-spread-out-slots" as Deprecated -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33392) Add the documentation page for balanced tasks scheduling
RocMarshal created FLINK-33392: -- Summary: Add the documentation page for balanced tasks scheduling Key: FLINK-33392 URL: https://issues.apache.org/jira/browse/FLINK-33392 Project: Flink Issue Type: Sub-task Components: Runtime / Task Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33391) Support tasks balancing at TM level for Adaptive Scheduler
RocMarshal created FLINK-33391: -- Summary: Support tasks balancing at TM level for Adaptive Scheduler Key: FLINK-33391 URL: https://issues.apache.org/jira/browse/FLINK-33391 Project: Flink Issue Type: Sub-task Components: Runtime / Task Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33390) Support slot balancing at TM level for Adaptive Scheduler
RocMarshal created FLINK-33390: -- Summary: Support slot balancing at TM level for Adaptive Scheduler Key: FLINK-33390 URL: https://issues.apache.org/jira/browse/FLINK-33390 Project: Flink Issue Type: Sub-task Components: Runtime / Task Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33389) Introduce the assigner for Adaptive Scheduler to pursuit task balancing based slots level
RocMarshal created FLINK-33389: -- Summary: Introduce the assigner for Adaptive Scheduler to pursuit task balancing based slots level Key: FLINK-33389 URL: https://issues.apache.org/jira/browse/FLINK-33389 Project: Flink Issue Type: Sub-task Components: Runtime / Task Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33388) Implement slots to taskmanagers balancing for the Default Scheduler
RocMarshal created FLINK-33388: -- Summary: Implement slots to taskmanagers balancing for the Default Scheduler Key: FLINK-33388 URL: https://issues.apache.org/jira/browse/FLINK-33388 Project: Flink Issue Type: Sub-task Components: Runtime / Task Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33387) Introduce the abstraction and the interface about loading
RocMarshal created FLINK-33387: -- Summary: Introduce the abstraction and the interface about loading Key: FLINK-33387 URL: https://issues.apache.org/jira/browse/FLINK-33387 Project: Flink Issue Type: Sub-task Components: Runtime / Task Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33386) Introduce the strategy for Default Scheduler to pursue tasks balancing based on slots level.
RocMarshal created FLINK-33386: -- Summary: Introduce the strategy for Default Scheduler to pursue tasks balancing based on slots level. Key: FLINK-33386 URL: https://issues.apache.org/jira/browse/FLINK-33386 Project: Flink Issue Type: Sub-task Components: Runtime / Task Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33320) Support Dynamic Logger Level Adjustment
RocMarshal created FLINK-33320: -- Summary: Support Dynamic Logger Level Adjustment Key: FLINK-33320 URL: https://issues.apache.org/jira/browse/FLINK-33320 Project: Flink Issue Type: Improvement Components: Runtime / REST Reporter: RocMarshal During the process of routine program debugging or troubleshooting, analyzing system logs is a common approach. Comprehensive and detailed system logs contribute to improved visibility of internal system execution information and also enhance the efficiency of program debugging or issue troubleshooting.However, comprehensive and detailed log settings can lead to the following issues: # A sharp increase in log volume, accelerating disk occupancy. # Potential risks of system performance degradation due to a large volume of log printing. # The need to simplify log configuration subsequently. Therefore, introducing a mechanism to dynamically adjust the online log output level in the event of diagnosing online issues or debugging programs could be meaningful. This mechanism should ideally provide the following two basic capabilities: # Dynamically adjust log levels. # Query the current log levels of the JM/TM in the cluster. The proposal doc: https://docs.google.com/document/d/1s2XQzet_8oPhMs3WyDhP_pPhAE3d1Gdw_qR4W0nKtlY/edit?usp=sharing -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-31757) Optimize Flink tasks un-balanced scheduling
RocMarshal created FLINK-31757: -- Summary: Optimize Flink tasks un-balanced scheduling Key: FLINK-31757 URL: https://issues.apache.org/jira/browse/FLINK-31757 Project: Flink Issue Type: Improvement Components: Runtime / Task Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-30511) Ignore the Exception in user-timer Triggerble when recover form state.
RocMarshal created FLINK-30511:
--
Summary: Ignore the Exception in user-timer Triggerble when
recover form state.
Key: FLINK-30511
URL: https://issues.apache.org/jira/browse/FLINK-30511
Project: Flink
Issue Type: Bug
Components: API / DataStream
Affects Versions: 1.16.0
Environment: Flink 1.16.0
java8
deployment Mode: miniCluster in IDC; standalone, yarn-application.
Reporter: RocMarshal
Attachments: 截屏2022-12-27 18.51.12.png, 截屏2022-12-27 19.20.00.png
* Code segment:
{code:java}
public class OnTimerDemo {
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
conf.setString("taskmanager.numberOfTaskSlots", "4");
conf.setString("state.checkpoint-storage", "filesystem");
conf.setString("state.checkpoints.dir", "file:///tmp/flinkjob");
conf.setString("execution.checkpointing.interval", "30s");
//conf.setString("execution.savepoint.path",
"file:///tmp/flinkjob/159561b8c97c9e0b4f9eeb649086796a/chk-1"); // Anchor-A:
StreamExecutionEnvironment env =
StreamExecutionEnvironment.createLocalEnvironmentWithWebUI(conf);
env.setParallelism(1);
EnvironmentSettings envSetting = EnvironmentSettings
.newInstance()
.inStreamingMode()
.build();
StreamTableEnvironment tableEnv = StreamTableEnvironment.create(env,
envSetting);
String sourceDDL = "CREATE TABLE orders (\n" +
" id INT,\n" +
" app INT,\n" +
" user_id STRING" +
") WITH (\n" +
" 'connector' = 'datagen',\n" +
" 'rows-per-second'='1',\n" +
" 'fields.app.min'='1',\n" +
" 'fields.app.max'='10',\n" +
" 'fields.user_id.length'='10'\n" +
")";
tableEnv.executeSql(sourceDDL);
Table query = tableEnv.sqlQuery("select * from orders");
DataStream rowDataStream = tableEnv.toAppendStream(query,
Row.class);
TypeInformation[] returnTypes = new TypeInformation[4];
returnTypes[0] = Types.INT;
returnTypes[1] = Types.INT; // Anchor-B:
returnTypes[2] = Types.INT;
returnTypes[3] = Types.INT;
rowDataStream.keyBy(new KeySelector() {
@Override
public String getKey(Row value) throws Exception {
return value.getFieldAs(2);
}
}).process(new KeyedProcessFunction() {
private Row firstRow;
@Override
public void processElement(Row value, Context ctx,
Collector out) throws Exception {
if (firstRow == null) {
firstRow = value;
}
ctx.timerService().registerProcessingTimeTimer(System.currentTimeMillis() +
3000);
}
@Override
public void onTimer(long timestamp, OnTimerContext ctx,
Collector out) throws Exception {
Row colRow = new Row(4);
colRow.setField(0, 0);
colRow.setField(1, 1);
colRow.setField(2, 2);
colRow.setField(3, 3);
out.collect(colRow); // Anchor-C
}
}).name("TargetTestUDF")
.returns(new RowTypeInfo(returnTypes))
.print();
env.execute(OnTimerDemo.class.getSimpleName());
}
}
{code}
* Recurrence steps
** Run the job without state.
** Collect the latest available checkpoint path as 'checkpoint-path-a'
** Stop the job.
** Fill the real value of 'checkpoint-path-a' into 'Anchor-A' line and
un-comment the line.
** Set 'returnTypes[1] = Types.INT;' -> 'returnTypes[1] = Types.LONG;' at the
'Anchor-B' line.
** Then add break-point at 'StreamTask#handleAsyncException' method.
** Run the job. The 'java.lang.ClassCastException: java.lang.Integer cannot be
cast to java.lang.Long' exception caused at the 'Anchor-C' line will ignore at
'StreamTask#handleAsyncException' method.
** So, The framework can't catch the same exception in the case.
* Root cause:
** !截屏2022-12-27 18.51.12.png!
** When job started from state data, the Task#restoreAndInvoke would be
called. The exception 'java.lang.ClassCastException: java.lang.Integer cannot
be cast to java.lang.Long' was ignored at the above 'handleAsyncException'
method instead of catching at catch-block of 'Task#restoreAndInvoke'.
!截屏2022-12-27 19.20.00.png!
Could it be set as the framework's missing handling of exceptions?
If so, I prefer to re-throw the exce
[jira] [Created] (FLINK-29548) Remove deprecated class files of the 'flink-test-utils' module.
RocMarshal created FLINK-29548: -- Summary: Remove deprecated class files of the 'flink-test-utils' module. Key: FLINK-29548 URL: https://issues.apache.org/jira/browse/FLINK-29548 Project: Flink Issue Type: Improvement Components: Tests Reporter: RocMarshal Fix For: 1.16.1 Remove flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/test/util/MiniClusterResource.java flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/test/util/MiniClusterResourceConfiguration.java -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-29050) [JUnit5 Migration] Module: flink-hadoop-compatibility
RocMarshal created FLINK-29050: -- Summary: [JUnit5 Migration] Module: flink-hadoop-compatibility Key: FLINK-29050 URL: https://issues.apache.org/jira/browse/FLINK-29050 Project: Flink Issue Type: Sub-task Components: Connectors / Hadoop Compatibility, Tests Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-29049) Remove the deprecated classes in the flink-java module.
RocMarshal created FLINK-29049: -- Summary: Remove the deprecated classes in the flink-java module. Key: FLINK-29049 URL: https://issues.apache.org/jira/browse/FLINK-29049 Project: Flink Issue Type: Technical Debt Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-28659) [JUnit5 Migration] Module: flink-java
RocMarshal created FLINK-28659: -- Summary: [JUnit5 Migration] Module: flink-java Key: FLINK-28659 URL: https://issues.apache.org/jira/browse/FLINK-28659 Project: Flink Issue Type: Improvement Components: Tests Affects Versions: 1.16.0 Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-27267) [JUnit5 Migration] Module: flink-contrib
RocMarshal created FLINK-27267: -- Summary: [JUnit5 Migration] Module: flink-contrib Key: FLINK-27267 URL: https://issues.apache.org/jira/browse/FLINK-27267 Project: Flink Issue Type: Sub-task Components: Tests Affects Versions: 1.16.0 Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-27143) [JUnit5 Migration] Module: flink-docs
RocMarshal created FLINK-27143: -- Summary: [JUnit5 Migration] Module: flink-docs Key: FLINK-27143 URL: https://issues.apache.org/jira/browse/FLINK-27143 Project: Flink Issue Type: Sub-task Components: Tests Affects Versions: 1.16.0 Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-26758) [JUnit5 Migration] Module: flink-container
RocMarshal created FLINK-26758: -- Summary: [JUnit5 Migration] Module: flink-container Key: FLINK-26758 URL: https://issues.apache.org/jira/browse/FLINK-26758 Project: Flink Issue Type: Sub-task Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-26747) [JUnit5 Migration] Module: flink-external-resources
RocMarshal created FLINK-26747: -- Summary: [JUnit5 Migration] Module: flink-external-resources Key: FLINK-26747 URL: https://issues.apache.org/jira/browse/FLINK-26747 Project: Flink Issue Type: Improvement Components: Tests Affects Versions: 1.16.0 Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-26496) [JUnit5 Migration] Module: flink-yarn-test
RocMarshal created FLINK-26496: -- Summary: [JUnit5 Migration] Module: flink-yarn-test Key: FLINK-26496 URL: https://issues.apache.org/jira/browse/FLINK-26496 Project: Flink Issue Type: Improvement Components: Deployment / YARN, Tests Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25800) Update wrong links in the datastream/execution_mode.md page.
RocMarshal created FLINK-25800: -- Summary: Update wrong links in the datastream/execution_mode.md page. Key: FLINK-25800 URL: https://issues.apache.org/jira/browse/FLINK-25800 Project: Flink Issue Type: Improvement Components: Documentation Reporter: RocMarshal * flink/docs/content/docs/dev/datastream/execution_mode.md * flink/docs/content.zh/docs/dev/datastream/execution_mode.md -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25799) Translate table/filesystem.md page into Chinese.
RocMarshal created FLINK-25799: -- Summary: Translate table/filesystem.md page into Chinese. Key: FLINK-25799 URL: https://issues.apache.org/jira/browse/FLINK-25799 Project: Flink Issue Type: Improvement Components: Documentation Reporter: RocMarshal docs/content.zh/docs/connectors/table/filesystem.md -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25798) Translate datastream/formats/text_files.md page into Chinese.
RocMarshal created FLINK-25798: -- Summary: Translate datastream/formats/text_files.md page into Chinese. Key: FLINK-25798 URL: https://issues.apache.org/jira/browse/FLINK-25798 Project: Flink Issue Type: Improvement Components: Documentation Reporter: RocMarshal * docs/content.zh/docs/connectors/datastream/formats/text_files.md -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25797) Translate datastream/formats/parquet.md page into Chinese.
RocMarshal created FLINK-25797: -- Summary: Translate datastream/formats/parquet.md page into Chinese. Key: FLINK-25797 URL: https://issues.apache.org/jira/browse/FLINK-25797 Project: Flink Issue Type: Improvement Components: Documentation Reporter: RocMarshal file: docs/content.zh/docs/connectors/datastream/formats/parquet.md -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25789) Translate the formats/hadoop page into Chinese.
RocMarshal created FLINK-25789: -- Summary: Translate the formats/hadoop page into Chinese. Key: FLINK-25789 URL: https://issues.apache.org/jira/browse/FLINK-25789 Project: Flink Issue Type: Improvement Components: Documentation Reporter: RocMarshal location: docs/content.zh/docs/connectors/datastream/formats/hadoop.md -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25783) Translate azure_table_storage.md page into Chinese.
RocMarshal created FLINK-25783: -- Summary: Translate azure_table_storage.md page into Chinese. Key: FLINK-25783 URL: https://issues.apache.org/jira/browse/FLINK-25783 Project: Flink Issue Type: Improvement Components: Documentation Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25782) Translate datastream filesystem.md page into Chinese.
RocMarshal created FLINK-25782: -- Summary: Translate datastream filesystem.md page into Chinese. Key: FLINK-25782 URL: https://issues.apache.org/jira/browse/FLINK-25782 Project: Flink Issue Type: Improvement Components: Documentation Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25761) Translate Avro format page into Chinese.
RocMarshal created FLINK-25761: -- Summary: Translate Avro format page into Chinese. Key: FLINK-25761 URL: https://issues.apache.org/jira/browse/FLINK-25761 Project: Flink Issue Type: Improvement Components: Documentation Reporter: RocMarshal file location: flink/docs/content.zh/docs/connectors/datastream/formats/avro.md -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25666) Remove LegacyTypeInfoDataTypeConverter
RocMarshal created FLINK-25666: -- Summary: Remove LegacyTypeInfoDataTypeConverter Key: FLINK-25666 URL: https://issues.apache.org/jira/browse/FLINK-25666 Project: Flink Issue Type: Technical Debt Components: Table SQL / Runtime Affects Versions: 1.15.0 Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25380) Introduce DynamicTableSink & DynamicTableSource for RabbitMQ
RocMarshal created FLINK-25380: -- Summary: Introduce DynamicTableSink & DynamicTableSource for RabbitMQ Key: FLINK-25380 URL: https://issues.apache.org/jira/browse/FLINK-25380 Project: Flink Issue Type: Improvement Components: Connectors/ RabbitMQ Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-25347) Replace deprecated TableSchema with Schema in MergeTableLikeUtil
RocMarshal created FLINK-25347: -- Summary: Replace deprecated TableSchema with Schema in MergeTableLikeUtil Key: FLINK-25347 URL: https://issues.apache.org/jira/browse/FLINK-25347 Project: Flink Issue Type: Technical Debt Components: Table SQL / API Reporter: RocMarshal -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-18422) Update Prefer tag in documentation 'Fault Tolerance training lesson'
RocMarshal created FLINK-18422: -- Summary: Update Prefer tag in documentation 'Fault Tolerance training lesson' Key: FLINK-18422 URL: https://issues.apache.org/jira/browse/FLINK-18422 Project: Flink Issue Type: Improvement Components: Documentation, Documentation / Training Affects Versions: 1.10.1, 1.10.0 Reporter: RocMarshal Update Prefer tag in documentation 'Fault Tolerance training lesson' according to [Prefer|http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Reminder-Prefer-link-tag-in-documentation-td42362.html]. The location is: docs/learn-flink/fault_tolerance.md -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-17472) StreamExecutionEnvironment and ExecutionEnvironment in Yarn mode
RocMarshal created FLINK-17472: -- Summary: StreamExecutionEnvironment and ExecutionEnvironment in Yarn mode Key: FLINK-17472 URL: https://issues.apache.org/jira/browse/FLINK-17472 Project: Flink Issue Type: New Feature Components: Client / Job Submission, Deployment / YARN Affects Versions: 1.10.0 Reporter: RocMarshal Expect to have such a mode of submission. Build the task directly in the Environment, and then submit the task in yarn mode. Just like RemoteStreamEnvironment, as long as you specify the parameters of the yarn cluster (host, port) or yarn configuration directory and HADOOP_USER_NAME, you can use the topology built by Env to submit the task . This submission method is best to minimize the transmission of resources required by yarn to start flink-jobmanager and taskmanagerrunner to ensure that flink can deploy tasks on the yarn cluster as quickly as possible. The simple demo as shown in the attachment .the parameter named 'env' containes all the operators about job ,like sources,maps,etc.. -- This message was sent by Atlassian Jira (v8.3.4#803005)
