Re: [PR] [FLINK-34516] Move CheckpointingMode to flink-core [flink]
masteryhx commented on code in PR #24381:
URL: https://github.com/apache/flink/pull/24381#discussion_r1533247878
##
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/environment/CheckpointConfig.java:
##
@@ -177,20 +177,44 @@ public boolean isCheckpointingEnabled() {
* Gets the checkpointing mode (exactly-once vs. at-least-once).
*
* @return The checkpointing mode.
+ * @deprecated Use {@link #getConsistencyMode} instead.
*/
-public CheckpointingMode getCheckpointingMode() {
+@Deprecated
+public org.apache.flink.streaming.api.CheckpointingMode
getCheckpointingMode() {
return
configuration.get(ExecutionCheckpointingOptions.CHECKPOINTING_MODE);
}
/**
* Sets the checkpointing mode (exactly-once vs. at-least-once).
*
* @param checkpointingMode The checkpointing mode.
+ * @deprecated Use {@link #setConsistencyMode} instead.
*/
-public void setCheckpointingMode(CheckpointingMode checkpointingMode) {
+@Deprecated
+public void setCheckpointingMode(
+org.apache.flink.streaming.api.CheckpointingMode
checkpointingMode) {
configuration.set(ExecutionCheckpointingOptions.CHECKPOINTING_MODE,
checkpointingMode);
}
+/**
+ * Gets the checkpointing consistency mode (exactly-once vs.
at-least-once).
+ *
+ * @return The checkpointing mode.
+ */
+public CheckpointingMode getConsistencyMode() {
Review Comment:
This looks good to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [FLINK-34516] Move CheckpointingMode to flink-core [flink]
Zakelly commented on code in PR #24381:
URL: https://github.com/apache/flink/pull/24381#discussion_r1533244572
##
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/environment/CheckpointConfig.java:
##
@@ -177,20 +177,44 @@ public boolean isCheckpointingEnabled() {
* Gets the checkpointing mode (exactly-once vs. at-least-once).
*
* @return The checkpointing mode.
+ * @deprecated Use {@link #getConsistencyMode} instead.
*/
-public CheckpointingMode getCheckpointingMode() {
+@Deprecated
+public org.apache.flink.streaming.api.CheckpointingMode
getCheckpointingMode() {
return
configuration.get(ExecutionCheckpointingOptions.CHECKPOINTING_MODE);
}
/**
* Sets the checkpointing mode (exactly-once vs. at-least-once).
*
* @param checkpointingMode The checkpointing mode.
+ * @deprecated Use {@link #setConsistencyMode} instead.
*/
-public void setCheckpointingMode(CheckpointingMode checkpointingMode) {
+@Deprecated
+public void setCheckpointingMode(
+org.apache.flink.streaming.api.CheckpointingMode
checkpointingMode) {
configuration.set(ExecutionCheckpointingOptions.CHECKPOINTING_MODE,
checkpointingMode);
}
+/**
+ * Gets the checkpointing consistency mode (exactly-once vs.
at-least-once).
+ *
+ * @return The checkpointing mode.
+ */
+public CheckpointingMode getConsistencyMode() {
Review Comment:
I'd change the API name `get/setConsistencyMode` to
`get/setCheckpointingConsistencyMode` and add some description showing the
relationship between the name `CheckpointingConsistencyMode` and
`CheckpointingMode`. But I'd suggest keep `CheckpointingMode` as it is since it
is a first class citizen of Flink and widely accepted by users. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Resolved] (FLINK-34647) Path normalization is allocation intensive
[ https://issues.apache.org/jira/browse/FLINK-34647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hangxiang Yu resolved FLINK-34647. -- Fix Version/s: 1.20.0 Assignee: David Schlosnagle Resolution: Fixed Merged 32144689 into master > Path normalization is allocation intensive > -- > > Key: FLINK-34647 > URL: https://issues.apache.org/jira/browse/FLINK-34647 > Project: Flink > Issue Type: Improvement > Components: API / Core, Runtime / Checkpointing >Reporter: David Schlosnagle >Assignee: David Schlosnagle >Priority: Major > Labels: pull-request-available > Fix For: 1.20.0 > > Attachments: image-2024-03-11-20-38-12-573.png > > > While investigating allocation stalls and GC pressure of a Flink streaming > pipeline, I noticed significant allocations in JFR from Flink path > normalization: > !image-2024-03-11-20-38-12-573.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] [FLINK-34647][core] Optimize Path normalization [flink]
masteryhx closed pull request #24473: [FLINK-34647][core] Optimize Path normalization URL: https://github.com/apache/flink/pull/24473 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-34516] Move CheckpointingMode to flink-core [flink]
masteryhx commented on code in PR #24381:
URL: https://github.com/apache/flink/pull/24381#discussion_r1533238343
##
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/environment/CheckpointConfig.java:
##
@@ -177,20 +177,44 @@ public boolean isCheckpointingEnabled() {
* Gets the checkpointing mode (exactly-once vs. at-least-once).
*
* @return The checkpointing mode.
+ * @deprecated Use {@link #getConsistencyMode} instead.
*/
-public CheckpointingMode getCheckpointingMode() {
+@Deprecated
+public org.apache.flink.streaming.api.CheckpointingMode
getCheckpointingMode() {
return
configuration.get(ExecutionCheckpointingOptions.CHECKPOINTING_MODE);
}
/**
* Sets the checkpointing mode (exactly-once vs. at-least-once).
*
* @param checkpointingMode The checkpointing mode.
+ * @deprecated Use {@link #setConsistencyMode} instead.
*/
-public void setCheckpointingMode(CheckpointingMode checkpointingMode) {
+@Deprecated
+public void setCheckpointingMode(
+org.apache.flink.streaming.api.CheckpointingMode
checkpointingMode) {
configuration.set(ExecutionCheckpointingOptions.CHECKPOINTING_MODE,
checkpointingMode);
}
+/**
+ * Gets the checkpointing consistency mode (exactly-once vs.
at-least-once).
+ *
+ * @return The checkpointing mode.
+ */
+public CheckpointingMode getConsistencyMode() {
Review Comment:
I just have a minor concern that users may lost about `CheckpointingMode`
and `ConsistencyMode`.
How about renaming this to `CheckpointingConsistencyMode` and adding some
descriptions about the relationship between them ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Comment Edited] (FLINK-34702) Rank should not convert to StreamExecDuplicate when the input is not insert only
[
https://issues.apache.org/jira/browse/FLINK-34702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829374#comment-17829374
]
Jacky Lau edited comment on FLINK-34702 at 3/21/24 4:32 AM:
When adopting Solution 3 it was found to be unviable for two main reasons:
# The rule cannot be placed within the physical rewrite phase. This approach
is somewhat tricky, as noted in my poc. The reason is that the
MiniBatchAssigner operator is whether the proctime / eventime is related to
the downstream operators. The current Rank operator does not require
watermarks, whereas the Deduplicate operator for rowtime does. This can be
observed in the return value of requireWatermark. See the plan change in the
test {{{}testMiniBatchInferFirstRowOnRowtime{}}}.
# The RelTraitSet produced by the Deduplicate operator differs from that of
the Rank operator. Although we can modify the RelTrait of the deduplication
operator using the code below, there is no way to change the RelTraits of other
downstream operators belonging to the deduplication operator, unless we run the
logic of the FlinkChangelogModeInferenceProgram again. The local-global
two-phase rule can be dealt with within the physical rewrite phase because the
RelTraitSet of the local aggregate is same with the upstream node. See the plan
change in the test
testUpdatableRankWithDeduplicate
{code:java}
ModifyKindSetTrait modifyKindSetTrait;
UpdateKindTrait updateKindTrait;
if (!isLastRow && !isRowtime) {
// only proctime first row deduplicate does not produce UPDATE changes
modifyKindSetTrait = ModifyKindSetTrait.INSERT_ONLY();
updateKindTrait = UpdateKindTrait.NONE();
} else {
// other deduplicate produce update changes
modifyKindSetTrait = ModifyKindSetTrait.ALL_CHANGES();
updateKindTrait = rank.getTraitSet()
.getTrait(UpdateKindTraitDef.INSTANCE());
}
RelTraitSet duplicateTraitSet = rank.getTraitSet()
.replace(modifyKindSetTrait)
.replace(updateKindTrait); {code}
was (Author: jackylau):
When adopting Solution 3 it was found to be unviable for two main reasons:
# The rule cannot be placed within the physical rewrite phase. This approach
is somewhat tricky, as noted in my poc. The reason is that the
MiniBatchAssigner operator is whether the proctime / eventime is related to
the downstream operators. The current Rank operator does not require
watermarks, whereas the Deduplicate operator for rowtime does. This can be
observed in the return value of requireWatermark. See the plan change in the
test {{{}testMiniBatchInferFirstRowOnRowtime{}}}.
# The RelTraitSet produced by the Deduplicate operator differs from that of
the Rank operator. Although we can modify the RelTrait of the deduplication
operator using the code below, there is no way to change the RelTraits of other
downstream operators belonging to the deduplication operator, unless we un the
logic of the FlinkChangelogModeInferenceProgram again. The local-global
two-phase rule can be dealt with within the physical rewrite phase because the
RelTraitSet of the local aggregate is same with the upstream node. See the plan
change in the test
testUpdatableRankWithDeduplicate
{code:java}
ModifyKindSetTrait modifyKindSetTrait;
UpdateKindTrait updateKindTrait;
if (!isLastRow && !isRowtime) {
// only proctime first row deduplicate does not produce UPDATE changes
modifyKindSetTrait = ModifyKindSetTrait.INSERT_ONLY();
updateKindTrait = UpdateKindTrait.NONE();
} else {
// other deduplicate produce update changes
modifyKindSetTrait = ModifyKindSetTrait.ALL_CHANGES();
updateKindTrait = rank.getTraitSet()
.getTrait(UpdateKindTraitDef.INSTANCE());
}
RelTraitSet duplicateTraitSet = rank.getTraitSet()
.replace(modifyKindSetTrait)
.replace(updateKindTrait); {code}
> Rank should not convert to StreamExecDuplicate when the input is not insert
> only
>
>
> Key: FLINK-34702
> URL: https://issues.apache.org/jira/browse/FLINK-34702
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.20.0
>Reporter: Jacky Lau
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.20.0
>
>
> {code:java}
> @Test
> def testSimpleFirstRowOnBuiltinProctime1(): Unit = {
> val sqlQuery =
> """
> |SELECT *
> |FROM (
> | SELECT *,
> |ROW_NUMBER() OVER (PARTITION BY a ORDER BY PROCTIME() ASC) as
> rowNum
> | FROM (select a, count(b) as b from MyTable group by a)
> |)
> |WHERE rowNum = 1
> """.stripMargin
> util.verifyExecPlan(sqlQuery)
> } {code}
> Exception:
> org.apache.flink.table.api.TableException: StreamPhysicalDeduplicate doesn't
>
[jira] [Comment Edited] (FLINK-34702) Rank should not convert to StreamExecDuplicate when the input is not insert only
[
https://issues.apache.org/jira/browse/FLINK-34702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829374#comment-17829374
]
Jacky Lau edited comment on FLINK-34702 at 3/21/24 4:31 AM:
When adopting Solution 3 it was found to be unviable for two main reasons:
# The rule cannot be placed within the physical rewrite phase. This approach
is somewhat tricky, as noted in my poc. The reason is that the
MiniBatchAssigner operator is whether the proctime / eventime is related to
the downstream operators. The current Rank operator does not require
watermarks, whereas the Deduplicate operator for rowtime does. This can be
observed in the return value of requireWatermark. See the plan change in the
test {{{}testMiniBatchInferFirstRowOnRowtime{}}}.
# The RelTraitSet produced by the Deduplicate operator differs from that of
the Rank operator. Although we can modify the RelTrait of the deduplication
operator using the code below, there is no way to change the RelTraits of other
downstream operators belonging to the deduplication operator, unless we un the
logic of the FlinkChangelogModeInferenceProgram again. The local-global
two-phase rule can be dealt with within the physical rewrite phase because the
RelTraitSet of the local aggregate is same with the upstream node. See the plan
change in the test
testUpdatableRankWithDeduplicate
{code:java}
ModifyKindSetTrait modifyKindSetTrait;
UpdateKindTrait updateKindTrait;
if (!isLastRow && !isRowtime) {
// only proctime first row deduplicate does not produce UPDATE changes
modifyKindSetTrait = ModifyKindSetTrait.INSERT_ONLY();
updateKindTrait = UpdateKindTrait.NONE();
} else {
// other deduplicate produce update changes
modifyKindSetTrait = ModifyKindSetTrait.ALL_CHANGES();
updateKindTrait = rank.getTraitSet()
.getTrait(UpdateKindTraitDef.INSTANCE());
}
RelTraitSet duplicateTraitSet = rank.getTraitSet()
.replace(modifyKindSetTrait)
.replace(updateKindTrait); {code}
was (Author: jackylau):
When opting for Solution 3, it was found to be unviable for two main reasons:
# The rule cannot be placed within the physical rewrite phase. This approach
is somewhat tricky, as noted in my poc. The reason is that the
MiniBatchAssigner operator is whether the proctime / eventime is related to
the downstream operators. The current Rank operator does not require
watermarks, whereas the Deduplicate operator for rowtime does. This can be
observed in the return value of requireWatermark. See the plan change in the
test {{{}testMiniBatchInferFirstRowOnRowtime{}}}.
# The RelTraitSet produced by the Deduplicate operator differs from that of
the Rank operator. Although we can modify the RelTrait of the deduplication
operator using the code below, there is no way to change the RelTraits of other
downstream operators belonging to the deduplication operator, unless we un the
logic of the FlinkChangelogModeInferenceProgram again. The local-global
two-phase rule can be dealt with within the physical rewrite phase because the
RelTraitSet of the local aggregate is same with the upstream node. See the plan
change in the test
testUpdatableRankWithDeduplicate
{code:java}
ModifyKindSetTrait modifyKindSetTrait;
UpdateKindTrait updateKindTrait;
if (!isLastRow && !isRowtime) {
// only proctime first row deduplicate does not produce UPDATE changes
modifyKindSetTrait = ModifyKindSetTrait.INSERT_ONLY();
updateKindTrait = UpdateKindTrait.NONE();
} else {
// other deduplicate produce update changes
modifyKindSetTrait = ModifyKindSetTrait.ALL_CHANGES();
updateKindTrait = rank.getTraitSet()
.getTrait(UpdateKindTraitDef.INSTANCE());
}
RelTraitSet duplicateTraitSet = rank.getTraitSet()
.replace(modifyKindSetTrait)
.replace(updateKindTrait); {code}
> Rank should not convert to StreamExecDuplicate when the input is not insert
> only
>
>
> Key: FLINK-34702
> URL: https://issues.apache.org/jira/browse/FLINK-34702
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.20.0
>Reporter: Jacky Lau
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.20.0
>
>
> {code:java}
> @Test
> def testSimpleFirstRowOnBuiltinProctime1(): Unit = {
> val sqlQuery =
> """
> |SELECT *
> |FROM (
> | SELECT *,
> |ROW_NUMBER() OVER (PARTITION BY a ORDER BY PROCTIME() ASC) as
> rowNum
> | FROM (select a, count(b) as b from MyTable group by a)
> |)
> |WHERE rowNum = 1
> """.stripMargin
> util.verifyExecPlan(sqlQuery)
> } {code}
> Exception:
> org.apache.flink.table.api.TableException: StreamPhysicalDeduplicate doesn't
>
[jira] [Commented] (FLINK-34702) Rank should not convert to StreamExecDuplicate when the input is not insert only
[
https://issues.apache.org/jira/browse/FLINK-34702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829374#comment-17829374
]
Jacky Lau commented on FLINK-34702:
---
When opting for Solution 3, it was found to be unviable for two main reasons:
# The rule cannot be placed within the physical rewrite phase. This approach
is somewhat tricky, as noted in my poc. The reason is that the
MiniBatchAssigner operator is whether the proctime / eventime is related to
the downstream operators. The current Rank operator does not require
watermarks, whereas the Deduplicate operator for rowtime does. This can be
observed in the return value of requireWatermark. See the plan change in the
test {{{}testMiniBatchInferFirstRowOnRowtime{}}}.
# The RelTraitSet produced by the Deduplicate operator differs from that of
the Rank operator. Although we can modify the RelTrait of the deduplication
operator using the code below, there is no way to change the RelTraits of other
downstream operators belonging to the deduplication operator, unless we un the
logic of the FlinkChangelogModeInferenceProgram again. The local-global
two-phase rule can be dealt with within the physical rewrite phase because the
RelTraitSet of the local aggregate is same with the upstream node. See the plan
change in the test
testUpdatableRankWithDeduplicate
{code:java}
ModifyKindSetTrait modifyKindSetTrait;
UpdateKindTrait updateKindTrait;
if (!isLastRow && !isRowtime) {
// only proctime first row deduplicate does not produce UPDATE changes
modifyKindSetTrait = ModifyKindSetTrait.INSERT_ONLY();
updateKindTrait = UpdateKindTrait.NONE();
} else {
// other deduplicate produce update changes
modifyKindSetTrait = ModifyKindSetTrait.ALL_CHANGES();
updateKindTrait = rank.getTraitSet()
.getTrait(UpdateKindTraitDef.INSTANCE());
}
RelTraitSet duplicateTraitSet = rank.getTraitSet()
.replace(modifyKindSetTrait)
.replace(updateKindTrait); {code}
> Rank should not convert to StreamExecDuplicate when the input is not insert
> only
>
>
> Key: FLINK-34702
> URL: https://issues.apache.org/jira/browse/FLINK-34702
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.20.0
>Reporter: Jacky Lau
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.20.0
>
>
> {code:java}
> @Test
> def testSimpleFirstRowOnBuiltinProctime1(): Unit = {
> val sqlQuery =
> """
> |SELECT *
> |FROM (
> | SELECT *,
> |ROW_NUMBER() OVER (PARTITION BY a ORDER BY PROCTIME() ASC) as
> rowNum
> | FROM (select a, count(b) as b from MyTable group by a)
> |)
> |WHERE rowNum = 1
> """.stripMargin
> util.verifyExecPlan(sqlQuery)
> } {code}
> Exception:
> org.apache.flink.table.api.TableException: StreamPhysicalDeduplicate doesn't
> support consuming update changes which is produced by node
> GroupAggregate(groupBy=[a], select=[a, COUNT(b) AS b])
> because the StreamPhysicalDeduplicate can not consuming update changes now
> while StreamExecRank can.
> so we should not convert the FlinkLogicalRank to StreamPhysicalDeduplicate in
> this case. and we can defer whether input contains update change in the
> "optimize the physical plan" phase.
> so we can add an option to solve it. and when the StreamPhysicalDeduplicate
> can support consuming update changes , we can deprecate it
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] [FLINK-34712][release] Generate reference data for state migration tests based on release-1.19.0 [flink]
masteryhx commented on PR #24517: URL: https://github.com/apache/flink/pull/24517#issuecomment-2011189235 > @masteryhx I've tried manually regenerate the specified test `mvn clean package -Pgenerate-migration-test-data -Dgenerate.classes=StatefulJobSnapshotMigrationITCase -Dgenerate.version=1.19 -nsu -Dfast -DskipTests` > > a strange log says the test class does not exist: > > ``` > main: > [java] 743 [main] INFO org.apache.flink.test.migration.MigrationTestsSnapshotGenerator [] - Start test data generating for module /Users/lilin/work/git/flink/flink-tests and version 1.19 > [java] 749 [main] WARN org.apache.flink.test.migration.MigrationTestsSnapshotGenerator [] - Class StatefulJobSnapshotMigrationITCase does not exist. > [INFO] Executed tasks > [INFO] > [INFO] BUILD SUCCESS > [INFO] > [INFO] Total time: 01:42 min > [INFO] Finished at: 2024-03-20T23:35:58+08:00 > ``` > > Do you know what's wrong here? (If you know this, I haven't had time to dive into it yet) 1. The generate.classes should be configured as its complete path, e.g. org.apache.flink.test.checkpointing.StatefulJobSnapshotMigrationITCase 2. I just also tried to generate it but seems no result. I took a quick look and haven't found the reason. Maybe there are some issues for the generator logic. You could also debug this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-34898) Cannot create named STRUCT with a single field
[
https://issues.apache.org/jira/browse/FLINK-34898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829371#comment-17829371
]
Yubin Li commented on FLINK-34898:
--
I use the latest version to run the syntax and work fine, maybe you could try
1.19.
!image-2024-03-21-12-00-00-183.png|width=688,height=161!
> Cannot create named STRUCT with a single field
> --
>
> Key: FLINK-34898
> URL: https://issues.apache.org/jira/browse/FLINK-34898
> Project: Flink
> Issue Type: Bug
>Reporter: Chloe He
>Priority: Major
> Attachments: image-2024-03-21-12-00-00-183.png
>
>
> I'm trying to create named structs using Flink SQL and I found a previous
> ticket https://issues.apache.org/jira/browse/FLINK-9161 that mentions the use
> of the following syntax:
> {code:java}
> SELECT CAST(('a', 1) as ROW) AS row1;
> {code}
> However, my named struct has a single field and effectively it should look
> something like `\{"a": 1}`. I can't seem to be able to find a way to
> construct this. I have experimented with a few different syntax and it either
> throws parsing error or casting error:
> {code:java}
> Cast function cannot convert value of type INTEGER to type
> RecordType(VARCHAR(2147483647) a) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-34900) Check compatibility for classes in flink-core-api that skip japicmp
[ https://issues.apache.org/jira/browse/FLINK-34900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weijie Guo updated FLINK-34900: --- Description: As we said in FLINK-34899, some classes are temporarily ignored by japicmp. Before the next release, we need to manually check if any of these classes have been changed in an incompatible way then. > Check compatibility for classes in flink-core-api that skip japicmp > --- > > Key: FLINK-34900 > URL: https://issues.apache.org/jira/browse/FLINK-34900 > Project: Flink > Issue Type: Sub-task > Components: API / Core >Reporter: Weijie Guo >Assignee: Weijie Guo >Priority: Major > > As we said in FLINK-34899, some classes are temporarily ignored by japicmp. > Before the next release, we need to manually check if any of these classes > have been changed in an incompatible way then. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-34898) Cannot create named STRUCT with a single field
[
https://issues.apache.org/jira/browse/FLINK-34898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yubin Li updated FLINK-34898:
-
Attachment: image-2024-03-21-12-00-00-183.png
> Cannot create named STRUCT with a single field
> --
>
> Key: FLINK-34898
> URL: https://issues.apache.org/jira/browse/FLINK-34898
> Project: Flink
> Issue Type: Bug
>Reporter: Chloe He
>Priority: Major
> Attachments: image-2024-03-21-12-00-00-183.png
>
>
> I'm trying to create named structs using Flink SQL and I found a previous
> ticket https://issues.apache.org/jira/browse/FLINK-9161 that mentions the use
> of the following syntax:
> {code:java}
> SELECT CAST(('a', 1) as ROW) AS row1;
> {code}
> However, my named struct has a single field and effectively it should look
> something like `\{"a": 1}`. I can't seem to be able to find a way to
> construct this. I have experimented with a few different syntax and it either
> throws parsing error or casting error:
> {code:java}
> Cast function cannot convert value of type INTEGER to type
> RecordType(VARCHAR(2147483647) a) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (FLINK-34899) Include all classes in flink-core-api that skip the japicmp check back
[ https://issues.apache.org/jira/browse/FLINK-34899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weijie Guo reassigned FLINK-34899: -- Assignee: Weijie Guo > Include all classes in flink-core-api that skip the japicmp check back > -- > > Key: FLINK-34899 > URL: https://issues.apache.org/jira/browse/FLINK-34899 > Project: Flink > Issue Type: Sub-task > Components: API / Core >Reporter: Weijie Guo >Assignee: Weijie Guo >Priority: Major > > We have moved some classes from flink-core to flink-core-api without changing > the package name, but the japicmp check will also consider this incompatible > changes. As a result, we excluded them for now. But we'll have to include > them back in the next release. > The list of these classes: > org.apache.flink.api.common.functions.Function > org.apache.flink.api.java.functions.KeySelector -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34900) Check compatibility for classes in flink-core-api that skip japicmp
Weijie Guo created FLINK-34900: -- Summary: Check compatibility for classes in flink-core-api that skip japicmp Key: FLINK-34900 URL: https://issues.apache.org/jira/browse/FLINK-34900 Project: Flink Issue Type: Sub-task Components: API / Core Reporter: Weijie Guo Assignee: Weijie Guo -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (FLINK-26088) Add Elasticsearch 8.0 support
[ https://issues.apache.org/jira/browse/FLINK-26088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weijie Guo closed FLINK-26088. -- Fix Version/s: elasticsearch-3.1.0 Resolution: Done main via 963c5058c9821682d76d9033a3417dfdb5f30ff7. > Add Elasticsearch 8.0 support > - > > Key: FLINK-26088 > URL: https://issues.apache.org/jira/browse/FLINK-26088 > Project: Flink > Issue Type: Improvement > Components: Connectors / ElasticSearch >Reporter: Yuhao Bi >Assignee: Matheus Felisberto >Priority: Major > Labels: pull-request-available, stale-assigned > Fix For: elasticsearch-3.1.0 > > > Since Elasticsearch 8.0 is officially released, I think it's time to consider > adding es8 connector support. > The High Level REST Client we used for connection [is marked deprecated in es > 7.15.0|https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high.html]. > Maybe we can migrate to use the new [Java API > Client|https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/8.0/index.html] > at this time. > Elasticsearch8.0 release note: > [https://www.elastic.co/guide/en/elasticsearch/reference/8.0/release-notes-8.0.0.html] > release highlights: > [https://www.elastic.co/guide/en/elasticsearch/reference/8.0/release-highlights.html] > REST API compatibility: > https://www.elastic.co/guide/en/elasticsearch/reference/8.0/rest-api-compatibility.html -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] [FLINK-26088][Connectors/ElasticSearch] Add Elasticsearch 8.0 support [flink-connector-elasticsearch]
boring-cyborg[bot] commented on PR #53: URL: https://github.com/apache/flink-connector-elasticsearch/pull/53#issuecomment-2011167239 Awesome work, congrats on your first merged pull request! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-26088][Connectors/ElasticSearch] Add Elasticsearch 8.0 support [flink-connector-elasticsearch]
reswqa merged PR #53: URL: https://github.com/apache/flink-connector-elasticsearch/pull/53 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-34899) Include all classes in flink-core-api that skip the japicmp check back
[ https://issues.apache.org/jira/browse/FLINK-34899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weijie Guo updated FLINK-34899: --- Description: We have moved some classes from flink-core to flink-core-api without changing the package name, but the japicmp check will also consider this incompatible changes. As a result, we excluded them for now. But we'll have to include them back in the next release. The list of these classes: org.apache.flink.api.common.functions.Function org.apache.flink.api.java.functions.KeySelector was: We have moved some classes from flink-core to flink-core-api without changing the package name, but the japicmp check will also consider this incompatible changes. As a result, we excluded them for now. But we'll have to include them back in the next release. > Include all classes in flink-core-api that skip the japicmp check back > -- > > Key: FLINK-34899 > URL: https://issues.apache.org/jira/browse/FLINK-34899 > Project: Flink > Issue Type: Sub-task > Components: API / Core >Reporter: Weijie Guo >Priority: Major > > We have moved some classes from flink-core to flink-core-api without changing > the package name, but the japicmp check will also consider this incompatible > changes. As a result, we excluded them for now. But we'll have to include > them back in the next release. > The list of these classes: > org.apache.flink.api.common.functions.Function > org.apache.flink.api.java.functions.KeySelector -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-34899) Include all classes in flink-core-api that skip the japicmp check back
[ https://issues.apache.org/jira/browse/FLINK-34899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weijie Guo updated FLINK-34899: --- Summary: Include all classes in flink-core-api that skip the japicmp check back (was: Remove all classes that skip the japicmp check for flink-core-api) > Include all classes in flink-core-api that skip the japicmp check back > -- > > Key: FLINK-34899 > URL: https://issues.apache.org/jira/browse/FLINK-34899 > Project: Flink > Issue Type: Sub-task > Components: API / Core >Reporter: Weijie Guo >Priority: Major > > We have moved some classes from flink-core to flink-core-api without changing > the package name, but the japicmp check will also consider this incompatible > changes. As a result, we excluded them for now. But we'll have to include > them back in the next release. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-34899) Remove all classes that skip the japicmp check for flink-core-api
[ https://issues.apache.org/jira/browse/FLINK-34899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weijie Guo updated FLINK-34899: --- Description: We have moved some classes from flink-core to flink-core-api without changing the package name, but the japicmp check will also consider this incompatible changes. As a result, we excluded them for now. But we'll have to include them back in the next release. was: We have moved some classes from flink-core to flink-core-api without changing the package name, but the japicmp check will also consider this incompatible changes. As a result, we excluded them for now. > Remove all classes that skip the japicmp check for flink-core-api > - > > Key: FLINK-34899 > URL: https://issues.apache.org/jira/browse/FLINK-34899 > Project: Flink > Issue Type: Sub-task > Components: API / Core >Reporter: Weijie Guo >Priority: Major > > We have moved some classes from flink-core to flink-core-api without changing > the package name, but the japicmp check will also consider this incompatible > changes. As a result, we excluded them for now. But we'll have to include > them back in the next release. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-34899) Remove all classes that skip the japicmp check for flink-core-api
[ https://issues.apache.org/jira/browse/FLINK-34899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weijie Guo updated FLINK-34899: --- Description: We have moved some classes from flink-core to flink-core-api without changing the package name, but the japicmp check will also consider this incompatible changes. As a result, we excluded them for now. > Remove all classes that skip the japicmp check for flink-core-api > - > > Key: FLINK-34899 > URL: https://issues.apache.org/jira/browse/FLINK-34899 > Project: Flink > Issue Type: Sub-task > Components: API / Core >Reporter: Weijie Guo >Priority: Major > > We have moved some classes from flink-core to flink-core-api without changing > the package name, but the japicmp check will also consider this incompatible > changes. As a result, we excluded them for now. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34899) Remove all classes that skip the japicmp check for flink-core-api
Weijie Guo created FLINK-34899: -- Summary: Remove all classes that skip the japicmp check for flink-core-api Key: FLINK-34899 URL: https://issues.apache.org/jira/browse/FLINK-34899 Project: Flink Issue Type: Sub-task Components: API / Core Reporter: Weijie Guo -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] [FLINK-34668][checkpoint] Report state handle of file merging directory to JM [flink]
Zakelly commented on code in PR #24513:
URL: https://github.com/apache/flink/pull/24513#discussion_r1533201932
##
flink-runtime/src/main/java/org/apache/flink/runtime/state/filemerging/SegmentFileStateHandle.java:
##
@@ -62,19 +77,58 @@ public class SegmentFileStateHandle implements
StreamStateHandle {
* @param scope The state's scope, whether it is exclusive or shared.
*/
public SegmentFileStateHandle(
-Path filePath, long startPos, long stateSize,
CheckpointedStateScope scope) {
+Path directoryPath,
+Path filePath,
+long startPos,
+long stateSize,
+CheckpointedStateScope scope) {
this.filePath = filePath;
this.stateSize = stateSize;
this.startPos = startPos;
this.scope = scope;
+this.directoryStateHandle =
+DirectoryStreamStateHandle.forPathWithSize(
+new File(directoryPath.getPath()).toPath());
Review Comment:
Will `forPathWithSize` be inefficiency?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [FLINK-34689][MySQL][Feature] check binlog_row_value_optoins [flink-cdc]
SML0127 commented on PR #3148: URL: https://github.com/apache/flink-cdc/pull/3148#issuecomment-2011101317 @ruanhang1993 PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] (FLINK-34898) Cannot create named STRUCT with a single field
[ https://issues.apache.org/jira/browse/FLINK-34898 ]
Chloe He deleted comment on FLINK-34898:
--
was (Author: JIRAUSER304541):
I also can't wrap this in an ARRAY like so
{code:java}
SELECT * FROM (VALUES
ARRAY[CAST((1, 2) as ROW)]
); {code}
This raises
{code:java}
java.lang.UnsupportedOperationException: class
org.apache.calcite.sql.SqlBasicCall: ROW('a', 1) {code}
> Cannot create named STRUCT with a single field
> --
>
> Key: FLINK-34898
> URL: https://issues.apache.org/jira/browse/FLINK-34898
> Project: Flink
> Issue Type: Bug
>Reporter: Chloe He
>Priority: Major
>
> I'm trying to create named structs using Flink SQL and I found a previous
> ticket https://issues.apache.org/jira/browse/FLINK-9161 that mentions the use
> of the following syntax:
> {code:java}
> SELECT CAST(('a', 1) as ROW) AS row1;
> {code}
> However, my named struct has a single field and effectively it should look
> something like `\{"a": 1}`. I can't seem to be able to find a way to
> construct this. I have experimented with a few different syntax and it either
> throws parsing error or casting error:
> {code:java}
> Cast function cannot convert value of type INTEGER to type
> RecordType(VARCHAR(2147483647) a) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-34898) Cannot create named STRUCT with a single field
[
https://issues.apache.org/jira/browse/FLINK-34898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829349#comment-17829349
]
Chloe He commented on FLINK-34898:
--
I also can't wrap this in an ARRAY like so
{code:java}
SELECT * FROM (VALUES
ARRAY[CAST((1, 2) as ROW)]
); {code}
This raises
{code:java}
java.lang.UnsupportedOperationException: class
org.apache.calcite.sql.SqlBasicCall: ROW('a', 1) {code}
> Cannot create named STRUCT with a single field
> --
>
> Key: FLINK-34898
> URL: https://issues.apache.org/jira/browse/FLINK-34898
> Project: Flink
> Issue Type: Bug
>Reporter: Chloe He
>Priority: Major
>
> I'm trying to create named structs using Flink SQL and I found a previous
> ticket https://issues.apache.org/jira/browse/FLINK-9161 that mentions the use
> of the following syntax:
> {code:java}
> SELECT CAST(('a', 1) as ROW) AS row1;
> {code}
> However, my named struct has a single field and effectively it should look
> something like `\{"a": 1}`. I can't seem to be able to find a way to
> construct this. I have experimented with a few different syntax and it either
> throws parsing error or casting error:
> {code:java}
> Cast function cannot convert value of type INTEGER to type
> RecordType(VARCHAR(2147483647) a) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] [FLINK-34526][runtime] Actively disconnect the TM in RM to reduce restart time [flink]
qinf commented on PR #24539: URL: https://github.com/apache/flink/pull/24539#issuecomment-2011045063 @1996fanrui @RocMarshal Could you help review this PR in your free time? Thank you. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Flink 34770 [flink-cdc]
lvyanquan closed pull request #3183: Flink 34770 URL: https://github.com/apache/flink-cdc/pull/3183 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-34898) Cannot create named STRUCT with a single field
[
https://issues.apache.org/jira/browse/FLINK-34898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chloe He updated FLINK-34898:
-
Description:
I'm trying to create named structs using Flink SQL and I found a previous
ticket https://issues.apache.org/jira/browse/FLINK-9161 that mentions the use
of the following syntax:
{code:java}
SELECT CAST(('a', 1) as ROW) AS row1;
{code}
However, my named struct has a single field and effectively it should look
something like `\{"a": 1}`. I can't seem to be able to find a way to construct
this. I have experimented with a few different syntax and it either throws
parsing error or casting error:
{code:java}
Cast function cannot convert value of type INTEGER to type
RecordType(VARCHAR(2147483647) a) {code}
was:
I'm trying to create named structs using Flink SQL and I found a previous
ticket https://issues.apache.org/jira/browse/FLINK-9161 that mentions the use
of the following syntax:
```sql
SELECT CAST(('a', 1) as ROW) AS row1;
```
However, my named struct has a single field and effectively it should look
something like `\{"a": 1}`. I can't seem to be able to find a way to construct
this. I have experimented with a few different syntax and it either throws
parsing error or casting error:
```
Cast function cannot convert value of type INTEGER to type
RecordType(VARCHAR(2147483647) a)
```
> Cannot create named STRUCT with a single field
> --
>
> Key: FLINK-34898
> URL: https://issues.apache.org/jira/browse/FLINK-34898
> Project: Flink
> Issue Type: Bug
>Reporter: Chloe He
>Priority: Major
>
> I'm trying to create named structs using Flink SQL and I found a previous
> ticket https://issues.apache.org/jira/browse/FLINK-9161 that mentions the use
> of the following syntax:
>
> {code:java}
> SELECT CAST(('a', 1) as ROW) AS row1;
> {code}
> However, my named struct has a single field and effectively it should look
> something like `\{"a": 1}`. I can't seem to be able to find a way to
> construct this. I have experimented with a few different syntax and it either
> throws parsing error or casting error:
> {code:java}
> Cast function cannot convert value of type INTEGER to type
> RecordType(VARCHAR(2147483647) a) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-34898) Cannot create named STRUCT with a single field
Chloe He created FLINK-34898:
Summary: Cannot create named STRUCT with a single field
Key: FLINK-34898
URL: https://issues.apache.org/jira/browse/FLINK-34898
Project: Flink
Issue Type: Bug
Reporter: Chloe He
I'm trying to create named structs using Flink SQL and I found a previous
ticket https://issues.apache.org/jira/browse/FLINK-9161 that mentions the use
of the following syntax:
```sql
SELECT CAST(('a', 1) as ROW) AS row1;
```
However, my named struct has a single field and effectively it should look
something like `\{"a": 1}`. I can't seem to be able to find a way to construct
this. I have experimented with a few different syntax and it either throws
parsing error or casting error:
```
Cast function cannot convert value of type INTEGER to type
RecordType(VARCHAR(2147483647) a)
```
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-34898) Cannot create named STRUCT with a single field
[
https://issues.apache.org/jira/browse/FLINK-34898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chloe He updated FLINK-34898:
-
Description:
I'm trying to create named structs using Flink SQL and I found a previous
ticket https://issues.apache.org/jira/browse/FLINK-9161 that mentions the use
of the following syntax:
{code:java}
SELECT CAST(('a', 1) as ROW) AS row1;
{code}
However, my named struct has a single field and effectively it should look
something like `\{"a": 1}`. I can't seem to be able to find a way to construct
this. I have experimented with a few different syntax and it either throws
parsing error or casting error:
{code:java}
Cast function cannot convert value of type INTEGER to type
RecordType(VARCHAR(2147483647) a) {code}
was:
I'm trying to create named structs using Flink SQL and I found a previous
ticket https://issues.apache.org/jira/browse/FLINK-9161 that mentions the use
of the following syntax:
{code:java}
SELECT CAST(('a', 1) as ROW) AS row1;
{code}
However, my named struct has a single field and effectively it should look
something like `\{"a": 1}`. I can't seem to be able to find a way to construct
this. I have experimented with a few different syntax and it either throws
parsing error or casting error:
{code:java}
Cast function cannot convert value of type INTEGER to type
RecordType(VARCHAR(2147483647) a) {code}
> Cannot create named STRUCT with a single field
> --
>
> Key: FLINK-34898
> URL: https://issues.apache.org/jira/browse/FLINK-34898
> Project: Flink
> Issue Type: Bug
>Reporter: Chloe He
>Priority: Major
>
> I'm trying to create named structs using Flink SQL and I found a previous
> ticket https://issues.apache.org/jira/browse/FLINK-9161 that mentions the use
> of the following syntax:
> {code:java}
> SELECT CAST(('a', 1) as ROW) AS row1;
> {code}
> However, my named struct has a single field and effectively it should look
> something like `\{"a": 1}`. I can't seem to be able to find a way to
> construct this. I have experimented with a few different syntax and it either
> throws parsing error or casting error:
> {code:java}
> Cast function cannot convert value of type INTEGER to type
> RecordType(VARCHAR(2147483647) a) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] [FLINK-31664][table]Add ARRAY_INTERSECT function [flink]
liuyongvs commented on PR #23171: URL: https://github.com/apache/flink/pull/23171#issuecomment-2011038228 hi @dawidwys will you help review this https://github.com/apache/flink/pull/24526 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-34732][cdc][ci] Add document dead link check for Flink CDC Documentation [flink-cdc]
GOODBOY008 commented on PR #3168: URL: https://github.com/apache/flink-cdc/pull/3168#issuecomment-2011028273 @PatrickRen Rebased to master , PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-33220) PyFlink support for Datagen connector
[ https://issues.apache.org/jira/browse/FLINK-33220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829346#comment-17829346 ] Dian Fu commented on FLINK-33220: - [~liu.chong] I missed this ticket. Feel free to submit the PR and ping me~ > PyFlink support for Datagen connector > - > > Key: FLINK-33220 > URL: https://issues.apache.org/jira/browse/FLINK-33220 > Project: Flink > Issue Type: New Feature > Components: API / Python >Reporter: Liu Chong >Priority: Minor > > This is a simple Jira to propose the support of Datagen in PyFlink datastream > API as a built-in source connector -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (FLINK-33220) PyFlink support for Datagen connector
[ https://issues.apache.org/jira/browse/FLINK-33220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829346#comment-17829346 ] Dian Fu edited comment on FLINK-33220 at 3/21/24 1:24 AM: -- [~liu.chong] Sorry, I missed this ticket. Feel free to submit the PR and ping me~ was (Author: dianfu): [~liu.chong] I missed this ticket. Feel free to submit the PR and ping me~ > PyFlink support for Datagen connector > - > > Key: FLINK-33220 > URL: https://issues.apache.org/jira/browse/FLINK-33220 > Project: Flink > Issue Type: New Feature > Components: API / Python >Reporter: Liu Chong >Priority: Minor > > This is a simple Jira to propose the support of Datagen in PyFlink datastream > API as a built-in source connector -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] [Improve][pipeline]Support set checkpoint. [flink-cdc]
lightzhao commented on PR #3101: URL: https://github.com/apache/flink-cdc/pull/3101#issuecomment-2011007376 @leonardBang PTAL. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Bug][pipeline]Set default values for the parallelism. [flink-cdc]
lightzhao commented on PR #3100: URL: https://github.com/apache/flink-cdc/pull/3100#issuecomment-2011007083 @leonardBang PTAL. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Closed] (FLINK-34701) Publish the Dockerfiles for the new release
[ https://issues.apache.org/jira/browse/FLINK-34701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lincoln lee closed FLINK-34701. --- Resolution: Fixed > Publish the Dockerfiles for the new release > --- > > Key: FLINK-34701 > URL: https://issues.apache.org/jira/browse/FLINK-34701 > Project: Flink > Issue Type: Sub-task >Reporter: lincoln lee >Assignee: lincoln lee >Priority: Major > Labels: pull-request-available > > Note: the official Dockerfiles fetch the binary distribution of the target > Flink version from an Apache mirror. After publishing the binary release > artifacts, mirrors can take some hours to start serving the new artifacts, so > you may want to wait to do this step until you are ready to continue with the > "Promote the release" steps in the follow-up Jira. > Follow the [release instructions in the flink-docker > repo|https://github.com/apache/flink-docker#release-workflow] to build the > new Dockerfiles and send an updated manifest to Docker Hub so the new images > are built and published. > > > h3. Expectations > * Dockerfiles in [flink-docker|https://github.com/apache/flink-docker] > updated for the new Flink release and pull request opened on the Docker > official-images with an updated manifest -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-34701) Publish the Dockerfiles for the new release
[
https://issues.apache.org/jira/browse/FLINK-34701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829340#comment-17829340
]
lincoln lee commented on FLINK-34701:
-
merged in
[{{official-images}}|https://github.com/docker-library/official-images] master:
fe836e1e82ee9a807d9e9825d98a9a47b8c5
> Publish the Dockerfiles for the new release
> ---
>
> Key: FLINK-34701
> URL: https://issues.apache.org/jira/browse/FLINK-34701
> Project: Flink
> Issue Type: Sub-task
>Reporter: lincoln lee
>Assignee: lincoln lee
>Priority: Major
> Labels: pull-request-available
>
> Note: the official Dockerfiles fetch the binary distribution of the target
> Flink version from an Apache mirror. After publishing the binary release
> artifacts, mirrors can take some hours to start serving the new artifacts, so
> you may want to wait to do this step until you are ready to continue with the
> "Promote the release" steps in the follow-up Jira.
> Follow the [release instructions in the flink-docker
> repo|https://github.com/apache/flink-docker#release-workflow] to build the
> new Dockerfiles and send an updated manifest to Docker Hub so the new images
> are built and published.
>
>
> h3. Expectations
> * Dockerfiles in [flink-docker|https://github.com/apache/flink-docker]
> updated for the new Flink release and pull request opened on the Docker
> official-images with an updated manifest
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] [FLINK-34892][ci] Fix python test failure due to config file change [flink-connector-aws]
dannycranmer commented on PR #133: URL: https://github.com/apache/flink-connector-aws/pull/133#issuecomment-2010729129 Thanks @z3d1k. Waiting for actions to complete before merge -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-32706][table] Add built-in SPLIT_STRING function [flink]
hanyuzheng7 commented on code in PR #24365:
URL: https://github.com/apache/flink/pull/24365#discussion_r1532815010
##
flink-python/pyflink/table/expression.py:
##
@@ -1609,6 +1609,17 @@ def array_min(self) -> 'Expression':
"""
return _unary_op("arrayMin")(self)
+def split(self, delimiter) -> 'Expression':
Review Comment:
I just support Java one, so I think we don't need add python test(s) for
`split`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [FLINK-27146] [Filesystem] Migrate to Junit5 [flink]
kottmann commented on PR #22789: URL: https://github.com/apache/flink/pull/22789#issuecomment-2010479115 @ferenc-csaky Thank you for your insightful feedback and for directing me towards the assertj decision. I updated my branch by rebasing it onto master and incorporating the changes related to assertj. Additionally, I will make another commit to address the remaining feedback that isn't connected to assertj. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-26570][statefun] Remote module configuration interpolation [flink-statefun]
nicklester commented on PR #309: URL: https://github.com/apache/flink-statefun/pull/309#issuecomment-2010471575 I still see value in this PR. When deploying into a managed environment such as Kinesis Data Analytics on AWS it is not possible to use something like 'jar -uf' to modify the jar at deploy time. We see value in being able to pass a hostname for API Gateway as an environment variable, for instance, to simplify multi environment deployment in an AWS CDK stack. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-34696) GSRecoverableWriterCommitter is generating excessive data blobs
[
https://issues.apache.org/jira/browse/FLINK-34696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829246#comment-17829246
]
Galen Warren commented on FLINK-34696:
--
I'm not aware of any special considerations in RecoverableWriter for batch
mode, but to be honest I'm not very knowledgeable about batch mode in general.
Maybe someone else knows more about that ...
{quote}The code above also avoids inserting the intermediate compose blob
identifier at index 0. That looks expensive if the list is very big (and as far
as I see not a LinkedList)
{quote}
I don't think it's inserting anything, really, as the blobs are immutable. I
think that just controls the order of the bytes in the newly created blob.
> GSRecoverableWriterCommitter is generating excessive data blobs
> ---
>
> Key: FLINK-34696
> URL: https://issues.apache.org/jira/browse/FLINK-34696
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / FileSystem
>Reporter: Simon-Shlomo Poil
>Priority: Major
>
> The `composeBlobs` method in
> `org.apache.flink.fs.gs.writer.GSRecoverableWriterCommitter` is designed to
> merge multiple small blobs into a single large blob using Google Cloud
> Storage's compose method. This process is iterative, combining the result
> from the previous iteration with 31 new blobs until all blobs are merged.
> Upon completion of the composition, the method proceeds to remove the
> temporary blobs.
> *Issue:*
> This methodology results in significant, unnecessary data storage consumption
> during the blob composition process, incurring considerable costs due to
> Google Cloud Storage pricing models.
> *Example to Illustrate the Problem:*
> - Initial state: 64 blobs, each 1 GB in size (totaling 64 GB).
> - After 1st step: 32 blobs are merged into a single blob, increasing total
> storage to 96 GB (64 original + 32 GB new).
> - After 2nd step: The newly created 32 GB blob is merged with 31 more blobs,
> raising the total to 159 GB.
> - After 3rd step: The final blob is merged, culminating in a total of 223 GB
> to combine the original 64 GB of data. This results in an overhead of 159 GB.
> *Impact:*
> This inefficiency has a profound impact, especially at scale, where terabytes
> of data can incur overheads in the petabyte range, leading to unexpectedly
> high costs. Additionally, we have observed an increase in storage exceptions
> thrown by the Google Storage library, potentially linked to this issue.
> *Suggested Solution:*
> To mitigate this problem, we propose modifying the `composeBlobs` method to
> immediately delete source blobs once they have been successfully combined.
> This change could significantly reduce data duplication and associated costs.
> However, the implications for data recovery and integrity need careful
> consideration to ensure that this optimization does not compromise the
> ability to recover data in case of a failure during the composition process.
> *Steps to Reproduce:*
> 1. Initiate the blob composition process in an environment with a significant
> number of blobs (e.g., 64 blobs of 1 GB each).
> 2. Observe the temporary increase in data storage as blobs are iteratively
> combined.
> 3. Note the final amount of data storage used compared to the initial total
> size of the blobs.
> *Expected Behavior:*
> The blob composition process should minimize unnecessary data storage use,
> efficiently managing resources to combine blobs without generating excessive
> temporary data overhead.
> *Actual Behavior:*
> The current implementation results in significant temporary increases in data
> storage, leading to high costs and potential system instability due to
> frequent storage exceptions.
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-34706) Promote release 1.19
[
https://issues.apache.org/jira/browse/FLINK-34706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829244#comment-17829244
]
lincoln lee commented on FLINK-34706:
-
# (/) Website pull request to [list the
release|http://flink.apache.org/downloads.html] merged
# (/) Release announced on the user@ mailing list: [[announcement
link|https://lists.apache.org/thread/sofmxytbh6y20nwot1gywqqc2lqxn4hm]|https://lists.apache.org/thread/72nmfwsgs7sqkw7mykz4h36hgb7wo04d]
# (/) Blog post published, if applicable:[ blog
post|https://flink.apache.org/2024/03/18/announcing-the-release-of-apache-flink-1.19/]
# (/) Release recorded in [reporter.apache.org:
https://reporter.apache.org/addrelease.html?flink|https://reporter.apache.org/addrelease.html?flink]
# (/) Release announced on social media:
[Twitter|https://twitter.com/ApacheFlink/status/1638839542403981312?ref_src=twsrc%5Etfw%7Ctwcamp%5Etweetembed%7Ctwterm%5E1638839542403981312%7Ctwgr%5E7f3046f67668cf3ebbd929ef126a32473db2a1b5%7Ctwcon%5Es1_c10_url=https%3A%2F%2Fpublish.twitter.com%2F%3Fquery%3Dhttps3A2F2Ftwitter.com2FApacheFlink2Fstatus2F1638839542403981312widget%3DTweet]
# (/) Completion declared on the dev@ [mailing list
|https://lists.apache.org/thread/z8sfwlppsodcyng62c584n76b69b16fc]
# (/) Update Homebrew:
[https://docs.brew.sh/How-To-Open-a-Homebrew-Pull-Request] (seems to be done
automatically - at least for minor releases for both minor and major
releases): [https://formulae.brew.sh/formula/apache-flink#default]
# (/) No need to update quickstart scripts in {{{}flink-web{}}}, under the
{{q/}} directory (alread use global version variables)
# Updated the japicmp configuration: Done in
https://issues.apache.org/jira/browse/FLINK-34707
# Update the list of previous version in {{docs/config.toml}} on the master
branch: Done in
# Set {{show_outdated_warning: true}} in {{docs/config.toml}} in the branch
of the _previous_ Flink version: (for 1.17)
# (/) Update stable and master alias in
[https://github.com/apache/flink/blob/master/.github/workflows/docs.yml] Done
in
[a6a4667|https://github.com/apache/flink/commit/a6a4667202a0f89fe63ff4f2e476c0200ec66e63]
> Promote release 1.19
>
>
> Key: FLINK-34706
> URL: https://issues.apache.org/jira/browse/FLINK-34706
> Project: Flink
> Issue Type: New Feature
>Affects Versions: 1.18.0
>Reporter: Lincoln Lee
>Assignee: lincoln lee
>Priority: Major
> Labels: pull-request-available
>
> Once the release has been finalized (FLINK-32920), the last step of the
> process is to promote the release within the project and beyond. Please wait
> for 24h after finalizing the release in accordance with the [ASF release
> policy|http://www.apache.org/legal/release-policy.html#release-announcements].
> *Final checklist to declare this issue resolved:*
> # Website pull request to [list the
> release|http://flink.apache.org/downloads.html] merged
> # Release announced on the user@ mailing list.
> # Blog post published, if applicable.
> # Release recorded in
> [reporter.apache.org|https://reporter.apache.org/addrelease.html?flink].
> # Release announced on social media.
> # Completion declared on the dev@ mailing list.
> # Update Homebrew:
> [https://docs.brew.sh/How-To-Open-a-Homebrew-Pull-Request] (seems to be done
> automatically - at least for minor releases for both minor and major
> releases)
> # Updated the japicmp configuration
> ** corresponding SNAPSHOT branch japicmp reference version set to the just
> released version, and API compatibiltity checks for {{@PublicEvolving}} was
> enabled
> ** (minor version release only) master branch japicmp reference version set
> to the just released version
> ** (minor version release only) master branch japicmp exclusions have been
> cleared
> # Update the list of previous version in {{docs/config.toml}} on the master
> branch.
> # Set {{show_outdated_warning: true}} in {{docs/config.toml}} in the branch
> of the _now deprecated_ Flink version (i.e. 1.16 if 1.18.0 is released)
> # Update stable and master alias in
> [https://github.com/apache/flink/blob/master/.github/workflows/docs.yml]
> # Open discussion thread for End of Life for Unsupported version (i.e. 1.16)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] [FLINK-34706][docs] Add 1.19 to PreviousDocs list. [flink]
flinkbot commented on PR #24548: URL: https://github.com/apache/flink/pull/24548#issuecomment-2010028945 ## CI report: * d73fe76e42235b2ddc1937b70c1f1b4eaf5aa38c UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-34706][docs] Deprecates 1.17 docs. [flink]
flinkbot commented on PR #24547: URL: https://github.com/apache/flink/pull/24547#issuecomment-2010015809 ## CI report: * aea991f5e66edb1f26c7ef16e71c564083b58a73 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (FLINK-34696) GSRecoverableWriterCommitter is generating excessive data blobs
[ https://issues.apache.org/jira/browse/FLINK-34696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829228#comment-17829228 ] Simon-Shlomo Poil edited comment on FLINK-34696 at 3/20/24 4:22 PM: I checked using the CLI gcloud storage objects compose blobA blobB blobA that worked. But yes indeed the documentation is a bit unclear. How the actual compose works on the GCS side is a good question. If I understand correctly - this committer is called regardless of the execution mode: ie. BATCH / STREAMING? In BATCH mode checkpointing is not supported, so the code should not depend on frequent checkpointing to work efficiently. The code above also avoids inserting the intermediate compose blob identifier at index 0. That looks expensive if the list is very big (and as far as I see not a LinkedList) was (Author: sisp): I checked using the CLI gcloud storage objects compose blobA blobB blobA that worked. But yes indeed the documentation is a bit unclear. How the actual compose works on the GCS side is a good question. If I understand correctly - this committer is called regardless of the execution mode: ie. BATCH / STREAMING? In BATCH mode checkpointing is not supported, so the code should depend on frequent checkpointing to work efficiently. The code above also avoids inserting the intermediate compose blob identifier at index 0. That looks expensive if the list is very big (and as far as I see not a LinkedList) > GSRecoverableWriterCommitter is generating excessive data blobs > --- > > Key: FLINK-34696 > URL: https://issues.apache.org/jira/browse/FLINK-34696 > Project: Flink > Issue Type: Improvement > Components: Connectors / FileSystem >Reporter: Simon-Shlomo Poil >Priority: Major > > The `composeBlobs` method in > `org.apache.flink.fs.gs.writer.GSRecoverableWriterCommitter` is designed to > merge multiple small blobs into a single large blob using Google Cloud > Storage's compose method. This process is iterative, combining the result > from the previous iteration with 31 new blobs until all blobs are merged. > Upon completion of the composition, the method proceeds to remove the > temporary blobs. > *Issue:* > This methodology results in significant, unnecessary data storage consumption > during the blob composition process, incurring considerable costs due to > Google Cloud Storage pricing models. > *Example to Illustrate the Problem:* > - Initial state: 64 blobs, each 1 GB in size (totaling 64 GB). > - After 1st step: 32 blobs are merged into a single blob, increasing total > storage to 96 GB (64 original + 32 GB new). > - After 2nd step: The newly created 32 GB blob is merged with 31 more blobs, > raising the total to 159 GB. > - After 3rd step: The final blob is merged, culminating in a total of 223 GB > to combine the original 64 GB of data. This results in an overhead of 159 GB. > *Impact:* > This inefficiency has a profound impact, especially at scale, where terabytes > of data can incur overheads in the petabyte range, leading to unexpectedly > high costs. Additionally, we have observed an increase in storage exceptions > thrown by the Google Storage library, potentially linked to this issue. > *Suggested Solution:* > To mitigate this problem, we propose modifying the `composeBlobs` method to > immediately delete source blobs once they have been successfully combined. > This change could significantly reduce data duplication and associated costs. > However, the implications for data recovery and integrity need careful > consideration to ensure that this optimization does not compromise the > ability to recover data in case of a failure during the composition process. > *Steps to Reproduce:* > 1. Initiate the blob composition process in an environment with a significant > number of blobs (e.g., 64 blobs of 1 GB each). > 2. Observe the temporary increase in data storage as blobs are iteratively > combined. > 3. Note the final amount of data storage used compared to the initial total > size of the blobs. > *Expected Behavior:* > The blob composition process should minimize unnecessary data storage use, > efficiently managing resources to combine blobs without generating excessive > temporary data overhead. > *Actual Behavior:* > The current implementation results in significant temporary increases in data > storage, leading to high costs and potential system instability due to > frequent storage exceptions. > > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-34696) GSRecoverableWriterCommitter is generating excessive data blobs
[ https://issues.apache.org/jira/browse/FLINK-34696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829228#comment-17829228 ] Simon-Shlomo Poil commented on FLINK-34696: --- I checked using the CLI gcloud storage objects compose blobA blobB blobA that worked. But yes indeed the documentation is a bit unclear. How the actual compose works on the GCS side is a good question. If I understand correctly - this committer is called regardless of the execution mode: ie. BATCH / STREAMING? In BATCH mode checkpointing is not supported, so the code should depend on frequent checkpointing to work efficiently. The code above also avoids inserting the intermediate compose blob identifier at index 0. That looks expensive if the list is very big (and as far as I see not a LinkedList) > GSRecoverableWriterCommitter is generating excessive data blobs > --- > > Key: FLINK-34696 > URL: https://issues.apache.org/jira/browse/FLINK-34696 > Project: Flink > Issue Type: Improvement > Components: Connectors / FileSystem >Reporter: Simon-Shlomo Poil >Priority: Major > > The `composeBlobs` method in > `org.apache.flink.fs.gs.writer.GSRecoverableWriterCommitter` is designed to > merge multiple small blobs into a single large blob using Google Cloud > Storage's compose method. This process is iterative, combining the result > from the previous iteration with 31 new blobs until all blobs are merged. > Upon completion of the composition, the method proceeds to remove the > temporary blobs. > *Issue:* > This methodology results in significant, unnecessary data storage consumption > during the blob composition process, incurring considerable costs due to > Google Cloud Storage pricing models. > *Example to Illustrate the Problem:* > - Initial state: 64 blobs, each 1 GB in size (totaling 64 GB). > - After 1st step: 32 blobs are merged into a single blob, increasing total > storage to 96 GB (64 original + 32 GB new). > - After 2nd step: The newly created 32 GB blob is merged with 31 more blobs, > raising the total to 159 GB. > - After 3rd step: The final blob is merged, culminating in a total of 223 GB > to combine the original 64 GB of data. This results in an overhead of 159 GB. > *Impact:* > This inefficiency has a profound impact, especially at scale, where terabytes > of data can incur overheads in the petabyte range, leading to unexpectedly > high costs. Additionally, we have observed an increase in storage exceptions > thrown by the Google Storage library, potentially linked to this issue. > *Suggested Solution:* > To mitigate this problem, we propose modifying the `composeBlobs` method to > immediately delete source blobs once they have been successfully combined. > This change could significantly reduce data duplication and associated costs. > However, the implications for data recovery and integrity need careful > consideration to ensure that this optimization does not compromise the > ability to recover data in case of a failure during the composition process. > *Steps to Reproduce:* > 1. Initiate the blob composition process in an environment with a significant > number of blobs (e.g., 64 blobs of 1 GB each). > 2. Observe the temporary increase in data storage as blobs are iteratively > combined. > 3. Note the final amount of data storage used compared to the initial total > size of the blobs. > *Expected Behavior:* > The blob composition process should minimize unnecessary data storage use, > efficiently managing resources to combine blobs without generating excessive > temporary data overhead. > *Actual Behavior:* > The current implementation results in significant temporary increases in data > storage, leading to high costs and potential system instability due to > frequent storage exceptions. > > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (FLINK-34713) Updates the docs stable version
[
https://issues.apache.org/jira/browse/FLINK-34713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
lincoln lee closed FLINK-34713.
---
Resolution: Fixed
Already fixed by 3d67a2b9b208ed662f81c1ef43d76d10a82a7d88 on release-1.19 branch
> Updates the docs stable version
> ---
>
> Key: FLINK-34713
> URL: https://issues.apache.org/jira/browse/FLINK-34713
> Project: Flink
> Issue Type: Sub-task
>Reporter: Lincoln Lee
>Assignee: lincoln lee
>Priority: Major
>
> Update docs to "stable" in {{docs/config.toml}} in the branch of the
> _just-released_ version:
> * Change V{{{}ersion{}}} from {{{}x.y-SNAPSHOT }}to \{{{}x.y.z{}}}, i.e.
> {{1.6-SNAPSHOT}} to {{1.6.0}}
> * Change V{{{}ersionTitle{}}} from {{x.y-SNAPSHOT}} to {{{}x.y{}}}, i.e.
> {{1.6-SNAPSHOT}} to {{1.6}}
> * Change Branch from {{master}} to {{{}release-x.y{}}}, i.e. {{master}} to
> {{release-1.6}}
> * Change {{baseURL}} from
> {{//[ci.apache.org/projects/flink/flink-docs-master|http://ci.apache.org/projects/flink/flink-docs-master]}}
> to
> {{//[ci.apache.org/projects/flink/flink-docs-release-x.y|http://ci.apache.org/projects/flink/flink-docs-release-x.y]}}
> * Change {{javadocs_baseurl}} from
> {{//[ci.apache.org/projects/flink/flink-docs-master|http://ci.apache.org/projects/flink/flink-docs-master]}}
> to
> {{//[ci.apache.org/projects/flink/flink-docs-release-x.y|http://ci.apache.org/projects/flink/flink-docs-release-x.y]}}
> * Change {{IsStable}} to {{true}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] [FLINK-34712][release] Generate reference data for state migration tests based on release-1.19.0 [flink]
lincoln-lil commented on PR #24517: URL: https://github.com/apache/flink/pull/24517#issuecomment-2009918611 @masteryhx I've tried manually regenerate the specified test `mvn clean package -Pgenerate-migration-test-data -Dgenerate.classes=StatefulJobSnapshotMigrationITCase -Dgenerate.version=1.19 -nsu -Dfast -DskipTests` a strange log says the test class does not exist: ``` main: [java] 743 [main] INFO org.apache.flink.test.migration.MigrationTestsSnapshotGenerator [] - Start test data generating for module /Users/lilin/work/git/flink/flink-tests and version 1.19 [java] 749 [main] WARN org.apache.flink.test.migration.MigrationTestsSnapshotGenerator [] - Class StatefulJobSnapshotMigrationITCase does not exist. [INFO] Executed tasks [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 01:42 min [INFO] Finished at: 2024-03-20T23:35:58+08:00 ``` Do you know what's wrong here? (If you know this, I haven't had time to dive into it yet) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-34893] Bump checkstyle to 9.3 [flink]
snuyanzin commented on code in PR #24540: URL: https://github.com/apache/flink/pull/24540#discussion_r1532320054 ## tools/maven/checkstyle.xml: ## @@ -238,16 +238,9 @@ This file is based on the checkstyle file of Apache Beam. Review Comment: Thanks, there is another link in the same file, I updated it as well -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-34897][test] Enables JobMasterServiceLeadershipRunnerTest#testJobMasterServiceLeadershipRunnerCloseWhenElectionServiceGrantLeaderShip again. [flink]
flinkbot commented on PR #24546: URL: https://github.com/apache/flink/pull/24546#issuecomment-2009806089 ## CI report: * 234bd00f873cb44f04131539ecae98ef7d2d9b9d UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-34897) JobMasterServiceLeadershipRunnerTest#testJobMasterServiceLeadershipRunnerCloseWhenElectionServiceGrantLeaderShip needs to be enabled again
[
https://issues.apache.org/jira/browse/FLINK-34897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-34897:
---
Labels: pull-request-available (was: )
> JobMasterServiceLeadershipRunnerTest#testJobMasterServiceLeadershipRunnerCloseWhenElectionServiceGrantLeaderShip
> needs to be enabled again
> --
>
> Key: FLINK-34897
> URL: https://issues.apache.org/jira/browse/FLINK-34897
> Project: Flink
> Issue Type: Technical Debt
> Components: Runtime / Coordination
>Affects Versions: 1.17.2, 1.19.0, 1.18.1, 1.20.0
>Reporter: Matthias Pohl
>Assignee: Matthias Pohl
>Priority: Major
> Labels: pull-request-available
>
> While working on FLINK-34672 I noticed that
> {{JobMasterServiceLeadershipRunnerTest#testJobMasterServiceLeadershipRunnerCloseWhenElectionServiceGrantLeaderShip}}
> is disabled without a reason.
> It looks like I disabled it accidentally as part of FLINK-31783.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[PR] [FLINK-34897][test] Enables JobMasterServiceLeadershipRunnerTest#test… [flink]
XComp opened a new pull request, #24546: URL: https://github.com/apache/flink/pull/24546 … ## What is the purpose of the change *(For example: This pull request makes task deployment go through the blob server, rather than through RPC. That way we avoid re-transferring them on each deployment (during recovery).)* ## Brief change log *(for example:)* - *The TaskInfo is stored in the blob store on job creation time as a persistent artifact* - *Deployments RPC transmits only the blob storage reference* - *TaskManagers retrieve the TaskInfo from the blob cache* ## Verifying this change Please make sure both new and modified tests in this PR follows the conventions defined in our code quality guide: https://flink.apache.org/contributing/code-style-and-quality-common.html#testing *(Please pick either of the following options)* This change is a trivial rework / code cleanup without any test coverage. *(or)* This change is already covered by existing tests, such as *(please describe tests)*. *(or)* This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (100MB)* - *Extended integration test for recovery after master (JobManager) failure* - *Added test that validates that TaskInfo is transferred only once across recoveries* - *Manually verified the change by running a 4 node cluster with 2 JobManagers and 4 TaskManagers, a stateful streaming program, and killing one JobManager and two TaskManagers during the execution, verifying that recovery happens correctly.* ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / no) - The serializers: (yes / no / don't know) - The runtime per-record code paths (performance sensitive): (yes / no / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know) - The S3 file system connector: (yes / no / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (FLINK-34897) JobMasterServiceLeadershipRunnerTest#testJobMasterServiceLeadershipRunnerCloseWhenElectionServiceGrantLeaderShip needs to be enabled again
Matthias Pohl created FLINK-34897:
-
Summary:
JobMasterServiceLeadershipRunnerTest#testJobMasterServiceLeadershipRunnerCloseWhenElectionServiceGrantLeaderShip
needs to be enabled again
Key: FLINK-34897
URL: https://issues.apache.org/jira/browse/FLINK-34897
Project: Flink
Issue Type: Technical Debt
Components: Runtime / Coordination
Affects Versions: 1.18.1, 1.19.0, 1.17.2, 1.20.0
Reporter: Matthias Pohl
While working on FLINK-34672 I noticed that
{{JobMasterServiceLeadershipRunnerTest#testJobMasterServiceLeadershipRunnerCloseWhenElectionServiceGrantLeaderShip}}
is disabled without a reason.
It looks like I disabled it accidentally as part of FLINK-31783.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (FLINK-34897) JobMasterServiceLeadershipRunnerTest#testJobMasterServiceLeadershipRunnerCloseWhenElectionServiceGrantLeaderShip needs to be enabled again
[
https://issues.apache.org/jira/browse/FLINK-34897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias Pohl reassigned FLINK-34897:
-
Assignee: Matthias Pohl
> JobMasterServiceLeadershipRunnerTest#testJobMasterServiceLeadershipRunnerCloseWhenElectionServiceGrantLeaderShip
> needs to be enabled again
> --
>
> Key: FLINK-34897
> URL: https://issues.apache.org/jira/browse/FLINK-34897
> Project: Flink
> Issue Type: Technical Debt
> Components: Runtime / Coordination
>Affects Versions: 1.17.2, 1.19.0, 1.18.1, 1.20.0
>Reporter: Matthias Pohl
>Assignee: Matthias Pohl
>Priority: Major
>
> While working on FLINK-34672 I noticed that
> {{JobMasterServiceLeadershipRunnerTest#testJobMasterServiceLeadershipRunnerCloseWhenElectionServiceGrantLeaderShip}}
> is disabled without a reason.
> It looks like I disabled it accidentally as part of FLINK-31783.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] [FLINK-34893] Bump checkstyle to 9.3 [flink]
RyanSkraba commented on code in PR #24540: URL: https://github.com/apache/flink/pull/24540#discussion_r1532074768 ## tools/maven/checkstyle.xml: ## @@ -238,16 +238,9 @@ This file is based on the checkstyle file of Apache Beam. - Review Comment: ```suggestion ``` The `scope` property was renamed, but it still exists. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-34893] Bump checkstyle to 9.3 [flink]
RyanSkraba commented on code in PR #24540: URL: https://github.com/apache/flink/pull/24540#discussion_r1532031293 ## tools/maven/checkstyle.xml: ## @@ -238,16 +238,9 @@ This file is based on the checkstyle file of Apache Beam. Review Comment: While you're here! ```suggestion ``` ## tools/maven/checkstyle.xml: ## @@ -238,16 +238,9 @@ This file is based on the checkstyle file of Apache Beam. - Review Comment: ```suggestion ``` The `scope` property was renamed, but it still exists. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Closed] (FLINK-34745) Parsing temporal table join throws cryptic exceptions
[
https://issues.apache.org/jira/browse/FLINK-34745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dawid Wysakowicz closed FLINK-34745.
Resolution: Fixed
Fixed in 4142c4386a92f1ec5016583f4832f8869782765e
> Parsing temporal table join throws cryptic exceptions
> -
>
> Key: FLINK-34745
> URL: https://issues.apache.org/jira/browse/FLINK-34745
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.19.0
>Reporter: Dawid Wysakowicz
>Assignee: Dawid Wysakowicz
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.20.0
>
>
> 1. Wrong expression type in {{AS OF}}:
> {code}
> SELECT * " +
> "FROM Orders AS o JOIN " +
> "RatesHistoryWithPK FOR SYSTEM_TIME AS OF 'o.rowtime' AS r " +
> "ON o.currency = r.currency
> {code}
> throws:
> {code}
> java.lang.AssertionError: cannot convert CHAR literal to class
> org.apache.calcite.util.TimestampString
> {code}
> 2. Not a simple table reference in {{AS OF}}
> {code}
> SELECT * " +
> "FROM Orders AS o JOIN " +
> "RatesHistoryWithPK FOR SYSTEM_TIME AS OF o.rowtime + INTERVAL '1'
> SECOND AS r " +
> "ON o.currency = r.currency
> {code}
> throws:
> {code}
> java.lang.AssertionError: no unique expression found for {id: o.rowtime,
> prefix: 1}; count is 0
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] [FLINK-34745] Improve validations for a period in Time Travel [flink]
dawidwys merged PR #24534: URL: https://github.com/apache/flink/pull/24534 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-34896][table] Migrate CorrelateSortToRankRule to java [flink]
flinkbot commented on PR #24545: URL: https://github.com/apache/flink/pull/24545#issuecomment-2009564801 ## CI report: * 83bedd5de8ddc74d3095feb937b1728b611db4a1 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-34896) Migrate CorrelateSortToRankRule
[ https://issues.apache.org/jira/browse/FLINK-34896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-34896: --- Labels: pull-request-available (was: ) > Migrate CorrelateSortToRankRule > --- > > Key: FLINK-34896 > URL: https://issues.apache.org/jira/browse/FLINK-34896 > Project: Flink > Issue Type: Sub-task >Reporter: Sergey Nuyanzin >Assignee: Sergey Nuyanzin >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] [FLINK-34896][table] Migrate CorrelateSortToRankRule to java [flink]
snuyanzin opened a new pull request, #24545: URL: https://github.com/apache/flink/pull/24545 ## What is the purpose of the change A bit reasoning for the change >The reason is that since 1.28.0 ( [CALCITE-4787](https://issues.apache.org/jira/browse/CALCITE-4787) - Move core to use Immutables instead of ImmutableBeans ) Calcite started to use Immutables (https://immutables.github.io/) and since 1.29.0 removed ImmutableBeans ( [CALCITE-4839](https://issues.apache.org/jira/browse/CALCITE-4839) - Remove remnants of ImmutableBeans post 1.28 release ). All rule configuration related api which is not Immutables based is marked as deprecated. Since Immutables implies code generation while java compilation it is seems impossible to use for rules in Scala code. The PR migrates `CorrelateSortToRankRule` to java it doesn't touch `CorrelateSortToRankRuleTest` to be sure that java version continues passing it ## Verifying this change This change is already covered by existing tests ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: ( no) - The serializers: (no ) - The runtime per-record code paths (performance sensitive): (no ) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no) - The S3 file system connector: (no ) ## Documentation - Does this pull request introduce a new feature? (no) - If yes, how is the feature documented? (not applicable) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (FLINK-34896) Mi
Sergey Nuyanzin created FLINK-34896: --- Summary: Mi Key: FLINK-34896 URL: https://issues.apache.org/jira/browse/FLINK-34896 Project: Flink Issue Type: Sub-task Reporter: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (FLINK-34896) Migrate CorrelateSortToRankRule
[ https://issues.apache.org/jira/browse/FLINK-34896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Nuyanzin reassigned FLINK-34896: --- Assignee: Sergey Nuyanzin > Migrate CorrelateSortToRankRule > --- > > Key: FLINK-34896 > URL: https://issues.apache.org/jira/browse/FLINK-34896 > Project: Flink > Issue Type: Sub-task >Reporter: Sergey Nuyanzin >Assignee: Sergey Nuyanzin >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-34896) Migrate CorrelateSortToRankRule
[ https://issues.apache.org/jira/browse/FLINK-34896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Nuyanzin updated FLINK-34896: Summary: Migrate CorrelateSortToRankRule (was: Mi) > Migrate CorrelateSortToRankRule > --- > > Key: FLINK-34896 > URL: https://issues.apache.org/jira/browse/FLINK-34896 > Project: Flink > Issue Type: Sub-task >Reporter: Sergey Nuyanzin >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] [FLINK-34239][core] Add copy() method for SerializerConfig [flink]
flinkbot commented on PR #24544: URL: https://github.com/apache/flink/pull/24544#issuecomment-2009531400 ## CI report: * 3d3ca469055b60fe00261751c3cb86c988c7c7bf UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-34239) Introduce a deep copy method of SerializerConfig for merging with Table configs in org.apache.flink.table.catalog.DataTypeFactoryImpl
[ https://issues.apache.org/jira/browse/FLINK-34239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-34239: --- Labels: pull-request-available (was: ) > Introduce a deep copy method of SerializerConfig for merging with Table > configs in org.apache.flink.table.catalog.DataTypeFactoryImpl > -- > > Key: FLINK-34239 > URL: https://issues.apache.org/jira/browse/FLINK-34239 > Project: Flink > Issue Type: Sub-task > Components: API / Core >Affects Versions: 1.19.0 >Reporter: Zhanghao Chen >Assignee: Kumar Mallikarjuna >Priority: Major > Labels: pull-request-available > > *Problem* > Currently, > org.apache.flink.table.catalog.DataTypeFactoryImpl#createSerializerExecutionConfig > will create a deep-copy of the SerializerConfig and merge Table config into > it. However, the deep copy is done by manully calling the getter and setter > methods of SerializerConfig, and is prone to human errors, e.g. missing > copying a newly added field in SerializerConfig. > *Proposal* > Introduce a deep copy method for SerializerConfig and replace the curr impl > in > org.apache.flink.table.catalog.DataTypeFactoryImpl#createSerializerExecutionConfig. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] [FLINK-34239][core] Add copy() method for SerializerConfig [flink]
kumar-mallikarjuna opened a new pull request, #24544: URL: https://github.com/apache/flink/pull/24544 ## What is the purpose of the change `org.apache.flink.table.catalog.DataTypeFactoryImpl#createSerializerConfig` is currently manually making a copy of a `SerializerConfig` object. This is cumbersome and error-prone. This change moves this copying logic inside a copy method in `SerializerConfig(Impl)`. ## Brief change log - A copy method is introduced in `SerializerConfig` and `SerializerConfigImpl`. - `DataTypeFactoryImpl#createSerializerExecutionConfig` reuses this method. ## Verifying this change This change added tests and can be verified as follows: - Added a unit test for `SerializerConfigImply.copy()` ## Does this pull request potentially affect one of the following parts: - Dependencies (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no) - The serializers: (don't know - changes the underlying implementation but no functional change to the serializers) - The runtime per-record code paths (performance sensitive): (no) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no) - The S3 file system connector: (no) ## Documentation - Does this pull request introduce a new feature? (no) - If yes, how is the feature documented? (not applicable) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-34895][table] Migrate FlinkRewriteSubQueryRule to java [flink]
flinkbot commented on PR #24543: URL: https://github.com/apache/flink/pull/24543#issuecomment-2009485635 ## CI report: * 7dc3990422cb330fb1aa8fcdd50872f74dcb283e UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-26088][Connectors/ElasticSearch] Add Elasticsearch 8.0 support [flink-connector-elasticsearch]
mtfelisb commented on PR #53: URL: https://github.com/apache/flink-connector-elasticsearch/pull/53#issuecomment-2009477539 > Thanks @mtfelisb, looks good to me. Could you please rebase and squash all commits? Thank you for the review, @reswqa! Just did it :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] [FLINK-34895][table] Migrate FlinkRewriteSubQueryRule to java [flink]
snuyanzin opened a new pull request, #24543: URL: https://github.com/apache/flink/pull/24543 ## What is the purpose of the change A bit reasoning for the change >The reason is that since 1.28.0 ( [CALCITE-4787](https://issues.apache.org/jira/browse/CALCITE-4787) - Move core to use Immutables instead of ImmutableBeans ) Calcite started to use Immutables (https://immutables.github.io/) and since 1.29.0 removed ImmutableBeans ( [CALCITE-4839](https://issues.apache.org/jira/browse/CALCITE-4839) - Remove remnants of ImmutableBeans post 1.28 release ). All rule configuration related api which is not Immutables based is marked as deprecated. Since Immutables implies code generation while java compilation it is seems impossible to use for rules in Scala code. The PR migrates `FlinkRewriteSubQueryRule` to java it doesn't touch `FlinkRewriteSubQueryRuleTest` to be sure that java version continues passing it ## Verifying this change This change is already covered by existing tests ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: ( no) - The serializers: (no ) - The runtime per-record code paths (performance sensitive): (no ) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no) - The S3 file system connector: (no ) ## Documentation - Does this pull request introduce a new feature? (no) - If yes, how is the feature documented? (not applicable) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (FLINK-34895) Migrate FlinkRewriteSubQueryRule
Sergey Nuyanzin created FLINK-34895: --- Summary: Migrate FlinkRewriteSubQueryRule Key: FLINK-34895 URL: https://issues.apache.org/jira/browse/FLINK-34895 Project: Flink Issue Type: Sub-task Components: Table SQL / Planner Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-34895) Migrate FlinkRewriteSubQueryRule
[ https://issues.apache.org/jira/browse/FLINK-34895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-34895: --- Labels: pull-request-available (was: ) > Migrate FlinkRewriteSubQueryRule > > > Key: FLINK-34895 > URL: https://issues.apache.org/jira/browse/FLINK-34895 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / Planner >Reporter: Sergey Nuyanzin >Assignee: Sergey Nuyanzin >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] [FLINK-34549][API] Introduce config, context and processingTimerService for DataStream API V2 [flink]
reswqa opened a new pull request, #24541: URL: https://github.com/apache/flink/pull/24541 ## What is the purpose of the change *Implementation of FLIP-410. This PR will Introduce config, context and processingTimerService for DataStream API V2.* ## Brief change log - *Introduce ProcessConfigurable to configurate processing of stream. * - *Introduce RuntimeContext* - *Support process timer service* ## Verifying this change Can be verified by unit tests. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no - The serializers: no - The runtime per-record code paths (performance sensitive): no - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? no -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-34894) Migrate JoinDependentConditionDerivationRule
[ https://issues.apache.org/jira/browse/FLINK-34894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-34894: --- Labels: pull-request-available (was: ) > Migrate JoinDependentConditionDerivationRule > > > Key: FLINK-34894 > URL: https://issues.apache.org/jira/browse/FLINK-34894 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / Planner >Reporter: Sergey Nuyanzin >Assignee: Sergey Nuyanzin >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-34672) HA deadlock between JobMasterServiceLeadershipRunner and DefaultLeaderElectionService
[
https://issues.apache.org/jira/browse/FLINK-34672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias Pohl updated FLINK-34672:
--
Affects Version/s: 1.19.0
1.17.2
1.20.0
> HA deadlock between JobMasterServiceLeadershipRunner and
> DefaultLeaderElectionService
> -
>
> Key: FLINK-34672
> URL: https://issues.apache.org/jira/browse/FLINK-34672
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
>Affects Versions: 1.17.2, 1.19.0, 1.18.1, 1.20.0
>Reporter: Chesnay Schepler
>Assignee: Matthias Pohl
>Priority: Major
> Fix For: 1.18.2, 1.20.0, 1.19.1
>
>
> We recently observed a deadlock in the JM within the HA system.
> (see below for the thread dump)
> [~mapohl] and I looked a bit into it and there appears to be a race condition
> when leadership is revoked while a JobMaster is being started.
> It appears to be caused by
> {{JobMasterServiceLeadershipRunner#createNewJobMasterServiceProcess}}
> forwarding futures while holding a lock; depending on whether the forwarded
> future is already complete the next stage may or may not run while holding
> that same lock.
> We haven't determined yet whether we should be holding that lock or not.
> {code}
> "DefaultLeaderElectionService-leadershipOperationExecutor-thread-1" #131
> daemon prio=5 os_prio=0 cpu=157.44ms elapsed=78749.65s tid=0x7f531f43d000
> nid=0x19d waiting for monitor entry [0x7f53084fd000]
>java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.flink.runtime.jobmaster.JobMasterServiceLeadershipRunner.runIfStateRunning(JobMasterServiceLeadershipRunner.java:462)
> - waiting to lock <0xf1c0e088> (a java.lang.Object)
> at
> org.apache.flink.runtime.jobmaster.JobMasterServiceLeadershipRunner.revokeLeadership(JobMasterServiceLeadershipRunner.java:397)
> at
> org.apache.flink.runtime.leaderelection.DefaultLeaderElectionService.notifyLeaderContenderOfLeadershipLoss(DefaultLeaderElectionService.java:484)
> at
> org.apache.flink.runtime.leaderelection.DefaultLeaderElectionService$$Lambda$1252/0x000840ddec40.accept(Unknown
> Source)
> at java.util.HashMap.forEach(java.base@11.0.22/HashMap.java:1337)
> at
> org.apache.flink.runtime.leaderelection.DefaultLeaderElectionService.onRevokeLeadershipInternal(DefaultLeaderElectionService.java:452)
> at
> org.apache.flink.runtime.leaderelection.DefaultLeaderElectionService$$Lambda$1251/0x000840dcf840.run(Unknown
> Source)
> at
> org.apache.flink.runtime.leaderelection.DefaultLeaderElectionService.lambda$runInLeaderEventThread$3(DefaultLeaderElectionService.java:549)
> - locked <0xf0e3f4d8> (a java.lang.Object)
> at
> org.apache.flink.runtime.leaderelection.DefaultLeaderElectionService$$Lambda$1075/0x000840c23040.run(Unknown
> Source)
> at
> java.util.concurrent.CompletableFuture$AsyncRun.run(java.base@11.0.22/CompletableFuture.java:1736)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.22/ThreadPoolExecutor.java:1128)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.22/ThreadPoolExecutor.java:628)
> at java.lang.Thread.run(java.base@11.0.22/Thread.java:829)
> {code}
> {code}
> "jobmanager-io-thread-1" #636 daemon prio=5 os_prio=0 cpu=125.56ms
> elapsed=78699.01s tid=0x7f5321c6e800 nid=0x396 waiting for monitor entry
> [0x7f530567d000]
>java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.flink.runtime.leaderelection.DefaultLeaderElectionService.hasLeadership(DefaultLeaderElectionService.java:366)
> - waiting to lock <0xf0e3f4d8> (a java.lang.Object)
> at
> org.apache.flink.runtime.leaderelection.DefaultLeaderElection.hasLeadership(DefaultLeaderElection.java:52)
> at
> org.apache.flink.runtime.jobmaster.JobMasterServiceLeadershipRunner.isValidLeader(JobMasterServiceLeadershipRunner.java:509)
> at
> org.apache.flink.runtime.jobmaster.JobMasterServiceLeadershipRunner.lambda$forwardIfValidLeader$15(JobMasterServiceLeadershipRunner.java:520)
> - locked <0xf1c0e088> (a java.lang.Object)
> at
> org.apache.flink.runtime.jobmaster.JobMasterServiceLeadershipRunner$$Lambda$1320/0x000840e1a840.accept(Unknown
> Source)
> at
> java.util.concurrent.CompletableFuture.uniWhenComplete(java.base@11.0.22/CompletableFuture.java:859)
> at
> java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(java.base@11.0.22/CompletableFuture.java:837)
> at
>
Re: [PR] [FLINK-34894][table] Migrate JoinDependentConditionDerivationRule to java [flink]
flinkbot commented on PR #24542: URL: https://github.com/apache/flink/pull/24542#issuecomment-2009455915 ## CI report: * 564b0ac7360a5212a2a22bce9968c2dbe71bcc1b UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-34549][API] Introduce config, context and processingTimerService for DataStream API V2 [flink]
flinkbot commented on PR #24541: URL: https://github.com/apache/flink/pull/24541#issuecomment-2009426855 ## CI report: * 08547b7fa9baa619ec7b62bc893ccc6abcc86bb3 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-34643) JobIDLoggingITCase failed
[
https://issues.apache.org/jira/browse/FLINK-34643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828930#comment-17828930
]
Ryan Skraba commented on FLINK-34643:
-
*
[https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=58429=logs=8fd9202e-fd17-5b26-353c-ac1ff76c8f28=ea7cf968-e585-52cb-e0fc-f48de023a7ca=8243]
Here are the older build logs that should have been added over the weekend.
*
[https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=58320=logs=2c3cbe13-dee0-5837-cf47-3053da9a8a78=b78d9d30-509a-5cea-1fef-db7abaa325ae=8349]
*
[https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=58320=logs=8fd9202e-fd17-5b26-353c-ac1ff76c8f28=ea7cf968-e585-52cb-e0fc-f48de023a7ca=8244]
*
[https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=58352=logs=a596f69e-60d2-5a4b-7d39-dc69e4cdaed3=712ade8c-ca16-5b76-3acd-14df33bc1cb1=8210]
*
[https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=58352=logs=8fd9202e-fd17-5b26-353c-ac1ff76c8f28=ea7cf968-e585-52cb-e0fc-f48de023a7ca=7904]
*
[https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=58359=logs=2c3cbe13-dee0-5837-cf47-3053da9a8a78=b78d9d30-509a-5cea-1fef-db7abaa325ae=8354]
*
[https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=58359=logs=8fd9202e-fd17-5b26-353c-ac1ff76c8f28=ea7cf968-e585-52cb-e0fc-f48de023a7ca=8249]
*
[https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=58366=logs=2c3cbe13-dee0-5837-cf47-3053da9a8a78=b78d9d30-509a-5cea-1fef-db7abaa325ae=8354]
*
[https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=58366=logs=8fd9202e-fd17-5b26-353c-ac1ff76c8f28=ea7cf968-e585-52cb-e0fc-f48de023a7ca=7903]
*
[https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=58398=logs=8fd9202e-fd17-5b26-353c-ac1ff76c8f28=ea7cf968-e585-52cb-e0fc-f48de023a7ca=8249]
> JobIDLoggingITCase failed
> -
>
> Key: FLINK-34643
> URL: https://issues.apache.org/jira/browse/FLINK-34643
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
>Affects Versions: 1.20.0
>Reporter: Matthias Pohl
>Assignee: Roman Khachatryan
>Priority: Major
> Labels: pull-request-available, test-stability
> Fix For: 1.20.0
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=58187=logs=8fd9202e-fd17-5b26-353c-ac1ff76c8f28=ea7cf968-e585-52cb-e0fc-f48de023a7ca=7897
> {code}
> Mar 09 01:24:23 01:24:23.498 [ERROR] Tests run: 1, Failures: 0, Errors: 1,
> Skipped: 0, Time elapsed: 4.209 s <<< FAILURE! -- in
> org.apache.flink.test.misc.JobIDLoggingITCase
> Mar 09 01:24:23 01:24:23.498 [ERROR]
> org.apache.flink.test.misc.JobIDLoggingITCase.testJobIDLogging(ClusterClient)
> -- Time elapsed: 1.459 s <<< ERROR!
> Mar 09 01:24:23 java.lang.IllegalStateException: Too few log events recorded
> for org.apache.flink.runtime.jobmaster.JobMaster (12) - this must be a bug in
> the test code
> Mar 09 01:24:23 at
> org.apache.flink.util.Preconditions.checkState(Preconditions.java:215)
> Mar 09 01:24:23 at
> org.apache.flink.test.misc.JobIDLoggingITCase.assertJobIDPresent(JobIDLoggingITCase.java:148)
> Mar 09 01:24:23 at
> org.apache.flink.test.misc.JobIDLoggingITCase.testJobIDLogging(JobIDLoggingITCase.java:132)
> Mar 09 01:24:23 at java.lang.reflect.Method.invoke(Method.java:498)
> Mar 09 01:24:23 at
> java.util.concurrent.RecursiveAction.exec(RecursiveAction.java:189)
> Mar 09 01:24:23 at
> java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
> Mar 09 01:24:23 at
> java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
> Mar 09 01:24:23 at
> java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
> Mar 09 01:24:23 at
> java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
> Mar 09 01:24:23
> {code}
> The other test failures of this build were also caused by the same test:
> *
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=58187=logs=2c3cbe13-dee0-5837-cf47-3053da9a8a78=b78d9d30-509a-5cea-1fef-db7abaa325ae=8349
> *
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=58187=logs=a596f69e-60d2-5a4b-7d39-dc69e4cdaed3=712ade8c-ca16-5b76-3acd-14df33bc1cb1=8209
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-34770) Unify download links of legacy connector to fat jar
[ https://issues.apache.org/jira/browse/FLINK-34770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-34770: --- Labels: pull-request-available (was: ) > Unify download links of legacy connector to fat jar > --- > > Key: FLINK-34770 > URL: https://issues.apache.org/jira/browse/FLINK-34770 > Project: Flink > Issue Type: Sub-task > Components: chinese-translation, Documentation, Flink CDC >Affects Versions: cdc-3.1.0 >Reporter: LvYanquan >Priority: Minor > Labels: pull-request-available > Fix For: cdc-3.1.0 > > > Some links will lead to download fat jar, but some links will lead to > download thin jar, and some are invalid. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] [FLINK-34893] Bump checkstyle to 9.3 [flink]
snuyanzin opened a new pull request, #24540: URL: https://github.com/apache/flink/pull/24540 ## What is the purpose of the change The issue with current checkstyle is that there is checkstyle IntellijIdea plugin And recently it dropped checkstyle 8 support [1] At the same time we can not move to Checkstyle 10 since 10.x requires java 11+ Also it moves version to properties to allow build with different checkstyle versions for instance after this PR it would be possible to build with checkstyle 10 as well (leveraging jdk11+) like `mvn clean install -DskipTests -Dcheckstyle.version=10.14.2` [1] https://github.com/jshiell/checkstyle-idea/blob/main/CHANGELOG.md ## Verifying this change This change is a trivial rework ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes ) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: ( no) - The serializers: ( no) - The runtime per-record code paths (performance sensitive): ( no) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: ( no) - The S3 file system connector: ( no ) ## Documentation - Does this pull request introduce a new feature? ( no) - If yes, how is the feature documented? (not applicable ) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Backport translated Chinese docs in master to release-3.0 branch [flink-cdc]
leonardBang merged PR #3179: URL: https://github.com/apache/flink-cdc/pull/3179 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (FLINK-34893) Bump Checkstyle to 9+
Sergey Nuyanzin created FLINK-34893: --- Summary: Bump Checkstyle to 9+ Key: FLINK-34893 URL: https://issues.apache.org/jira/browse/FLINK-34893 Project: Flink Issue Type: Bug Components: Build System Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin The issue with current checkstyle is that there is checkstyle IntellijIdea plugin And recently it dropped checkstyle 8 support [1] At the same time we can not move to Checkstyle 10 since 10.x requires java 11+ [1] https://github.com/jshiell/checkstyle-idea/blob/main/CHANGELOG.md -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-34894) Migrate JoinDependentConditionDerivationRule
Sergey Nuyanzin created FLINK-34894: --- Summary: Migrate JoinDependentConditionDerivationRule Key: FLINK-34894 URL: https://issues.apache.org/jira/browse/FLINK-34894 Project: Flink Issue Type: Sub-task Components: Table SQL / Planner Reporter: Sergey Nuyanzin Assignee: Sergey Nuyanzin -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] [FLINK-34892][ci] Fix python test failure due to config file change [flink-connector-aws]
z3d1k commented on code in PR #133:
URL:
https://github.com/apache/flink-connector-aws/pull/133#discussion_r1531899935
##
flink-python/dev/integration_test.sh:
##
@@ -41,10 +41,5 @@ FLINK_PYTHON_DIR=$(dirname "$CURRENT_DIR")
# set the FLINK_TEST_LIB_DIR to
"/flink-connector-aws/flink-python-connector-aws/target/dep..."
export FLINK_TEST_LIBS="${FLINK_PYTHON_DIR}/target/test-dependencies/*"
-# Temporarily update the installed 'pyflink_gateway_server.py' files with the
new one
-# Needed only until Flink 1.19 release
-echo "Checking ${FLINK_PYTHON_DIR} for 'pyflink_gateway_server.py'"
-find "${FLINK_PYTHON_DIR}/.tox" -name pyflink_gateway_server.py -exec cp
"${FLINK_PYTHON_DIR}/pyflink/pyflink_gateway_server.py" {} \;
Review Comment:
Removed to use `pyflink_gateway_server.py` provided by pyflink instead of
local copy.
##
flink-python/tox.ini:
##
@@ -21,7 +21,7 @@
# in multiple virtualenvs. This configuration file will run the
# test suite on all supported python versions.
# new environments will be excluded by default unless explicitly added to
envlist.
-envlist = {py37, py38, py39, py310}-cython
Review Comment:
The current fix does not work for python 3.7 because there is no 1.19
package released for this python version.
Since dependency on `apache-flink` python package is set as
`apache-flink>=1.17.0` - only the latest available Flink version is used during
testing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[PR] [FLINK-34892][ci] Fix python test failure due to config file change [flink-connector-aws]
z3d1k opened a new pull request, #133: URL: https://github.com/apache/flink-connector-aws/pull/133 ## Purpose of the change Removing `pyflink_gateway_server.py` override after Flink 1.19.0 release to support configuration file change. Override was initially introduced [here](https://github.com/apache/flink-connector-kafka/pull/69) and replicated for AWS connectors. ## Verifying this change Please make sure both new and modified tests in this PR follows the conventions defined in our code quality guide: https://flink.apache.org/contributing/code-style-and-quality-common.html#testing Local test run ## Significant changes *(Please check any boxes [x] if the answer is "yes". You can first publish the PR and check them afterwards, for convenience.)* - [ ] Dependencies have been added or upgraded - [ ] Public API has been changed (Public API is any class annotated with `@Public(Evolving)`) - [ ] Serializers have been changed - [ ] New feature has been introduced - If yes, how is this documented? not applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-34526][runtime] Actively disconnect the TM in RM to reduce restart time [flink]
flinkbot commented on PR #24539: URL: https://github.com/apache/flink/pull/24539#issuecomment-2009214558 ## CI report: * f4747e921f746298229adde25eefcb9f2b8e0cf4 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-34718) KeyedPartitionWindowedStream and NonPartitionWindowedStream IllegalStateException in AZP
[
https://issues.apache.org/jira/browse/FLINK-34718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828931#comment-17828931
]
Ryan Skraba commented on FLINK-34718:
-
*
[https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=58429=logs=8fd9202e-fd17-5b26-353c-ac1ff76c8f28=ea7cf968-e585-52cb-e0fc-f48de023a7ca=9664]
(before the fix was committed to master)
> KeyedPartitionWindowedStream and NonPartitionWindowedStream
> IllegalStateException in AZP
>
>
> Key: FLINK-34718
> URL: https://issues.apache.org/jira/browse/FLINK-34718
> Project: Flink
> Issue Type: Bug
> Components: API / DataStream
>Affects Versions: 1.20.0
>Reporter: Ryan Skraba
>Assignee: Ryan Skraba
>Priority: Critical
> Labels: pull-request-available, test-stability
> Fix For: 1.20.0
>
>
> [https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=58320=logs=8fd9202e-fd17-5b26-353c-ac1ff76c8f28=ea7cf968-e585-52cb-e0fc-f48de023a7ca=9646]
> 18 of the KeyedPartitionWindowedStreamITCase and
> NonKeyedPartitionWindowedStreamITCase unit tests introduced in FLINK-34543
> are failing in the adaptive scheduler profile, with errors similar to:
> {code:java}
> Mar 15 01:54:12 Caused by: java.lang.IllegalStateException: The adaptive
> scheduler supports pipelined data exchanges (violated by MapPartition
> (org.apache.flink.streaming.runtime.tasks.OneInputStreamTask) ->
> ddb598ad156ed281023ba4eebbe487e3).
> Mar 15 01:54:12 at
> org.apache.flink.util.Preconditions.checkState(Preconditions.java:215)
> Mar 15 01:54:12 at
> org.apache.flink.runtime.scheduler.adaptive.AdaptiveScheduler.assertPreconditions(AdaptiveScheduler.java:438)
> Mar 15 01:54:12 at
> org.apache.flink.runtime.scheduler.adaptive.AdaptiveScheduler.(AdaptiveScheduler.java:356)
> Mar 15 01:54:12 at
> org.apache.flink.runtime.scheduler.adaptive.AdaptiveSchedulerFactory.createInstance(AdaptiveSchedulerFactory.java:124)
> Mar 15 01:54:12 at
> org.apache.flink.runtime.jobmaster.DefaultSlotPoolServiceSchedulerFactory.createScheduler(DefaultSlotPoolServiceSchedulerFactory.java:121)
> Mar 15 01:54:12 at
> org.apache.flink.runtime.jobmaster.JobMaster.createScheduler(JobMaster.java:384)
> Mar 15 01:54:12 at
> org.apache.flink.runtime.jobmaster.JobMaster.(JobMaster.java:361)
> Mar 15 01:54:12 at
> org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.internalCreateJobMasterService(DefaultJobMasterServiceFactory.java:128)
> Mar 15 01:54:12 at
> org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.lambda$createJobMasterService$0(DefaultJobMasterServiceFactory.java:100)
> Mar 15 01:54:12 at
> org.apache.flink.util.function.FunctionUtils.lambda$uncheckedSupplier$4(FunctionUtils.java:112)
> Mar 15 01:54:12 at
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
> Mar 15 01:54:12 ... 4 more
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-34890) io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource :.java.sql.SQLException: ORA-01291: missing log file
[ https://issues.apache.org/jira/browse/FLINK-34890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828895#comment-17828895 ] Flink CDC Issue Import commented on FLINK-34890: Date: Nov 20, 2023, Author: https://github.com/jasonhewg so: Do you want to upgrade Flink cdc connectors. com. verica. 2.4.1 debezium. version 1.9.7. Final to 2.4.1. Final? Which version is the corresponding flink-cdc-connectors.com.ververica > io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource > :.java.sql.SQLException: ORA-01291: missing log file > - > > Key: FLINK-34890 > URL: https://issues.apache.org/jira/browse/FLINK-34890 > Project: Flink > Issue Type: Bug > Components: Flink CDC >Reporter: Flink CDC Issue Import >Priority: Major > Labels: github-import > > ### Search before asking > - [X] I searched in the > [issues](https://github.com/ververica/flink-cdc-connectors/issues) and found > nothing similar. > ### Flink version > 1.14.2 > ### Flink CDC version > 2.4.1 > ### Database and its version > oracle19c > ### Minimal reproduce step > 2023-11-03 13:37:44.675 WARN > [debezium-oracleconnector-oracle_logminer-change-event-source-coordinator] > io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource : > Failed to start Oracle LogMiner session, retrying... > 2023-11-03 13:37:45.286 > ERROR[debezium-oracleconnector-oracle_logminer-change-event-source-coordinator] > io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource : > Failed to start Oracle LogMiner after '5' attempts. > java.sql.SQLException: ORA-01291: missing log file > ORA-06512: at "SYS.DBMS_LOGMNR", line 72 > ORA-06512: at line 1 > at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:509) > ~[com.oracle.database.jdbc-ojdbc8-19.3.0.0.jar!/:19.3.0.0.0] > at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:461) > ~[com.oracle.database.jdbc-ojdbc8-19.3.0.0.jar!/:19.3.0.0.0] > > In addition, I clean the archive logs by executing commands from the rman > tool command: > > Crosscheck archivelog all; > Delete noprompt archivelog until time 'sysdate -1'; > > Through the above command, the archived logs were retained for one day. > During the insertion, deletion, and modification of the table within one day, > I also encountered the issue of losing the logs mentioned above. > May I ask how to solve this problem。 > ### What did you expect to see? > flink cdc normal run > ### What did you see instead? > java.sql.SQLException: ORA-01291: missing log file > ORA-06512: at "SYS.DBMS_LOGMNR", line 72 > ORA-06512: at line 1 > > io.debezium.connector.common.BaseSourceTask : Going to restart connector > after 10 sec. after a retriable exception > 2023-11-03 16:09:11.458 INFO [pool-18-thread-1] > io.debezium.jdbc.JdbcConnection : Connection gracefully closed > 2023-11-03 16:09:11.459 INFO [debezium-engine] > io.debezium.embedded.EmbeddedEngine : Retrieable exception thrown, connector > will be restarted > org.apache.kafka.connect.errors.RetriableException: An exception occurred in > the change event producer. This connector will be restarted. > at > io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:46) > ~[debezium-core-1.9.7.Final.jar!/:1.9.7.Final] > ### Anything else? > _No response_ > ### Are you willing to submit a PR? > - [X] I'm willing to submit a PR! > Imported from GitHub > Url: https://github.com/apache/flink-cdc/issues/2615 > Created by: https://github.com/jasonhewg > Labels: bug, > Created at: Mon Dec 18 19:07:44 CST 2023 > State: open -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] [FLINK-34894][table] Migrate JoinDependentConditionDerivationRule to java [flink]
snuyanzin opened a new pull request, #24542: URL: https://github.com/apache/flink/pull/24542 ## What is the purpose of the change A bit reasoning for the change >The reason is that since 1.28.0 ( [CALCITE-4787](https://issues.apache.org/jira/browse/CALCITE-4787) - Move core to use Immutables instead of ImmutableBeans ) Calcite started to use Immutables (https://immutables.github.io/) and since 1.29.0 removed ImmutableBeans ( [CALCITE-4839](https://issues.apache.org/jira/browse/CALCITE-4839) - Remove remnants of ImmutableBeans post 1.28 release ). All rule configuration related api which is not Immutables based is marked as deprecated. Since Immutables implies code generation while java compilation it is seems impossible to use for rules in Scala code. The PR migrates `JoinDependentConditionDerivationRule` to java it doesn't touch `JoinDependentConditionDerivationRuleTest` to be sure that java version continues passing it ## Verifying this change This change is already covered by existing tests ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: ( no) - The serializers: (no ) - The runtime per-record code paths (performance sensitive): (no ) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no) - The S3 file system connector: (no ) ## Documentation - Does this pull request introduce a new feature? (no) - If yes, how is the feature documented? (not applicable) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-34549) FLIP-410: Config, Context and Processing Timer Service of DataStream API V2
[ https://issues.apache.org/jira/browse/FLINK-34549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-34549: --- Labels: Umbrella pull-request-available (was: Umbrella) > FLIP-410: Config, Context and Processing Timer Service of DataStream API V2 > --- > > Key: FLINK-34549 > URL: https://issues.apache.org/jira/browse/FLINK-34549 > Project: Flink > Issue Type: Sub-task > Components: API / DataStream >Reporter: Weijie Guo >Assignee: Weijie Guo >Priority: Major > Labels: Umbrella, pull-request-available > > This is the umbrella ticket for FLIP-410: Config, Context and Processing > Timer Service of DataStream API V2. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] [FLINK-34739][docs] Translate "Connectors" Page for Flink CDC Chinese Documentation [flink-cdc]
loserwang1024 commented on PR #3178: URL: https://github.com/apache/flink-cdc/pull/3178#issuecomment-2009137069 @leonardBang , @LYanquan , @PatrickRen , CC -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (FLINK-34871) Oracle add data has error
Flink CDC Issue Import created FLINK-34871:
--
Summary: Oracle add data has error
Key: FLINK-34871
URL: https://issues.apache.org/jira/browse/FLINK-34871
Project: Flink
Issue Type: Bug
Components: Flink CDC
Reporter: Flink CDC Issue Import
### Search before asking
- [X] I searched in the
[issues|https://github.com/ververica/flink-cdc-connectors/issues) and found
nothing similar.
### Flink version
1.18.1
### Flink CDC version
3.0.0
### Database and its version
oracle 11g
### Minimal reproduce step
I run the document at
https://ververica.github.io/flink-cdc-connectors/release-3.0/content/connectors/oracle-cdc.html,when
i add a data to table,it has some error
Properties debeziumProperties = new Properties();
debeziumProperties.setProperty("log.mining.strategy", "online_catalog");
debeziumProperties.setProperty("include.schema.changes", "true");
debeziumProperties.setProperty("value.converter.schemas.enable",
"true");
OracleSourceBuilder.OracleIncrementalSource oracleChangeEventSource =
new OracleSourceBuilder()
.hostname("hostname")
.port(1521)
.databaseList("ORCL")
.schemaList("FLINKUSER")
.tableList("FLINKUSER.CREATETABLE")
.username("flinkuser")
.password("flinkpw")
.deserializer(new JsonDebeziumDeserializationSchema())
.includeSchemaChanges(true) // output the schema
changes as well
.startupOptions(StartupOptions.initial())
.debeziumProperties(debeziumProperties)
.splitSize(2)
.build();
StreamExecutionEnvironment env =
StreamExecutionEnvironment.getExecutionEnvironment();
env.enableCheckpointing(3000L);
DataStreamSource oracleChangeEventStream = env.fromSource(
oracleChangeEventSource,
WatermarkStrategy.noWatermarks(),
"OracleParallelSource")
.setParallelism(4)
.setParallelism(1);
oracleChangeEventStream.print(];
![image|https://github.com/ververica/flink-cdc-connectors/assets/43168824/14d5e0e2-daed-4d8a-bce5-1ed83ccd5c88]
### What did you expect to see?
Is it my configuration problem
### What did you see instead?
null error
### Anything else?
_No response_
### Are you willing to submit a PR?
- [X] I'm willing to submit a PR!
Imported from GitHub
Url: https://github.com/apache/flink-cdc/issues/3056
Created by: [ccczhouxin|https://github.com/ccczhouxin]
Labels: bug,
Created at: Fri Feb 02 10:05:38 CST 2024
State: open
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-34892) Nightly AWS connectors build fails on running python tests
[
https://issues.apache.org/jira/browse/FLINK-34892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-34892:
---
Labels: pull-request-available (was: )
> Nightly AWS connectors build fails on running python tests
> --
>
> Key: FLINK-34892
> URL: https://issues.apache.org/jira/browse/FLINK-34892
> Project: Flink
> Issue Type: Bug
> Components: Connectors / AWS
>Affects Versions: aws-connector-4.2.0
>Reporter: Aleksandr Pilipenko
>Priority: Major
> Labels: pull-request-available
>
> Build for externalized python connector code fails:
> [https://github.com/apache/flink-connector-aws/actions/runs/8351768294/job/22860710449]
> {code:java}
> 2024-03-20T00:14:35.5215863Z __
> FlinkKinesisTest.test_kinesis_streams_sink __
> 2024-03-20T00:14:35.5216781Z
> .tox/py310-cython/lib/python3.10/site-packages/pyflink/testing/test_case_utils.py:149:
> in setUp
> 2024-03-20T00:14:35.5217584Z self.env =
> StreamExecutionEnvironment.get_execution_environment()
> 2024-03-20T00:14:35.5218901Z
> .tox/py310-cython/lib/python3.10/site-packages/pyflink/datastream/stream_execution_environment.py:876:
> in get_execution_environment
> 2024-03-20T00:14:35.5219751Z gateway = get_gateway()
> 2024-03-20T00:14:35.5220635Z
> .tox/py310-cython/lib/python3.10/site-packages/pyflink/java_gateway.py:64: in
> get_gateway
> 2024-03-20T00:14:35.5221378Z _gateway = launch_gateway()
> 2024-03-20T00:14:35.5222111Z
> .tox/py310-cython/lib/python3.10/site-packages/pyflink/java_gateway.py:110:
> in launch_gateway
> 2024-03-20T00:14:35.5222956Z p = launch_gateway_server_process(env, args)
> 2024-03-20T00:14:35.5223854Z
> .tox/py310-cython/lib/python3.10/site-packages/pyflink/pyflink_gateway_server.py:262:
> in launch_gateway_server_process
> 2024-03-20T00:14:35.5224649Z java_executable = find_java_executable()
> 2024-03-20T00:14:35.5225583Z
> .tox/py310-cython/lib/python3.10/site-packages/pyflink/pyflink_gateway_server.py:75:
> in find_java_executable
> 2024-03-20T00:14:35.5226449Z java_home =
> read_from_config(KEY_ENV_JAVA_HOME, None, flink_conf_file)
> 2024-03-20T00:14:35.5227099Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> 2024-03-20T00:14:35.5227450Z
> 2024-03-20T00:14:35.5227774Z key = 'env.java.home', default_value = None
> 2024-03-20T00:14:35.5228925Z flink_conf_file =
> '/home/runner/work/flink-connector-aws/flink-connector-aws/flink-python/.tox/py310-cython/lib/python3.10/site-packages/pyflink/conf/flink-conf.yaml'
> 2024-03-20T00:14:35.5229778Z
> 2024-03-20T00:14:35.5230010Z def read_from_config(key, default_value,
> flink_conf_file):
> 2024-03-20T00:14:35.5230581Z value = default_value
> 2024-03-20T00:14:35.5231236Z # get the realpath of tainted path value
> to avoid CWE22 problem that constructs a path or URI
> 2024-03-20T00:14:35.5232195Z # using the tainted value and might
> allow an attacker to access, modify, or test the existence
> 2024-03-20T00:14:35.5232940Z # of critical or sensitive files.
> 2024-03-20T00:14:35.5233417Z > with
> open(os.path.realpath(flink_conf_file), "r") as f:
> 2024-03-20T00:14:35.5234874Z E FileNotFoundError: [Errno 2] No such
> file or directory:
> '/home/runner/work/flink-connector-aws/flink-connector-aws/flink-python/.tox/py310-cython/lib/python3.10/site-packages/pyflink/conf/flink-conf.yaml'
> 2024-03-20T00:14:35.5235954Z
> 2024-03-20T00:14:35.5236484Z
> .tox/py310-cython/lib/python3.10/site-packages/pyflink/pyflink_gateway_server.py:58:
> FileNotFoundError {code}
> Failure started after the release of apache-flink python package for 1.19.0
> due to change of default config file provided within artifact.
>
>
> Issue comes from outdated copy of pyflink_gateway_server.py created as part
> of [https://github.com/apache/flink-connector-kafka/pull/69] (same change is
> duplicated in AWS connectors repository).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-34891) RemoteStorageScannerTest causes exit 239
Ryan Skraba created FLINK-34891:
---
Summary: RemoteStorageScannerTest causes exit 239
Key: FLINK-34891
URL: https://issues.apache.org/jira/browse/FLINK-34891
Project: Flink
Issue Type: Bug
Components: Runtime / Network
Affects Versions: 1.20.0
Reporter: Ryan Skraba
[https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=58432=logs=f0ac5c25-1168-55a5-07ff-0e88223afed9=50bf7a25-bdc4-5e56-5478-c7b4511dde53=9121]
{code:java}
Mar 20 01:22:54 01:22:54.671 [ERROR]at
org.apache.maven.lifecycle.internal.MojoExecutor.doExecute(MojoExecutor.java:351)
Mar 20 01:22:54 01:22:54.671 [ERROR]at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:215)
Mar 20 01:22:54 01:22:54.671 [ERROR]at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:171)
Mar 20 01:22:54 01:22:54.671 [ERROR]at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:163)
Mar 20 01:22:54 01:22:54.671 [ERROR]at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
Mar 20 01:22:54 01:22:54.671 [ERROR]at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
Mar 20 01:22:54 01:22:54.671 [ERROR]at
org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56)
Mar 20 01:22:54 01:22:54.671 [ERROR]at
org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
Mar 20 01:22:54 01:22:54.671 [ERROR]at
org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:294)
Mar 20 01:22:54 01:22:54.671 [ERROR]at
org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192)
Mar 20 01:22:54 01:22:54.671 [ERROR]at
org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105)
Mar 20 01:22:54 01:22:54.671 [ERROR]at
org.apache.maven.cli.MavenCli.execute(MavenCli.java:960)
Mar 20 01:22:54 01:22:54.671 [ERROR]at
org.apache.maven.cli.MavenCli.doMain(MavenCli.java:293)
Mar 20 01:22:54 01:22:54.671 [ERROR]at
org.apache.maven.cli.MavenCli.main(MavenCli.java:196)
Mar 20 01:22:54 01:22:54.671 [ERROR]at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
Mar 20 01:22:54 01:22:54.671 [ERROR]at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
Mar 20 01:22:54 01:22:54.671 [ERROR]at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
Mar 20 01:22:54 01:22:54.671 [ERROR]at
java.base/java.lang.reflect.Method.invoke(Method.java:566)
Mar 20 01:22:54 01:22:54.671 [ERROR]at
org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:282)
Mar 20 01:22:54 01:22:54.671 [ERROR]at
org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:225)
Mar 20 01:22:54 01:22:54.671 [ERROR]at
org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:406)
Mar 20 01:22:54 01:22:54.671 [ERROR]at
org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:347)
Mar 20 01:22:54 01:22:54.671 [ERROR]at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
Mar 20 01:22:54 01:22:54.671 [ERROR]at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
Mar 20 01:22:54 01:22:54.671 [ERROR]at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
Mar 20 01:22:54 01:22:54.671 [ERROR]at
java.base/java.lang.reflect.Method.invoke(Method.java:566)
Mar 20 01:22:54 01:22:54.671 [ERROR]at
org.apache.maven.wrapper.BootstrapMainStarter.start(BootstrapMainStarter.java:52)
Mar 20 01:22:54 01:22:54.671 [ERROR]at
org.apache.maven.wrapper.WrapperExecutor.execute(WrapperExecutor.java:161)
Mar 20 01:22:54 01:22:54.671 [ERROR]at
org.apache.maven.wrapper.MavenWrapperMain.main(MavenWrapperMain.java:73)
Mar 20 01:22:54 01:22:54.671 [ERROR] Caused by:
org.apache.maven.surefire.booter.SurefireBooterForkException: The forked VM
terminated without properly saying goodbye. VM crash or System.exit called?
Mar 20 01:22:54 01:22:54.671 [ERROR] Command was /bin/sh -c cd
'/__w/2/s/flink-runtime' && '/usr/lib/jvm/jdk-11.0.19+7/bin/java'
'-XX:+UseG1GC' '-Xms256m' '-XX:+IgnoreUnrecognizedVMOptions'
'--add-opens=java.base/java.util=ALL-UNNAMED'
'--add-opens=java.base/java.lang=ALL-UNNAMED'
'--add-opens=java.base/java.net=ALL-UNNAMED'
'--add-opens=java.base/java.io=ALL-UNNAMED'
'--add-opens=java.base/java.util.concurrent=ALL-UNNAMED' '-Xmx768m' '-jar'
'/__w/2/s/flink-runtime/target/surefire/surefirebooter-20240320011505720_97.jar'
'/__w/2/s/flink-runtime/target/surefire' '2024-03-20T01-12-28_109-jvmRun1'
[jira] [Assigned] (FLINK-34739) "Connectors" Page for Flink CDC Chinese Documentation
[ https://issues.apache.org/jira/browse/FLINK-34739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leonard Xu reassigned FLINK-34739: -- Assignee: Hongshun Wang > "Connectors" Page for Flink CDC Chinese Documentation > - > > Key: FLINK-34739 > URL: https://issues.apache.org/jira/browse/FLINK-34739 > Project: Flink > Issue Type: Sub-task > Components: chinese-translation, Documentation, Flink CDC >Affects Versions: cdc-3.1.0 >Reporter: LvYanquan >Assignee: Hongshun Wang >Priority: Minor > Labels: pull-request-available > Fix For: cdc-3.1.0 > > > Translate pipeline connector pages > [https://github.com/apache/flink-cdc/tree/master/docs/content/docs/connectors] > into Chinese. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] [FLINK-34770][cdc][doc] fix download links. [flink-cdc]
lvyanquan commented on code in PR #3182: URL: https://github.com/apache/flink-cdc/pull/3182#discussion_r1531951686 ## docs/content.zh/docs/connectors/legacy-flink-cdc-sources/mongodb-cdc.md: ## @@ -47,9 +47,9 @@ MongoDB CDC 连接器允许从 MongoDB 读取快照数据和增量数据。 本 ```下载链接仅适用于稳定版本。``` -下载 [flink-sql-connector-mongodb-cdc-3.0-SNAPSHOT.jar](https://repo1.maven.org/maven2/com/ververica/flink-connector-mongodb-cdc/3.0.0/flink-connector-mongodb-cdc-3.0.0.jar) 把它放在 `/lib/`. Review Comment: done. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-34860) [Bug] Jackson version conflicts among MySQL/TiDB/MongoDB connectors
[ https://issues.apache.org/jira/browse/FLINK-34860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828851#comment-17828851 ] Flink CDC Issue Import commented on FLINK-34860: Date: Wed Jan 10 11:31:39 CST 2024, Author: [leonardBang|https://github.com/leonardBang] +1 from my side, thanks @link3280 for raising up this ticket. > [Bug] Jackson version conflicts among MySQL/TiDB/MongoDB connectors > --- > > Key: FLINK-34860 > URL: https://issues.apache.org/jira/browse/FLINK-34860 > Project: Flink > Issue Type: Bug > Components: Flink CDC >Reporter: Flink CDC Issue Import >Priority: Major > Labels: github-import > > ### Search before asking > - [X] I searched in the > [issues|https://github.com/ververica/flink-cdc-connectors/issues) and found > nothing similar. > ### Flink version > 1.14 > ### Flink CDC version > 3.0.0 > ### Database and its version > MySQL 5.7 > ### Minimal reproduce step > Put `flink-sql-connector-tidb-cdc-3.0.0.jar`, > `flink-sql-connector-mysql-cdc-3.0.0.jar` and > `flink-sql-connector-mongodb-cdc-3.0.0.jar` into lib folder at Flink home > (Flink 1.14 in my case], then create a MySQL CDC table and submit a simple > query like `select * from mysql_cdc_table`, there would be an error: > `java.io.InvalidClassException: > com.ververica.cdc.connectors.shaded.com.fasterxml.jackson.databind.cfg.MapperConfig; > incompatible types for field _mapperFeatures` > It's caused by inconsistent jackson versions of the CDC connectors: > - `flink-sql-connector-mongodb-cdc-3.0.0.jar`: 2.10.2 > - `flink-sql-connector-mysql-cdc-3.0.0.jar`: 2.13.2 > - `flink-sql-connector-tidb-cdc-3.0.0.jar`: 2.12.3 > ### What did you expect to see? > All CDC connectors use the same jackson version. > ### What did you see instead? > Inconsistent jackson versions among the CDC connectors which conflicts with > each other. > ### Anything else? > _No response_ > ### Are you willing to submit a PR? > - [X] I'm willing to submit a PR! > Imported from GitHub > Url: https://github.com/apache/flink-cdc/issues/2983 > Created by: [link3280|https://github.com/link3280] > Labels: bug, > Assignee: [link3280|https://github.com/link3280] > Created at: Wed Jan 10 10:28:58 CST 2024 > State: open -- This message was sent by Atlassian Jira (v8.20.10#820010)
