[GitHub] [flink] flinkbot edited a comment on pull request #15004: [FLINK-21253][table-planner-blink] Support grouping set syntax for GroupWindowAggregate
flinkbot edited a comment on pull request #15004: URL: https://github.com/apache/flink/pull/15004#issuecomment-785052726 ## CI report: * 3af09855fc47130d93b11e6d6ba1c3dedb6574a5 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=13733) * e2d2a09c2abc55337de23906b4aa348d3502f50d UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Closed] (FLINK-21627) When you insert multiple inserts with statementSet, you modify multiple inserts with OPTIONS('table-name '=' XXX '), but only the first one takes effect
[
https://issues.apache.org/jira/browse/FLINK-21627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
godfrey he closed FLINK-21627.
--
Resolution: Fixed
Fixed in 1.13.0: c6147f2df2e81c1b1db6aa43b3a827a8c4148af1
> When you insert multiple inserts with statementSet, you modify multiple
> inserts with OPTIONS('table-name '=' XXX '), but only the first one takes
> effect
>
>
> Key: FLINK-21627
> URL: https://issues.apache.org/jira/browse/FLINK-21627
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.12.0
>Reporter: wangtaiyang
>Assignee: godfrey he
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.13.0
>
>
> {code:java}
> //代码占位符
> StatementSet statementSet = tableEnvironment.createStatementSet();
> String sql1 = "insert into test select a,b,c from test_a_12342 /*+
> OPTIONS('table-name'='test_a_1')*/";
> String sql2 = "insert into test select a,b,c from test_a_12342 /*+
> OPTIONS('table-name'='test_a_2')*/";
> statementSet.addInsertSql(sql1);
> statementSet.addInsertSql(sql2);
> statementSet.execute();
> {code}
> Sql code as above, in the final after the insert is put test_a_1 table data
> into the two times, and test_a_2 data did not insert, is excuse me this bug
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #14830: [FLINK-20487][table-planner-blink] Support to consume retractions for window aggregate operator
flinkbot edited a comment on pull request #14830: URL: https://github.com/apache/flink/pull/14830#issuecomment-770878871 ## CI report: * a2587cacaa7abea0059a6aadf16cbb1c5a4f31b3 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=13700) * c39b7a4ecde12aca916b2228ffe734598749fa9d Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16721) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] godfreyhe closed pull request #15559: [FLINK-21627][table-planner-blink] The digest of TableScan & Sink should consider table hints
godfreyhe closed pull request #15559: URL: https://github.com/apache/flink/pull/15559 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #14830: [FLINK-20487][table-planner-blink] Support to consume retractions for window aggregate operator
flinkbot edited a comment on pull request #14830: URL: https://github.com/apache/flink/pull/14830#issuecomment-770878871 ## CI report: * a2587cacaa7abea0059a6aadf16cbb1c5a4f31b3 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=13700) * c39b7a4ecde12aca916b2228ffe734598749fa9d UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-20487) Support to consume retractions for window aggregate operator
[
https://issues.apache.org/jira/browse/FLINK-20487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324415#comment-17324415
]
Andy commented on FLINK-20487:
--
[~flink-jira-bot] [~jark] [~godfreyhe] I'm still working on the issue, please
take a look at pull request when you have time, thanks a lot.
> Support to consume retractions for window aggregate operator
>
>
> Key: FLINK-20487
> URL: https://issues.apache.org/jira/browse/FLINK-20487
> Project: Flink
> Issue Type: New Feature
> Components: Table SQL / Planner
>Reporter: jiayue.yu
>Assignee: Andy
>Priority: Major
> Labels: pull-request-available, stale-assigned
> Fix For: 1.14.0
>
>
> {code}
> EXCEPTION: org.apache.flink.table.api.TableException: Group Window Aggregate:
> Retraction on windowed GroupBy Aggregate is not supported yet.
> org.apache.flink.table.api.TableException: Group Window Aggregate: Retraction
> on windowed GroupBy Aggregate is not supported yet. please re-check sql
> grammar. Note: Windowed GroupBy Aggregate should not follow anon-windowed
> GroupBy aggregation. at
> org.apache.flink.table.planner.plan.nodes.physical.stream.StreamExecGroupWindowAggregateBase.translateToPlanInternal(StreamExecGroupWindowAggregateBase.scala:138)
> at
> org.apache.flink.table.planner.plan.nodes.physical.stream.StreamExecGroupWindowAggregateBase.translateToPlanInternal(StreamExecGroupWindowAggregateBase.scala:54)
> at
> org.apache.flink.table.planner.plan.nodes.exec.ExecNode.translateToPlan(ExecNode.scala:58)
> at
> org.apache.flink.table.planner.plan.nodes.exec.ExecNode.translateToPlan$(ExecNode.scala:56)
> at
> org.apache.flink.table.planner.plan.nodes.physical.stream.StreamExecGroupWindowAggregateBase.translateToPlan(StreamExecGroupWindowAggregateBase.scala:54)
> at
> org.apache.flink.table.planner.plan.nodes.physical.stream.StreamExecCalc.translateToPlanInternal(StreamExecCalc.scala:54)
> at
> org.apache.flink.table.planner.plan.nodes.physical.stream.StreamExecCalc.translateToPlanInternal(StreamExecCalc.scala:39)
> at
> org.apache.flink.table.planner.plan.nodes.exec.ExecNode.translateToPlan(ExecNode.scala:58)
> at
> org.apache.flink.table.planner.plan.nodes.exec.ExecNode.translateToPlan$(ExecNode.scala:56)
> at
> org.apache.flink.table.planner.plan.nodes.physical.stream.StreamExecCalcBase.translateToPlan(StreamExecCalcBase.scala:38)
> at
> org.apache.flink.table.planner.plan.nodes.physical.stream.StreamExecSink.translateToTransformation(StreamExecSink.scala:184)
> at
> org.apache.flink.table.planner.plan.nodes.physical.stream.StreamExecSink.translateToPlanInternal(StreamExecSink.scala:91)
> at
> org.apache.flink.table.planner.plan.nodes.physical.stream.StreamExecSink.translateToPlanInternal(StreamExecSink.scala:48)
> at
> org.apache.flink.table.planner.plan.nodes.exec.ExecNode.translateToPlan(ExecNode.scala:58)
> at
> org.apache.flink.table.planner.plan.nodes.exec.ExecNode.translateToPlan$(ExecNode.scala:56)
> at
> org.apache.flink.table.planner.plan.nodes.physical.stream.StreamExecSink.translateToPlan(StreamExecSink.scala:48)
> at
> org.apache.flink.table.planner.delegation.StreamPlanner.$anonfun$translateToPlan$1(StreamPlanner.scala:60)
> at
> scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:233) at
> scala.collection.Iterator.foreach(Iterator.scala:937) at
> scala.collection.Iterator.foreach$(Iterator.scala:937) at
> scala.collection.AbstractIterator.foreach(Iterator.scala:1425) at
> scala.collection.IterableLike.foreach(IterableLike.scala:70) at
> scala.collection.IterableLike.foreach$(IterableLike.scala:69) at
> scala.collection.AbstractIterable.foreach(Iterable.scala:54) at
> scala.collection.TraversableLike.map(TraversableLike.scala:233) at
> scala.collection.TraversableLike.map$(TraversableLike.scala:226) at
> scala.collection.AbstractTraversable.map(Traversable.scala:104) at
> org.apache.flink.table.planner.delegation.StreamPlanner.translateToPlan(StreamPlanner.scala:59)
> at
> org.apache.flink.table.planner.delegation.PlannerBase.translate(PlannerBase.scala:153)
> at
> org.apache.flink.table.api.internal.TableEnvironmentImpl.translate(TableEnvironmentImpl.java:682)
> at
> org.apache.flink.table.api.internal.TableEnvironmentImpl.insertIntoInternal(TableEnvironmentImpl.java:355)
> at
> org.apache.flink.table.api.internal.TableEnvironmentImpl.insertInto(TableEnvironmentImpl.java:334)
> {code}
>
> CASE:
> {code:sql}
> SELECT
> DATE_FORMAT(tumble_end(ROWTIME ,interval '1' hour),'-MM-dd HH') as
> stat_time,
> count(crypto_customer_number) first_phone_num
> FROM (
> SELECT
> ROWTIME,
> crypto_customer_number,
> row_number() over(partition by crypto_customer_number order by ROWTIME ) as
> rn
> FROM
[jira] [Commented] (FLINK-22099) Fix bug for semi/anti window join.
[ https://issues.apache.org/jira/browse/FLINK-22099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324414#comment-17324414 ] Andy commented on FLINK-22099: -- [~flink-jira-bot] [~jark] [~godfreyhe] I'm still working on the issue, please take a look at pull request when you have time, thanks a lot. > Fix bug for semi/anti window join. > -- > > Key: FLINK-22099 > URL: https://issues.apache.org/jira/browse/FLINK-22099 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / Planner >Reporter: Andy >Assignee: Andy >Priority: Minor > Labels: pull-request-available > Fix For: 1.13.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-22098) Fix bug for window join: plan is wrong if join condition contain 'IS NOT DISTINCT FROM'
[ https://issues.apache.org/jira/browse/FLINK-22098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324413#comment-17324413 ] Andy commented on FLINK-22098: -- [~flink-jira-bot] [~jark] [~godfreyhe] I'm still working on the issue, please take a look at pull request when you have time, thanks a lot. * [|https://issues.apache.org/jira/secure/AddComment!default.jspa?id=13357018] > Fix bug for window join: plan is wrong if join condition contain 'IS NOT > DISTINCT FROM' > --- > > Key: FLINK-22098 > URL: https://issues.apache.org/jira/browse/FLINK-22098 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / Planner >Reporter: Andy >Assignee: Andy >Priority: Minor > Labels: pull-request-available > Fix For: 1.13.0 > > > Plan test is wrong for window join if join condition contain 'IS NOT DISTINCT > FROM'. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (FLINK-22098) Fix bug for window join: plan is wrong if join condition contain 'IS NOT DISTINCT FROM'
[ https://issues.apache.org/jira/browse/FLINK-22098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324413#comment-17324413 ] Andy edited comment on FLINK-22098 at 4/18/21, 5:19 AM: [~flink-jira-bot] [~jark] [~godfreyhe] I'm still working on the issue, please take a look at pull request when you have time, thanks a lot. was (Author: qingru zhang): [~flink-jira-bot] [~jark] [~godfreyhe] I'm still working on the issue, please take a look at pull request when you have time, thanks a lot. * [|https://issues.apache.org/jira/secure/AddComment!default.jspa?id=13357018] > Fix bug for window join: plan is wrong if join condition contain 'IS NOT > DISTINCT FROM' > --- > > Key: FLINK-22098 > URL: https://issues.apache.org/jira/browse/FLINK-22098 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / Planner >Reporter: Andy >Assignee: Andy >Priority: Minor > Labels: pull-request-available > Fix For: 1.13.0 > > > Plan test is wrong for window join if join condition contain 'IS NOT DISTINCT > FROM'. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] flinkbot edited a comment on pull request #15479: [FLINK-19606][table-runtime-blink] Implement streaming window join operator
flinkbot edited a comment on pull request #15479: URL: https://github.com/apache/flink/pull/15479#issuecomment-812344008 ## CI report: * 93b9b3d8dac2c3ecd13e3e3ede46f4a73263e34d Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16015) * 36663891630b1ca8dbe82c3225286ffab432bb18 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16720) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15477: [FLINK-22099][table-planner-blink] Fix bug for semi/anti window join.
flinkbot edited a comment on pull request #15477: URL: https://github.com/apache/flink/pull/15477#issuecomment-812337208 ## CI report: * 68bacb55c3c718eddba8cf5596bf94c3e4931d40 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16011) * fb7df56df7e4b5e1cf9f1eee726f9d338c0187e6 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16719) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15476: [FLINK-22098][table-planner-blink] Fix bug for window join: plan is wrong if join condition contain 'IS NOT DISTINCT FROM'
flinkbot edited a comment on pull request #15476: URL: https://github.com/apache/flink/pull/15476#issuecomment-812307606 ## CI report: * 3023dfa33790b3140a132fd2f76f56a2caad4a78 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16009) * f85197c7bb0c334252f289fd440e041ebb8ca26f Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16718) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15479: [FLINK-19606][table-runtime-blink] Implement streaming window join operator
flinkbot edited a comment on pull request #15479: URL: https://github.com/apache/flink/pull/15479#issuecomment-812344008 ## CI report: * 93b9b3d8dac2c3ecd13e3e3ede46f4a73263e34d Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16015) * 36663891630b1ca8dbe82c3225286ffab432bb18 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15477: [FLINK-22099][table-planner-blink] Fix bug for semi/anti window join.
flinkbot edited a comment on pull request #15477: URL: https://github.com/apache/flink/pull/15477#issuecomment-812337208 ## CI report: * 68bacb55c3c718eddba8cf5596bf94c3e4931d40 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16011) * fb7df56df7e4b5e1cf9f1eee726f9d338c0187e6 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15476: [FLINK-22098][table-planner-blink] Fix bug for window join: plan is wrong if join condition contain 'IS NOT DISTINCT FROM'
flinkbot edited a comment on pull request #15476: URL: https://github.com/apache/flink/pull/15476#issuecomment-812307606 ## CI report: * 3023dfa33790b3140a132fd2f76f56a2caad4a78 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16009) * f85197c7bb0c334252f289fd440e041ebb8ca26f UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] hehuiyuan removed a comment on pull request #15615: [FLINK-22021][table-planner-blink] Fix exception when PushFilterIntoLegacyTableSourceScanRule is faced with INTERVAL types
hehuiyuan removed a comment on pull request #15615: URL: https://github.com/apache/flink/pull/15615#issuecomment-821067136 @flinkbot re-run -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-21302) Fix NPE when use row_number() in over agg
[
https://issues.apache.org/jira/browse/FLINK-21302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324405#comment-17324405
]
Andy commented on FLINK-21302:
--
[~flink-jira-bot] [~jark] [~godfreyhe] I'm still working on the issue, please
take a look at pull request when you have time, thanks a lot.
> Fix NPE when use row_number() in over agg
> -
>
> Key: FLINK-21302
> URL: https://issues.apache.org/jira/browse/FLINK-21302
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Reporter: Andy
>Assignee: Andy
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.13.0
>
>
> `NullPointException` would be thrown out if SQL contains row_number() in Over
> Aggregate.
> {code:scala}
> @Test
> def testRowNumberOnOver(): Unit = {
> val t = failingDataSource(TestData.tupleData5)
> .toTable(tEnv, 'a, 'b, 'c, 'd, 'e, 'proctime.proctime)
> tEnv.registerTable("MyTable", t)
> val sqlQuery = "SELECT a, ROW_NUMBER() OVER (PARTITION BY a ORDER BY
> proctime()) FROM MyTable"
> val sink = new TestingAppendSink
> tEnv.sqlQuery(sqlQuery).toAppendStream[Row].addSink(sink)
> env.execute()
> }{code}
>
> The following exception would be thrown out.
> {code:java}
> java.lang.NullPointerExceptionjava.lang.NullPointerException at
> scala.collection.mutable.ArrayOps$ofInt$.length$extension(ArrayOps.scala:240)
> at scala.collection.mutable.ArrayOps$ofInt.length(ArrayOps.scala:240) at
> scala.collection.SeqLike$class.size(SeqLike.scala:106) at
> scala.collection.mutable.ArrayOps$ofInt.size(ArrayOps.scala:234) at
> scala.collection.mutable.Builder$class.sizeHint(Builder.scala:69) at
> scala.collection.mutable.ArrayBuilder.sizeHint(ArrayBuilder.scala:22) at
> scala.collection.TraversableLike$class.builder$1(TraversableLike.scala:230)
> at scala.collection.TraversableLike$class.map(TraversableLike.scala:233) at
> scala.collection.mutable.ArrayOps$ofInt.map(ArrayOps.scala:234) at
> org.apache.flink.table.planner.codegen.agg.DeclarativeAggCodeGen.(DeclarativeAggCodeGen.scala:82)
> at
> org.apache.flink.table.planner.codegen.agg.AggsHandlerCodeGenerator$$anonfun$3.apply(AggsHandlerCodeGenerator.scala:222)
> at
> org.apache.flink.table.planner.codegen.agg.AggsHandlerCodeGenerator$$anonfun$3.apply(AggsHandlerCodeGenerator.scala:214)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
> at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186) at
> scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at
> scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:186) at
> org.apache.flink.table.planner.codegen.agg.AggsHandlerCodeGenerator.initialAggregateInformation(AggsHandlerCodeGenerator.scala:214)
> at
> org.apache.flink.table.planner.codegen.agg.AggsHandlerCodeGenerator.generateAggsHandler(AggsHandlerCodeGenerator.scala:325)
> at
> org.apache.flink.table.planner.plan.nodes.exec.stream.StreamExecOverAggregate.createUnboundedOverProcessFunction(StreamExecOverAggregate.java:262)
> at
> org.apache.flink.table.planner.plan.nodes.exec.stream.StreamExecOverAggregate.translateToPlanInternal(StreamExecOverAggregate.java:154)
> at
> org.apache.flink.table.planner.plan.nodes.exec.ExecNodeBase.translateToPlan(ExecNodeBase.java:128)
> at
> org.apache.flink.table.planner.plan.nodes.exec.ExecEdge.translateToPlan(ExecEdge.java:247)
> at
> org.apache.flink.table.planner.plan.nodes.exec.common.CommonExecCalc.translateToPlanInternal(CommonExecCalc.java:65)
> at
> org.apache.flink.table.planner.plan.nodes.exec.ExecNodeBase.translateToPlan(ExecNodeBase.java:128)
> at
> org.apache.flink.table.planner.plan.nodes.exec.ExecEdge.translateToPlan(ExecEdge.java:247)
> at
> org.apache.flink.table.planner.plan.nodes.exec.common.CommonExecLegacySink.translateToTransformation(CommonExecLegacySink.java:167)
> at
> org.apache.flink.table.planner.plan.nodes.exec.common.CommonExecLegacySink.translateToPlanInternal(CommonExecLegacySink.java:136)
> at
> org.apache.flink.table.planner.plan.nodes.exec.ExecNodeBase.translateToPlan(ExecNodeBase.java:128)
> at
> org.apache.flink.table.planner.delegation.StreamPlanner$$anonfun$translateToPlan$1.apply(StreamPlanner.scala:67)
> at
> org.apache.flink.table.planner.delegation.StreamPlanner$$anonfun$translateToPlan$1.apply(StreamPlanner.scala:66)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
[jira] [Comment Edited] (FLINK-19606) Implement streaming window join operator
[ https://issues.apache.org/jira/browse/FLINK-19606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324403#comment-17324403 ] Andy edited comment on FLINK-19606 at 4/18/21, 4:13 AM: [~flink-jira-bot] [~jark] [~godfreyhe]. I'm still working on the issue, please take a look at pull request when you have time, thanks a lot. was (Author: qingru zhang): [~flink-jira-bot] [~jark] [~godfreyhe]. I'm still working on the issue, please take a look when you have time, thanks a lot. > Implement streaming window join operator > > > Key: FLINK-19606 > URL: https://issues.apache.org/jira/browse/FLINK-19606 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / Runtime >Reporter: Jark Wu >Assignee: Andy >Priority: Major > Labels: pull-request-available, stale-assigned > Fix For: 1.14.0 > > > Implement streaming window join operator in blink runtime. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-21302) Fix NPE when use row_number() in over agg
[
https://issues.apache.org/jira/browse/FLINK-21302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy updated FLINK-21302:
-
Labels: pull-request-available (was: pull-request-available stale-assigned)
> Fix NPE when use row_number() in over agg
> -
>
> Key: FLINK-21302
> URL: https://issues.apache.org/jira/browse/FLINK-21302
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Reporter: Andy
>Assignee: Andy
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.13.0
>
>
> `NullPointException` would be thrown out if SQL contains row_number() in Over
> Aggregate.
> {code:scala}
> @Test
> def testRowNumberOnOver(): Unit = {
> val t = failingDataSource(TestData.tupleData5)
> .toTable(tEnv, 'a, 'b, 'c, 'd, 'e, 'proctime.proctime)
> tEnv.registerTable("MyTable", t)
> val sqlQuery = "SELECT a, ROW_NUMBER() OVER (PARTITION BY a ORDER BY
> proctime()) FROM MyTable"
> val sink = new TestingAppendSink
> tEnv.sqlQuery(sqlQuery).toAppendStream[Row].addSink(sink)
> env.execute()
> }{code}
>
> The following exception would be thrown out.
> {code:java}
> java.lang.NullPointerExceptionjava.lang.NullPointerException at
> scala.collection.mutable.ArrayOps$ofInt$.length$extension(ArrayOps.scala:240)
> at scala.collection.mutable.ArrayOps$ofInt.length(ArrayOps.scala:240) at
> scala.collection.SeqLike$class.size(SeqLike.scala:106) at
> scala.collection.mutable.ArrayOps$ofInt.size(ArrayOps.scala:234) at
> scala.collection.mutable.Builder$class.sizeHint(Builder.scala:69) at
> scala.collection.mutable.ArrayBuilder.sizeHint(ArrayBuilder.scala:22) at
> scala.collection.TraversableLike$class.builder$1(TraversableLike.scala:230)
> at scala.collection.TraversableLike$class.map(TraversableLike.scala:233) at
> scala.collection.mutable.ArrayOps$ofInt.map(ArrayOps.scala:234) at
> org.apache.flink.table.planner.codegen.agg.DeclarativeAggCodeGen.(DeclarativeAggCodeGen.scala:82)
> at
> org.apache.flink.table.planner.codegen.agg.AggsHandlerCodeGenerator$$anonfun$3.apply(AggsHandlerCodeGenerator.scala:222)
> at
> org.apache.flink.table.planner.codegen.agg.AggsHandlerCodeGenerator$$anonfun$3.apply(AggsHandlerCodeGenerator.scala:214)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
> at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186) at
> scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at
> scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:186) at
> org.apache.flink.table.planner.codegen.agg.AggsHandlerCodeGenerator.initialAggregateInformation(AggsHandlerCodeGenerator.scala:214)
> at
> org.apache.flink.table.planner.codegen.agg.AggsHandlerCodeGenerator.generateAggsHandler(AggsHandlerCodeGenerator.scala:325)
> at
> org.apache.flink.table.planner.plan.nodes.exec.stream.StreamExecOverAggregate.createUnboundedOverProcessFunction(StreamExecOverAggregate.java:262)
> at
> org.apache.flink.table.planner.plan.nodes.exec.stream.StreamExecOverAggregate.translateToPlanInternal(StreamExecOverAggregate.java:154)
> at
> org.apache.flink.table.planner.plan.nodes.exec.ExecNodeBase.translateToPlan(ExecNodeBase.java:128)
> at
> org.apache.flink.table.planner.plan.nodes.exec.ExecEdge.translateToPlan(ExecEdge.java:247)
> at
> org.apache.flink.table.planner.plan.nodes.exec.common.CommonExecCalc.translateToPlanInternal(CommonExecCalc.java:65)
> at
> org.apache.flink.table.planner.plan.nodes.exec.ExecNodeBase.translateToPlan(ExecNodeBase.java:128)
> at
> org.apache.flink.table.planner.plan.nodes.exec.ExecEdge.translateToPlan(ExecEdge.java:247)
> at
> org.apache.flink.table.planner.plan.nodes.exec.common.CommonExecLegacySink.translateToTransformation(CommonExecLegacySink.java:167)
> at
> org.apache.flink.table.planner.plan.nodes.exec.common.CommonExecLegacySink.translateToPlanInternal(CommonExecLegacySink.java:136)
> at
> org.apache.flink.table.planner.plan.nodes.exec.ExecNodeBase.translateToPlan(ExecNodeBase.java:128)
> at
> org.apache.flink.table.planner.delegation.StreamPlanner$$anonfun$translateToPlan$1.apply(StreamPlanner.scala:67)
> at
> org.apache.flink.table.planner.delegation.StreamPlanner$$anonfun$translateToPlan$1.apply(StreamPlanner.scala:66)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at scala.collection.Iterator$class.foreach(Iterator.scala:891) at
> scala.collection.AbstractIterator.foreach(Iterator.scala:1334) at
>
[jira] [Updated] (FLINK-21923) SplitAggregateRule will be abnormal, when the sum/count and avg in SQL at the same time
[
https://issues.apache.org/jira/browse/FLINK-21923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
tartarus updated FLINK-21923:
-
Labels: pull-request-available (was: pull-request-available stale-assigned)
> SplitAggregateRule will be abnormal, when the sum/count and avg in SQL at the
> same time
> ---
>
> Key: FLINK-21923
> URL: https://issues.apache.org/jira/browse/FLINK-21923
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
>Affects Versions: 1.10.0
>Reporter: tartarus
>Assignee: tartarus
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.13.0
>
>
> SplitAggregateRule optimizes one-layer aggregation to two-layer aggregation
> to improve computing performance under data skew.
> In the partial phase, avg will be translated into count and sum. If count
> already exists in the original SQL at this time, the engine will remove the
> duplicate count, and then add Project to calculate and restore the optimized
> count result value.
> {code:java}
> relBuilder.aggregate(
> relBuilder.groupKey(fullGroupSet,
> ImmutableList.of[ImmutableBitSet](fullGroupSet)),
> newPartialAggCalls)
> relBuilder.peek().asInstanceOf[FlinkLogicalAggregate]
> .setPartialFinalType(PartialFinalType.PARTIAL)
> {code}
> so `relBuilder.peek()` will return `FlinkLogicalCalc` not
> `FlinkLogicalAggregate`,
> then will throw exception like
> {code:java}
> java.lang.ClassCastException:
> org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalCalc cannot be
> cast to
> org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalAggregate
> at
> org.apache.flink.table.planner.plan.rules.logical.SplitAggregateRule.onMatch(SplitAggregateRule.scala:286)
> at
> org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:333)
> at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:542)
> at
> org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:407)
> at
> org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:243)
> at
> org.apache.calcite.plan.hep.HepInstruction$RuleInstance.execute(HepInstruction.java:127)
> at
> org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:202)
> at
> org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:189)
> at
> org.apache.flink.table.planner.plan.optimize.program.FlinkHepProgram.optimize(FlinkHepProgram.scala:69)
> at
> org.apache.flink.table.planner.plan.optimize.program.FlinkHepRuleSetProgram.optimize(FlinkHepRuleSetProgram.scala:87)
> at
> org.apache.flink.table.planner.plan.optimize.program.FlinkChainedProgram$$anonfun$optimize$1.apply(FlinkChainedProgram.scala:62)
> at
> org.apache.flink.table.planner.plan.optimize.program.FlinkChainedProgram$$anonfun$optimize$1.apply(FlinkChainedProgram.scala:58)
> at
> scala.collection.TraversableOnce$$anonfun$foldLeft$1.apply(TraversableOnce.scala:157)
> at
> scala.collection.TraversableOnce$$anonfun$foldLeft$1.apply(TraversableOnce.scala:157)
> at scala.collection.Iterator$class.foreach(Iterator.scala:891)
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
> at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
> at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
> at
> scala.collection.TraversableOnce$class.foldLeft(TraversableOnce.scala:157)
> at scala.collection.AbstractTraversable.foldLeft(Traversable.scala:104)
> at
> org.apache.flink.table.planner.plan.optimize.program.FlinkChainedProgram.optimize(FlinkChainedProgram.scala:57)
> at
> org.apache.flink.table.planner.plan.optimize.StreamCommonSubGraphBasedOptimizer.optimizeTree(StreamCommonSubGraphBasedOptimizer.scala:163)
> at
> org.apache.flink.table.planner.plan.optimize.StreamCommonSubGraphBasedOptimizer.doOptimize(StreamCommonSubGraphBasedOptimizer.scala:79)
> at
> org.apache.flink.table.planner.plan.optimize.CommonSubGraphBasedOptimizer.optimize(CommonSubGraphBasedOptimizer.scala:77)
> at
> org.apache.flink.table.planner.delegation.PlannerBase.optimize(PlannerBase.scala:284)
> at
> org.apache.flink.table.planner.utils.TableTestUtilBase.assertPlanEquals(TableTestBase.scala:889)
> at
> org.apache.flink.table.planner.utils.TableTestUtilBase.doVerifyPlan(TableTestBase.scala:780)
> at
> org.apache.flink.table.planner.utils.TableTestUtilBase.verifyPlan(TableTestBase.scala:283)
> at
> org.apache.flink.table.planner.plan.rules.logical.SplitAggregateRuleTest.testAggWithFilterClause2(SplitAggregateRuleTest.scala:205)
> {code}
> We can
[jira] [Commented] (FLINK-19606) Implement streaming window join operator
[ https://issues.apache.org/jira/browse/FLINK-19606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324403#comment-17324403 ] Andy commented on FLINK-19606: -- [~flink-jira-bot] [~jark] [~godfreyhe]. I'm still working on the issue, please take a look when you have time, thanks a lot. > Implement streaming window join operator > > > Key: FLINK-19606 > URL: https://issues.apache.org/jira/browse/FLINK-19606 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / Runtime >Reporter: Jark Wu >Assignee: Andy >Priority: Major > Labels: pull-request-available, stale-assigned > Fix For: 1.14.0 > > > Implement streaming window join operator in blink runtime. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-22099) Fix bug for semi/anti window join.
[ https://issues.apache.org/jira/browse/FLINK-22099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy updated FLINK-22099: - Labels: pull-request-available (was: pull-request-available stale-assigned) > Fix bug for semi/anti window join. > -- > > Key: FLINK-22099 > URL: https://issues.apache.org/jira/browse/FLINK-22099 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / Planner >Reporter: Andy >Assignee: Andy >Priority: Minor > Labels: pull-request-available > Fix For: 1.13.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-22098) Fix bug for window join: plan is wrong if join condition contain 'IS NOT DISTINCT FROM'
[ https://issues.apache.org/jira/browse/FLINK-22098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy updated FLINK-22098: - Labels: pull-request-available (was: pull-request-available stale-assigned) > Fix bug for window join: plan is wrong if join condition contain 'IS NOT > DISTINCT FROM' > --- > > Key: FLINK-22098 > URL: https://issues.apache.org/jira/browse/FLINK-22098 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / Planner >Reporter: Andy >Assignee: Andy >Priority: Minor > Labels: pull-request-available > Fix For: 1.13.0 > > > Plan test is wrong for window join if join condition contain 'IS NOT DISTINCT > FROM'. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (FLINK-22310) The result of LogicalWindowJsonDeserializer is incorrect
[ https://issues.apache.org/jira/browse/FLINK-22310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] godfrey he closed FLINK-22310. -- Resolution: Fixed Fixed in 1.13.0: d32ca07743c729e66656f119f057249e03e5c5e2 > The result of LogicalWindowJsonDeserializer is incorrect > > > Key: FLINK-22310 > URL: https://issues.apache.org/jira/browse/FLINK-22310 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / Planner >Reporter: godfrey he >Assignee: godfrey he >Priority: Critical > Labels: pull-request-available > Fix For: 1.13.0 > > > The reason is the wrong argument is given in LogicalWindowJsonDeserializer > when creating FieldReferenceExpression instance, see line#137 in > LogicalWindowJsonDeserializer. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] godfreyhe closed pull request #15645: [FLINK-22310][table-planner-blink] Fix the incorrect deserialization result of LogicalWindowJsonDeserializer
godfreyhe closed pull request #15645: URL: https://github.com/apache/flink/pull/15645 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Closed] (FLINK-19614) Further optimization of sort-merge based blocking shuffle
[ https://issues.apache.org/jira/browse/FLINK-19614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yingjie Cao closed FLINK-19614. --- Resolution: Resolved > Further optimization of sort-merge based blocking shuffle > - > > Key: FLINK-19614 > URL: https://issues.apache.org/jira/browse/FLINK-19614 > Project: Flink > Issue Type: Improvement > Components: Runtime / Network >Reporter: Yingjie Cao >Assignee: Yingjie Cao >Priority: Major > Fix For: 1.13.0 > > > FLINK-19582 introduces a basic sort-merge based blocking shuffle > implementation. We can further optimize it based on the approaches proposed > in > [https://docs.google.com/document/d/1mpekX6aAHJhBsQ0pS9MxDiFQjHQIuaJH0GAQHh0GlJ0/edit?usp=sharing|https://docs.google.com/document/d/1mpekX6aAHJhBsQ0pS9MxDiFQjHQIuaJH0GAQHh0GlJ0/edit?usp=sharing,]. > This is the umbrella ticket for the optimizations. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (FLINK-22307) Increase the default value of data writing cache size (not configurable) for sort-merge blocking shuffle
[ https://issues.apache.org/jira/browse/FLINK-22307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yingjie Cao closed FLINK-22307. --- Resolution: Resolved > Increase the default value of data writing cache size (not configurable) for > sort-merge blocking shuffle > > > Key: FLINK-22307 > URL: https://issues.apache.org/jira/browse/FLINK-22307 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Network >Reporter: Yingjie Cao >Priority: Major > Labels: pull-request-available > Fix For: 1.13.0 > > > Currently, the data writing cache is 8M, which is not enough if data > compression is enabled. By increasing the cache size to 16M, the performance > of our benchmark job can be increased by about 20%. (We may make it > configurable in the future) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-22307) Increase the default value of data writing cache size (not configurable) for sort-merge blocking shuffle
[ https://issues.apache.org/jira/browse/FLINK-22307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324396#comment-17324396 ] Yingjie Cao commented on FLINK-22307: - Merged via d29357791315ace0218308f9f96bf65c7c8079d7 on master. > Increase the default value of data writing cache size (not configurable) for > sort-merge blocking shuffle > > > Key: FLINK-22307 > URL: https://issues.apache.org/jira/browse/FLINK-22307 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Network >Reporter: Yingjie Cao >Priority: Major > Labels: pull-request-available > Fix For: 1.13.0 > > > Currently, the data writing cache is 8M, which is not enough if data > compression is enabled. By increasing the cache size to 16M, the performance > of our benchmark job can be increased by about 20%. (We may make it > configurable in the future) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-22305) Improve log messages of sort-merge blocking shuffle
[ https://issues.apache.org/jira/browse/FLINK-22305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324394#comment-17324394 ] Yingjie Cao commented on FLINK-22305: - Merged via 77e0478a7879bd41f9f52872d84c467d158b8098 on master. > Improve log messages of sort-merge blocking shuffle > --- > > Key: FLINK-22305 > URL: https://issues.apache.org/jira/browse/FLINK-22305 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Network >Reporter: Yingjie Cao >Priority: Major > Labels: pull-request-available > Fix For: 1.13.0 > > > Currently, the default value of taskmanager.network.sort-shuffle.min-buffers > is 64, which is pretty small. As suggested, we'd like to increase the default > value of taskmanager.network.sort-shuffle.min-buffers. By increasing the > default taskmanager.network.sort-shuffle.min-buffers, the corner case of very > small in-memory sort-buffer and write-buffer can be avoid, which is better > for performance. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (FLINK-22305) Improve log messages of sort-merge blocking shuffle
[ https://issues.apache.org/jira/browse/FLINK-22305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yingjie Cao closed FLINK-22305. --- Resolution: Resolved > Improve log messages of sort-merge blocking shuffle > --- > > Key: FLINK-22305 > URL: https://issues.apache.org/jira/browse/FLINK-22305 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Network >Reporter: Yingjie Cao >Priority: Major > Labels: pull-request-available > Fix For: 1.13.0 > > > Currently, the default value of taskmanager.network.sort-shuffle.min-buffers > is 64, which is pretty small. As suggested, we'd like to increase the default > value of taskmanager.network.sort-shuffle.min-buffers. By increasing the > default taskmanager.network.sort-shuffle.min-buffers, the corner case of very > small in-memory sort-buffer and write-buffer can be avoid, which is better > for performance. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] guoweiM closed pull request #15651: [FLINK-22307][network] Increase the default value of data writing cache size (not configurable) for sort-merge blocking shuffle
guoweiM closed pull request #15651: URL: https://github.com/apache/flink/pull/15651 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15651: [FLINK-22307][network] Increase the default value of data writing cache size (not configurable) for sort-merge blocking shuffle
flinkbot edited a comment on pull request #15651: URL: https://github.com/apache/flink/pull/15651#issuecomment-821767454 ## CI report: * 3d65a719c16d9aaf8a44f843e2e21d1719947559 UNKNOWN * c840d852f79ed239c164c4809f36529bed7ffe5d Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16700) * b955af657e6ef1801b4228c7d07dd63dff9f61ae UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] guoweiM closed pull request #15652: [FLINK-22305][network] Improve log messages of sort-merge blocking shuffle
guoweiM closed pull request #15652: URL: https://github.com/apache/flink/pull/15652 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] guoweiM commented on a change in pull request #15652: [FLINK-22305][network] Improve log messages of sort-merge blocking shuffle
guoweiM commented on a change in pull request #15652:
URL: https://github.com/apache/flink/pull/15652#discussion_r615332364
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/SortMergeResultPartition.java
##
@@ -160,18 +160,12 @@ public void setup() throws IOException {
}
int numRequiredBuffer = bufferPool.getNumberOfRequiredMemorySegments();
-String errorMessage =
-String.format(
-"Too few sort buffers, please increase %s to a larger
value (more than %d).",
-
NettyShuffleEnvironmentOptions.NETWORK_SORT_SHUFFLE_MIN_BUFFERS,
-2 * expectedWriteBuffers);
-if (numRequiredBuffer < 2 * expectedWriteBuffers) {
-LOG.warn(errorMessage);
-}
-
int numWriteBuffers = Math.min(numRequiredBuffer / 2,
expectedWriteBuffers);
if (numWriteBuffers < 1) {
-throw new IOException(errorMessage);
+throw new IOException(
+String.format(
+"Too few sort buffers, please increase %s.",
+
NettyShuffleEnvironmentOptions.NETWORK_SORT_SHUFFLE_MIN_BUFFERS));
Review comment:
Thanks for explaining.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Comment Edited] (FLINK-17510) StreamingKafkaITCase. testKafka timeouts on downloading Kafka
[
https://issues.apache.org/jira/browse/FLINK-17510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324387#comment-17324387
]
Guowei Ma edited comment on FLINK-17510 at 4/18/21, 2:52 AM:
-
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16711=logs=c88eea3b-64a0-564d-0031-9fdcd7b8abee=2b7514ee-e706-5046-657b-3430666e7bd9=17137
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16711=logs=68a897ab-3047-5660-245a-cce8f83859f6=d47e27f5-9721-5d5f-1cf3-62adbf3d115d=17334
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16692=logs=68a897ab-3047-5660-245a-cce8f83859f6=d47e27f5-9721-5d5f-1cf3-62adbf3d115d=17405
was (Author: maguowei):
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16711=logs=c88eea3b-64a0-564d-0031-9fdcd7b8abee=2b7514ee-e706-5046-657b-3430666e7bd9=17137
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16711=logs=68a897ab-3047-5660-245a-cce8f83859f6=d47e27f5-9721-5d5f-1cf3-62adbf3d115d=17334
> StreamingKafkaITCase. testKafka timeouts on downloading Kafka
> -
>
> Key: FLINK-17510
> URL: https://issues.apache.org/jira/browse/FLINK-17510
> Project: Flink
> Issue Type: Bug
> Components: Build System / Azure Pipelines, Connectors / Kafka, Tests
>Affects Versions: 1.11.3, 1.12.1, 1.13.0
>Reporter: Robert Metzger
>Priority: Critical
> Labels: test-stability
>
> CI:
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=585=logs=c88eea3b-64a0-564d-0031-9fdcd7b8abee=1e2bbe5b-4657-50be-1f07-d84bfce5b1f5
> {code}
> 2020-05-05T00:06:49.7268716Z [INFO]
> ---
> 2020-05-05T00:06:49.7268938Z [INFO] T E S T S
> 2020-05-05T00:06:49.7269282Z [INFO]
> ---
> 2020-05-05T00:06:50.5336315Z [INFO] Running
> org.apache.flink.tests.util.kafka.StreamingKafkaITCase
> 2020-05-05T00:11:26.8603439Z [ERROR] Tests run: 3, Failures: 0, Errors: 2,
> Skipped: 0, Time elapsed: 276.323 s <<< FAILURE! - in
> org.apache.flink.tests.util.kafka.StreamingKafkaITCase
> 2020-05-05T00:11:26.8604882Z [ERROR] testKafka[1:
> kafka-version:0.11.0.2](org.apache.flink.tests.util.kafka.StreamingKafkaITCase)
> Time elapsed: 120.024 s <<< ERROR!
> 2020-05-05T00:11:26.8605942Z java.io.IOException: Process ([wget, -q, -P,
> /tmp/junit2815750531595874769/downloads/1290570732,
> https://archive.apache.org/dist/kafka/0.11.0.2/kafka_2.11-0.11.0.2.tgz])
> exceeded timeout (12) or number of retries (3).
> 2020-05-05T00:11:26.8606732Z at
> org.apache.flink.tests.util.AutoClosableProcess$AutoClosableProcessBuilder.runBlockingWithRetry(AutoClosableProcess.java:132)
> 2020-05-05T00:11:26.8607321Z at
> org.apache.flink.tests.util.cache.AbstractDownloadCache.getOrDownload(AbstractDownloadCache.java:127)
> 2020-05-05T00:11:26.8607826Z at
> org.apache.flink.tests.util.cache.LolCache.getOrDownload(LolCache.java:31)
> 2020-05-05T00:11:26.8608343Z at
> org.apache.flink.tests.util.kafka.LocalStandaloneKafkaResource.setupKafkaDist(LocalStandaloneKafkaResource.java:98)
> 2020-05-05T00:11:26.8608892Z at
> org.apache.flink.tests.util.kafka.LocalStandaloneKafkaResource.before(LocalStandaloneKafkaResource.java:92)
> 2020-05-05T00:11:26.8609602Z at
> org.apache.flink.util.ExternalResource$1.evaluate(ExternalResource.java:46)
> 2020-05-05T00:11:26.8610026Z at
> org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> 2020-05-05T00:11:26.8610553Z at
> org.junit.rules.RunRules.evaluate(RunRules.java:20)
> 2020-05-05T00:11:26.8610958Z at
> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> 2020-05-05T00:11:26.8611388Z at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> 2020-05-05T00:11:26.8612214Z at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> 2020-05-05T00:11:26.8612706Z at
> org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> 2020-05-05T00:11:26.8613109Z at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> 2020-05-05T00:11:26.8613551Z at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> 2020-05-05T00:11:26.8614019Z at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> 2020-05-05T00:11:26.8614442Z at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> 2020-05-05T00:11:26.8614869Z at
> org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> 2020-05-05T00:11:26.8615251Z at
> org.junit.runners.Suite.runChild(Suite.java:128)
> 2020-05-05T00:11:26.8615654Z at
> org.junit.runners.Suite.runChild(Suite.java:27)
> 2020-05-05T00:11:26.8616060Z at
>
[jira] [Updated] (FLINK-22333) Elasticsearch7DynamicSinkITCase.testWritingDocuments failed due to deploy task timeout.
[
https://issues.apache.org/jira/browse/FLINK-22333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guowei Ma updated FLINK-22333:
--
Summary: Elasticsearch7DynamicSinkITCase.testWritingDocuments failed due to
deploy task timeout. (was:
Elasticsearch7DynamicSinkITCase.testWritingDocuments fail due to deploy task
timeout.)
> Elasticsearch7DynamicSinkITCase.testWritingDocuments failed due to deploy
> task timeout.
> ---
>
> Key: FLINK-22333
> URL: https://issues.apache.org/jira/browse/FLINK-22333
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
>Affects Versions: 1.13.0
>Reporter: Guowei Ma
>Priority: Major
> Labels: test-stability
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16694=logs=d44f43ce-542c-597d-bf94-b0718c71e5e8=03dca39c-73e8-5aaf-601d-328ae5c35f20=12329
> {code:java}
> 2021-04-16T23:37:23.5719280Z Apr 16 23:37:23
> org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
> 2021-04-16T23:37:23.5739250Z Apr 16 23:37:23 at
> org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:144)
> 2021-04-16T23:37:23.5759329Z Apr 16 23:37:23 at
> org.apache.flink.runtime.minicluster.MiniClusterJobClient.lambda$getJobExecutionResult$3(MiniClusterJobClient.java:137)
> 2021-04-16T23:37:23.5779145Z Apr 16 23:37:23 at
> java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616)
> 2021-04-16T23:37:23.5799204Z Apr 16 23:37:23 at
> java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591)
> 2021-04-16T23:37:23.5819302Z Apr 16 23:37:23 at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> 2021-04-16T23:37:23.5839106Z Apr 16 23:37:23 at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
> 2021-04-16T23:37:23.5859276Z Apr 16 23:37:23 at
> org.apache.flink.runtime.rpc.akka.AkkaInvocationHandler.lambda$invokeRpc$0(AkkaInvocationHandler.java:237)
> 2021-04-16T23:37:23.5868964Z Apr 16 23:37:23 at
> java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
> 2021-04-16T23:37:23.5869925Z Apr 16 23:37:23 at
> java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
> 2021-04-16T23:37:23.5919839Z Apr 16 23:37:23 at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> 2021-04-16T23:37:23.5959562Z Apr 16 23:37:23 at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
> 2021-04-16T23:37:23.5989732Z Apr 16 23:37:23 at
> org.apache.flink.runtime.concurrent.FutureUtils$1.onComplete(FutureUtils.java:1081)
> 2021-04-16T23:37:23.6019422Z Apr 16 23:37:23 at
> akka.dispatch.OnComplete.internal(Future.scala:264)
> 2021-04-16T23:37:23.6039067Z Apr 16 23:37:23 at
> akka.dispatch.OnComplete.internal(Future.scala:261)
> 2021-04-16T23:37:23.6060126Z Apr 16 23:37:23 at
> akka.dispatch.japi$CallbackBridge.apply(Future.scala:191)
> 2021-04-16T23:37:23.6089258Z Apr 16 23:37:23 at
> akka.dispatch.japi$CallbackBridge.apply(Future.scala:188)
> 2021-04-16T23:37:23.6119150Z Apr 16 23:37:23 at
> scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
> 2021-04-16T23:37:23.6139149Z Apr 16 23:37:23 at
> org.apache.flink.runtime.concurrent.Executors$DirectExecutionContext.execute(Executors.java:73)
> 2021-04-16T23:37:23.6159077Z Apr 16 23:37:23 at
> scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)
> 2021-04-16T23:37:23.6189432Z Apr 16 23:37:23 at
> scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)
> 2021-04-16T23:37:23.6215243Z Apr 16 23:37:23 at
> akka.pattern.PromiseActorRef.$bang(AskSupport.scala:572)
> 2021-04-16T23:37:23.6219148Z Apr 16 23:37:23 at
> akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:22)
> 2021-04-16T23:37:23.6220221Z Apr 16 23:37:23 at
> akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:21)
> 2021-04-16T23:37:23.6249411Z Apr 16 23:37:23 at
> scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:436)
> 2021-04-16T23:37:23.6259145Z Apr 16 23:37:23 at
> scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:435)
> 2021-04-16T23:37:23.6289272Z Apr 16 23:37:23 at
> scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
> 2021-04-16T23:37:23.6309243Z Apr 16 23:37:23 at
> akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
> 2021-04-16T23:37:23.6359306Z Apr 16 23:37:23 at
> akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:91)
> 2021-04-16T23:37:23.6369399Z Apr 16 23:37:23 at
>
[jira] [Created] (FLINK-22333) Elasticsearch7DynamicSinkITCase.testWritingDocuments fail due to deploy task timeout.
Guowei Ma created FLINK-22333:
-
Summary: Elasticsearch7DynamicSinkITCase.testWritingDocuments fail
due to deploy task timeout.
Key: FLINK-22333
URL: https://issues.apache.org/jira/browse/FLINK-22333
Project: Flink
Issue Type: Bug
Components: Runtime / Coordination
Affects Versions: 1.13.0
Reporter: Guowei Ma
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16694=logs=d44f43ce-542c-597d-bf94-b0718c71e5e8=03dca39c-73e8-5aaf-601d-328ae5c35f20=12329
{code:java}
2021-04-16T23:37:23.5719280Z Apr 16 23:37:23
org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
2021-04-16T23:37:23.5739250Z Apr 16 23:37:23at
org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:144)
2021-04-16T23:37:23.5759329Z Apr 16 23:37:23at
org.apache.flink.runtime.minicluster.MiniClusterJobClient.lambda$getJobExecutionResult$3(MiniClusterJobClient.java:137)
2021-04-16T23:37:23.5779145Z Apr 16 23:37:23at
java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616)
2021-04-16T23:37:23.5799204Z Apr 16 23:37:23at
java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591)
2021-04-16T23:37:23.5819302Z Apr 16 23:37:23at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
2021-04-16T23:37:23.5839106Z Apr 16 23:37:23at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
2021-04-16T23:37:23.5859276Z Apr 16 23:37:23at
org.apache.flink.runtime.rpc.akka.AkkaInvocationHandler.lambda$invokeRpc$0(AkkaInvocationHandler.java:237)
2021-04-16T23:37:23.5868964Z Apr 16 23:37:23at
java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
2021-04-16T23:37:23.5869925Z Apr 16 23:37:23at
java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
2021-04-16T23:37:23.5919839Z Apr 16 23:37:23at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
2021-04-16T23:37:23.5959562Z Apr 16 23:37:23at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
2021-04-16T23:37:23.5989732Z Apr 16 23:37:23at
org.apache.flink.runtime.concurrent.FutureUtils$1.onComplete(FutureUtils.java:1081)
2021-04-16T23:37:23.6019422Z Apr 16 23:37:23at
akka.dispatch.OnComplete.internal(Future.scala:264)
2021-04-16T23:37:23.6039067Z Apr 16 23:37:23at
akka.dispatch.OnComplete.internal(Future.scala:261)
2021-04-16T23:37:23.6060126Z Apr 16 23:37:23at
akka.dispatch.japi$CallbackBridge.apply(Future.scala:191)
2021-04-16T23:37:23.6089258Z Apr 16 23:37:23at
akka.dispatch.japi$CallbackBridge.apply(Future.scala:188)
2021-04-16T23:37:23.6119150Z Apr 16 23:37:23at
scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
2021-04-16T23:37:23.6139149Z Apr 16 23:37:23at
org.apache.flink.runtime.concurrent.Executors$DirectExecutionContext.execute(Executors.java:73)
2021-04-16T23:37:23.6159077Z Apr 16 23:37:23at
scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:44)
2021-04-16T23:37:23.6189432Z Apr 16 23:37:23at
scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:252)
2021-04-16T23:37:23.6215243Z Apr 16 23:37:23at
akka.pattern.PromiseActorRef.$bang(AskSupport.scala:572)
2021-04-16T23:37:23.6219148Z Apr 16 23:37:23at
akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:22)
2021-04-16T23:37:23.6220221Z Apr 16 23:37:23at
akka.pattern.PipeToSupport$PipeableFuture$$anonfun$pipeTo$1.applyOrElse(PipeToSupport.scala:21)
2021-04-16T23:37:23.6249411Z Apr 16 23:37:23at
scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:436)
2021-04-16T23:37:23.6259145Z Apr 16 23:37:23at
scala.concurrent.Future$$anonfun$andThen$1.apply(Future.scala:435)
2021-04-16T23:37:23.6289272Z Apr 16 23:37:23at
scala.concurrent.impl.CallbackRunnable.run(Promise.scala:36)
2021-04-16T23:37:23.6309243Z Apr 16 23:37:23at
akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
2021-04-16T23:37:23.6359306Z Apr 16 23:37:23at
akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:91)
2021-04-16T23:37:23.6369399Z Apr 16 23:37:23at
akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)
2021-04-16T23:37:23.6389444Z Apr 16 23:37:23at
akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)
2021-04-16T23:37:23.6429180Z Apr 16 23:37:23at
scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)
2021-04-16T23:37:23.6449179Z Apr 16 23:37:23at
akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:90)
2021-04-16T23:37:23.6479350Z Apr 16 23:37:23at
[jira] [Updated] (FLINK-22332) ConnectedComponentsWithObjectMapITCase.testJobWithoutObjectReuse failed due to NPE when calling "notifyDataAvailable"
[ https://issues.apache.org/jira/browse/FLINK-22332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guowei Ma updated FLINK-22332: -- Summary: ConnectedComponentsWithObjectMapITCase.testJobWithoutObjectReuse failed due to NPE when calling "notifyDataAvailable" (was: ConnectedComponentsWithObjectMapITCase.testJobWithoutObjectReuse due to NPE when calling "notifyDataAvailable") > ConnectedComponentsWithObjectMapITCase.testJobWithoutObjectReuse failed due > to NPE when calling "notifyDataAvailable" > -- > > Key: FLINK-22332 > URL: https://issues.apache.org/jira/browse/FLINK-22332 > Project: Flink > Issue Type: Bug > Components: Runtime / Network >Affects Versions: 1.11.3 >Reporter: Guowei Ma >Priority: Major > Labels: test-stability > > https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16711=logs=219e462f-e75e-506c-3671-5017d866ccf6=94b2a454-a9e3-5226-421d-758b172639ef=4476 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-22266) Harden JobMasterStopWithSavepointITCase
[
https://issues.apache.org/jira/browse/FLINK-22266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324390#comment-17324390
]
Guowei Ma commented on FLINK-22266:
---
Just for reporting
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16710=logs=8fd9202e-fd17-5b26-353c-ac1ff76c8f28=a0a633b8-47ef-5c5a-2806-3c13b9e48228=4589
> Harden JobMasterStopWithSavepointITCase
> ---
>
> Key: FLINK-22266
> URL: https://issues.apache.org/jira/browse/FLINK-22266
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing, Runtime / Coordination
>Affects Versions: 1.13.0
>Reporter: Guowei Ma
>Assignee: Dawid Wysakowicz
>Priority: Critical
> Labels: test-stability
> Fix For: 1.13.0
>
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16451=logs=5c8e7682-d68f-54d1-16a2-a09310218a49=f508e270-48d6-5f1e-3138-42a17e0714f0=3884
> {code:java}
> [ERROR]
> throwingExceptionOnCallbackWithNoRestartsShouldFailTheTerminate(org.apache.flink.runtime.jobmaster.JobMasterStopWithSavepointITCase)
> Time elapsed: 0.154 s <<< FAILURE!
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.flink.runtime.jobmaster.JobMasterStopWithSavepointITCase.throwingExceptionOnCallbackWithoutRestartsHelper(JobMasterStopWithSavepointITCase.java:154)
> at
> org.apache.flink.runtime.jobmaster.JobMasterStopWithSavepointITCase.throwingExceptionOnCallbackWithNoRestartsShouldFailTheTerminate(JobMasterStopWithSavepointITCase.java:138)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-22111) ClientTest.testSimpleRequests fail
[
https://issues.apache.org/jira/browse/FLINK-22111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324388#comment-17324388
]
Guowei Ma commented on FLINK-22111:
---
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16710=logs=d44f43ce-542c-597d-bf94-b0718c71e5e8=03dca39c-73e8-5aaf-601d-328ae5c35f20=14747
> ClientTest.testSimpleRequests fail
> --
>
> Key: FLINK-22111
> URL: https://issues.apache.org/jira/browse/FLINK-22111
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Queryable State
>Affects Versions: 1.13.0
>Reporter: Guowei Ma
>Priority: Major
> Labels: test-stability
>
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16056=logs=3d12d40f-c62d-5ec4-6acc-0efe94cc3e89=5d6e4255-0ea8-5e2a-f52c-c881b7872361=15421
> {code:java}
> 21:47:16,289 [nioEventLoopGroup-4-3] WARN
> org.apache.flink.shaded.netty4.io.netty.channel.ChannelInitializer [] -
> Failed to initialize a channel. Closing: [id: 0x40eab0f6, L:/172.29.0.2:43846
> - R:/172.29.0.2:42436]
> org.apache.flink.shaded.netty4.io.netty.channel.ChannelPipelineException:
> org.apache.flink.queryablestate.network.ClientTest$1 is not a @Sharable
> handler, so can't be added or removed multiple times.
> at
> org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.checkMultiplicity(DefaultChannelPipeline.java:600)
> ~[flink-shaded-netty-4.1.49.Final-12.0.jar:?]
> at
> org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.addLast(DefaultChannelPipeline.java:202)
> ~[flink-shaded-netty-4.1.49.Final-12.0.jar:?]
> at
> org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.addLast(DefaultChannelPipeline.java:381)
> ~[flink-shaded-netty-4.1.49.Final-12.0.jar:?]
> at
> org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.addLast(DefaultChannelPipeline.java:370)
> ~[flink-shaded-netty-4.1.49.Final-12.0.jar:?]
> at
> org.apache.flink.queryablestate.network.ClientTest$5.initChannel(ClientTest.java:897)
> ~[test-classes/:?]
> at
> org.apache.flink.queryablestate.network.ClientTest$5.initChannel(ClientTest.java:890)
> ~[test-classes/:?]
> at
> org.apache.flink.shaded.netty4.io.netty.channel.ChannelInitializer.initChannel(ChannelInitializer.java:129)
> [flink-shaded-netty-4.1.49.Final-12.0.jar:?]
> at
> org.apache.flink.shaded.netty4.io.netty.channel.ChannelInitializer.handlerAdded(ChannelInitializer.java:112)
> [flink-shaded-netty-4.1.49.Final-12.0.jar:?]
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.callHandlerAdded(AbstractChannelHandlerContext.java:938)
> [flink-shaded-netty-4.1.49.Final-12.0.jar:?]
> at
> org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.callHandlerAdded0(DefaultChannelPipeline.java:609)
> [flink-shaded-netty-4.1.49.Final-12.0.jar:?]
> at
> org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.access$100(DefaultChannelPipeline.java:46)
> [flink-shaded-netty-4.1.49.Final-12.0.jar:?]
> at
> org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline$PendingHandlerAddedTask.execute(DefaultChannelPipeline.java:1463)
> [flink-shaded-netty-4.1.49.Final-12.0.jar:?]
> at
> org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.callHandlerAddedForAllHandlers(DefaultChannelPipeline.java:1115)
> [flink-shaded-netty-4.1.49.Final-12.0.jar:?]
> at
> org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.invokeHandlerAddedIfNeeded(DefaultChannelPipeline.java:650)
> [flink-shaded-netty-4.1.49.Final-12.0.jar:?]
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannel$AbstractUnsafe.register0(AbstractChannel.java:502)
> [flink-shaded-netty-4.1.49.Final-12.0.jar:?]
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannel$AbstractUnsafe.access$200(AbstractChannel.java:417)
> [flink-shaded-netty-4.1.49.Final-12.0.jar:?]
> at
> org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannel$AbstractUnsafe$1.run(AbstractChannel.java:474)
> [flink-shaded-netty-4.1.49.Final-12.0.jar:?]
> at
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
> [flink-shaded-netty-4.1.49.Final-12.0.jar:?]
> at
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
> [flink-shaded-netty-4.1.49.Final-12.0.jar:?]
> at
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
> [flink-shaded-netty-4.1.49.Final-12.0.jar:?]
> at
>
[jira] [Created] (FLINK-22332) ConnectedComponentsWithObjectMapITCase.testJobWithoutObjectReuse due to NPE when calling "notifyDataAvailable"
Guowei Ma created FLINK-22332: - Summary: ConnectedComponentsWithObjectMapITCase.testJobWithoutObjectReuse due to NPE when calling "notifyDataAvailable" Key: FLINK-22332 URL: https://issues.apache.org/jira/browse/FLINK-22332 Project: Flink Issue Type: Bug Components: Runtime / Network Affects Versions: 1.11.3 Reporter: Guowei Ma https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16711=logs=219e462f-e75e-506c-3671-5017d866ccf6=94b2a454-a9e3-5226-421d-758b172639ef=4476 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-22307) Increase the default value of data writing cache size (not configurable) for sort-merge blocking shuffle
[ https://issues.apache.org/jira/browse/FLINK-22307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yingjie Cao updated FLINK-22307: Summary: Increase the default value of data writing cache size (not configurable) for sort-merge blocking shuffle (was: Increase the default value of data writing cache size for sort-merge blocking shuffle) > Increase the default value of data writing cache size (not configurable) for > sort-merge blocking shuffle > > > Key: FLINK-22307 > URL: https://issues.apache.org/jira/browse/FLINK-22307 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Network >Reporter: Yingjie Cao >Priority: Major > Labels: pull-request-available > Fix For: 1.13.0 > > > Currently, the data writing cache is 8M, which is not enough if data > compression is enabled. By increasing the cache size to 16M, the performance > of our benchmark job can be increased by about 20%. (We may make it > configurable in the future) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-22307) Increase the default value of data writing cache size for sort-merge blocking shuffle
[ https://issues.apache.org/jira/browse/FLINK-22307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yingjie Cao updated FLINK-22307: Summary: Increase the default value of data writing cache size for sort-merge blocking shuffle (was: Increase the data writing cache size of sort-merge blocking shuffle) > Increase the default value of data writing cache size for sort-merge blocking > shuffle > - > > Key: FLINK-22307 > URL: https://issues.apache.org/jira/browse/FLINK-22307 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Network >Reporter: Yingjie Cao >Priority: Major > Labels: pull-request-available > Fix For: 1.13.0 > > > Currently, the data writing cache is 8M, which is not enough if data > compression is enabled. By increasing the cache size to 16M, the performance > of our benchmark job can be increased by about 20%. (We may make it > configurable in the future) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (FLINK-17510) StreamingKafkaITCase. testKafka timeouts on downloading Kafka
[
https://issues.apache.org/jira/browse/FLINK-17510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324387#comment-17324387
]
Guowei Ma edited comment on FLINK-17510 at 4/18/21, 2:22 AM:
-
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16711=logs=c88eea3b-64a0-564d-0031-9fdcd7b8abee=2b7514ee-e706-5046-657b-3430666e7bd9=17137
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16711=logs=68a897ab-3047-5660-245a-cce8f83859f6=d47e27f5-9721-5d5f-1cf3-62adbf3d115d=17334
was (Author: maguowei):
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16711=logs=c88eea3b-64a0-564d-0031-9fdcd7b8abee=2b7514ee-e706-5046-657b-3430666e7bd9=17137
> StreamingKafkaITCase. testKafka timeouts on downloading Kafka
> -
>
> Key: FLINK-17510
> URL: https://issues.apache.org/jira/browse/FLINK-17510
> Project: Flink
> Issue Type: Bug
> Components: Build System / Azure Pipelines, Connectors / Kafka, Tests
>Affects Versions: 1.11.3, 1.12.1, 1.13.0
>Reporter: Robert Metzger
>Priority: Critical
> Labels: test-stability
>
> CI:
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=585=logs=c88eea3b-64a0-564d-0031-9fdcd7b8abee=1e2bbe5b-4657-50be-1f07-d84bfce5b1f5
> {code}
> 2020-05-05T00:06:49.7268716Z [INFO]
> ---
> 2020-05-05T00:06:49.7268938Z [INFO] T E S T S
> 2020-05-05T00:06:49.7269282Z [INFO]
> ---
> 2020-05-05T00:06:50.5336315Z [INFO] Running
> org.apache.flink.tests.util.kafka.StreamingKafkaITCase
> 2020-05-05T00:11:26.8603439Z [ERROR] Tests run: 3, Failures: 0, Errors: 2,
> Skipped: 0, Time elapsed: 276.323 s <<< FAILURE! - in
> org.apache.flink.tests.util.kafka.StreamingKafkaITCase
> 2020-05-05T00:11:26.8604882Z [ERROR] testKafka[1:
> kafka-version:0.11.0.2](org.apache.flink.tests.util.kafka.StreamingKafkaITCase)
> Time elapsed: 120.024 s <<< ERROR!
> 2020-05-05T00:11:26.8605942Z java.io.IOException: Process ([wget, -q, -P,
> /tmp/junit2815750531595874769/downloads/1290570732,
> https://archive.apache.org/dist/kafka/0.11.0.2/kafka_2.11-0.11.0.2.tgz])
> exceeded timeout (12) or number of retries (3).
> 2020-05-05T00:11:26.8606732Z at
> org.apache.flink.tests.util.AutoClosableProcess$AutoClosableProcessBuilder.runBlockingWithRetry(AutoClosableProcess.java:132)
> 2020-05-05T00:11:26.8607321Z at
> org.apache.flink.tests.util.cache.AbstractDownloadCache.getOrDownload(AbstractDownloadCache.java:127)
> 2020-05-05T00:11:26.8607826Z at
> org.apache.flink.tests.util.cache.LolCache.getOrDownload(LolCache.java:31)
> 2020-05-05T00:11:26.8608343Z at
> org.apache.flink.tests.util.kafka.LocalStandaloneKafkaResource.setupKafkaDist(LocalStandaloneKafkaResource.java:98)
> 2020-05-05T00:11:26.8608892Z at
> org.apache.flink.tests.util.kafka.LocalStandaloneKafkaResource.before(LocalStandaloneKafkaResource.java:92)
> 2020-05-05T00:11:26.8609602Z at
> org.apache.flink.util.ExternalResource$1.evaluate(ExternalResource.java:46)
> 2020-05-05T00:11:26.8610026Z at
> org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> 2020-05-05T00:11:26.8610553Z at
> org.junit.rules.RunRules.evaluate(RunRules.java:20)
> 2020-05-05T00:11:26.8610958Z at
> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> 2020-05-05T00:11:26.8611388Z at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> 2020-05-05T00:11:26.8612214Z at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> 2020-05-05T00:11:26.8612706Z at
> org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> 2020-05-05T00:11:26.8613109Z at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> 2020-05-05T00:11:26.8613551Z at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> 2020-05-05T00:11:26.8614019Z at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> 2020-05-05T00:11:26.8614442Z at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> 2020-05-05T00:11:26.8614869Z at
> org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> 2020-05-05T00:11:26.8615251Z at
> org.junit.runners.Suite.runChild(Suite.java:128)
> 2020-05-05T00:11:26.8615654Z at
> org.junit.runners.Suite.runChild(Suite.java:27)
> 2020-05-05T00:11:26.8616060Z at
> org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> 2020-05-05T00:11:26.8616465Z at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> 2020-05-05T00:11:26.8616893Z at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> 2020-05-05T00:11:26.8617893Z at
>
[jira] [Commented] (FLINK-17510) StreamingKafkaITCase. testKafka timeouts on downloading Kafka
[
https://issues.apache.org/jira/browse/FLINK-17510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324387#comment-17324387
]
Guowei Ma commented on FLINK-17510:
---
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=16711=logs=c88eea3b-64a0-564d-0031-9fdcd7b8abee=2b7514ee-e706-5046-657b-3430666e7bd9=17137
> StreamingKafkaITCase. testKafka timeouts on downloading Kafka
> -
>
> Key: FLINK-17510
> URL: https://issues.apache.org/jira/browse/FLINK-17510
> Project: Flink
> Issue Type: Bug
> Components: Build System / Azure Pipelines, Connectors / Kafka, Tests
>Affects Versions: 1.11.3, 1.12.1, 1.13.0
>Reporter: Robert Metzger
>Priority: Critical
> Labels: test-stability
>
> CI:
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=585=logs=c88eea3b-64a0-564d-0031-9fdcd7b8abee=1e2bbe5b-4657-50be-1f07-d84bfce5b1f5
> {code}
> 2020-05-05T00:06:49.7268716Z [INFO]
> ---
> 2020-05-05T00:06:49.7268938Z [INFO] T E S T S
> 2020-05-05T00:06:49.7269282Z [INFO]
> ---
> 2020-05-05T00:06:50.5336315Z [INFO] Running
> org.apache.flink.tests.util.kafka.StreamingKafkaITCase
> 2020-05-05T00:11:26.8603439Z [ERROR] Tests run: 3, Failures: 0, Errors: 2,
> Skipped: 0, Time elapsed: 276.323 s <<< FAILURE! - in
> org.apache.flink.tests.util.kafka.StreamingKafkaITCase
> 2020-05-05T00:11:26.8604882Z [ERROR] testKafka[1:
> kafka-version:0.11.0.2](org.apache.flink.tests.util.kafka.StreamingKafkaITCase)
> Time elapsed: 120.024 s <<< ERROR!
> 2020-05-05T00:11:26.8605942Z java.io.IOException: Process ([wget, -q, -P,
> /tmp/junit2815750531595874769/downloads/1290570732,
> https://archive.apache.org/dist/kafka/0.11.0.2/kafka_2.11-0.11.0.2.tgz])
> exceeded timeout (12) or number of retries (3).
> 2020-05-05T00:11:26.8606732Z at
> org.apache.flink.tests.util.AutoClosableProcess$AutoClosableProcessBuilder.runBlockingWithRetry(AutoClosableProcess.java:132)
> 2020-05-05T00:11:26.8607321Z at
> org.apache.flink.tests.util.cache.AbstractDownloadCache.getOrDownload(AbstractDownloadCache.java:127)
> 2020-05-05T00:11:26.8607826Z at
> org.apache.flink.tests.util.cache.LolCache.getOrDownload(LolCache.java:31)
> 2020-05-05T00:11:26.8608343Z at
> org.apache.flink.tests.util.kafka.LocalStandaloneKafkaResource.setupKafkaDist(LocalStandaloneKafkaResource.java:98)
> 2020-05-05T00:11:26.8608892Z at
> org.apache.flink.tests.util.kafka.LocalStandaloneKafkaResource.before(LocalStandaloneKafkaResource.java:92)
> 2020-05-05T00:11:26.8609602Z at
> org.apache.flink.util.ExternalResource$1.evaluate(ExternalResource.java:46)
> 2020-05-05T00:11:26.8610026Z at
> org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> 2020-05-05T00:11:26.8610553Z at
> org.junit.rules.RunRules.evaluate(RunRules.java:20)
> 2020-05-05T00:11:26.8610958Z at
> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> 2020-05-05T00:11:26.8611388Z at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> 2020-05-05T00:11:26.8612214Z at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> 2020-05-05T00:11:26.8612706Z at
> org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> 2020-05-05T00:11:26.8613109Z at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> 2020-05-05T00:11:26.8613551Z at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> 2020-05-05T00:11:26.8614019Z at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> 2020-05-05T00:11:26.8614442Z at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> 2020-05-05T00:11:26.8614869Z at
> org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> 2020-05-05T00:11:26.8615251Z at
> org.junit.runners.Suite.runChild(Suite.java:128)
> 2020-05-05T00:11:26.8615654Z at
> org.junit.runners.Suite.runChild(Suite.java:27)
> 2020-05-05T00:11:26.8616060Z at
> org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> 2020-05-05T00:11:26.8616465Z at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> 2020-05-05T00:11:26.8616893Z at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> 2020-05-05T00:11:26.8617893Z at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> 2020-05-05T00:11:26.8618490Z at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> 2020-05-05T00:11:26.8619056Z at
> org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> 2020-05-05T00:11:26.8619589Z at
> org.junit.runners.Suite.runChild(Suite.java:128)
> 2020-05-05T00:11:26.8620073Z at
> org.junit.runners.Suite.runChild(Suite.java:27)
>
[GitHub] [flink] wsry commented on a change in pull request #15652: [FLINK-22305][network] Improve log messages of sort-merge blocking shuffle
wsry commented on a change in pull request #15652:
URL: https://github.com/apache/flink/pull/15652#discussion_r615329427
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/SortMergeResultPartition.java
##
@@ -160,18 +160,12 @@ public void setup() throws IOException {
}
int numRequiredBuffer = bufferPool.getNumberOfRequiredMemorySegments();
-String errorMessage =
-String.format(
-"Too few sort buffers, please increase %s to a larger
value (more than %d).",
-
NettyShuffleEnvironmentOptions.NETWORK_SORT_SHUFFLE_MIN_BUFFERS,
-2 * expectedWriteBuffers);
-if (numRequiredBuffer < 2 * expectedWriteBuffers) {
-LOG.warn(errorMessage);
-}
-
int numWriteBuffers = Math.min(numRequiredBuffer / 2,
expectedWriteBuffers);
if (numWriteBuffers < 1) {
-throw new IOException(errorMessage);
+throw new IOException(
+String.format(
+"Too few sort buffers, please increase %s.",
+
NettyShuffleEnvironmentOptions.NETWORK_SORT_SHUFFLE_MIN_BUFFERS));
Review comment:
The error only occurs when only 1 buffer is configured which is rare and
increasing it to any value would be OK. Besides, we already give a suggested
value in the config option description which is good enough and should never
trigger any warnings or exceptions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] guoweiM commented on a change in pull request #15652: [FLINK-22305][network] Improve log messages of sort-merge blocking shuffle
guoweiM commented on a change in pull request #15652:
URL: https://github.com/apache/flink/pull/15652#discussion_r615328699
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/SortMergeResultPartition.java
##
@@ -160,18 +160,12 @@ public void setup() throws IOException {
}
int numRequiredBuffer = bufferPool.getNumberOfRequiredMemorySegments();
-String errorMessage =
-String.format(
-"Too few sort buffers, please increase %s to a larger
value (more than %d).",
-
NettyShuffleEnvironmentOptions.NETWORK_SORT_SHUFFLE_MIN_BUFFERS,
-2 * expectedWriteBuffers);
-if (numRequiredBuffer < 2 * expectedWriteBuffers) {
-LOG.warn(errorMessage);
-}
-
int numWriteBuffers = Math.min(numRequiredBuffer / 2,
expectedWriteBuffers);
if (numWriteBuffers < 1) {
-throw new IOException(errorMessage);
+throw new IOException(
+String.format(
+"Too few sort buffers, please increase %s.",
+
NettyShuffleEnvironmentOptions.NETWORK_SORT_SHUFFLE_MIN_BUFFERS));
Review comment:
I think this it might not be clearer than the before. The old message
tells user increase to what size of the network buffer but the new message does
not tell the size of network buffer that user should increase to.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] link3280 commented on a change in pull request #13377: [FLINK-18592] bugfix for StreamingFileSink
link3280 commented on a change in pull request #13377:
URL: https://github.com/apache/flink/pull/13377#discussion_r615326763
##
File path:
flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/fs/hdfs/HadoopRecoverableFsDataOutputStream.java
##
@@ -360,4 +366,121 @@ private static boolean waitUntilLeaseIsRevoked(final
FileSystem fs, final Path p
}
return isClosed;
}
+
+
+ /*
+* Run the dfs recover lease. recoverLease is asynchronous. It returns:
-false when it starts the lease recovery (i.e. lease recovery not *yet* done) -
true when the lease recovery has
+* succeeded or the file is closed.
+*
+* But, we have to be careful. Each time we call recoverLease, it
starts the recover lease process over from the beginning. We could put
ourselves in a situation
+* where we are doing nothing but starting a recovery, interrupting it
to start again, and so on.
+*
+* The namenode will try to recover the lease on the file's primary
node. If all is well, it should return near immediately.
+* But, as is common, it is the very primary node that has crashed and
so the namenode will be stuck waiting on a socket timeout before it will ask
another datanode to start the recovery.
+* It does not help if we call recoverLease in the meantime and in
particular, subsequent to the socket timeout, a recoverLease invocation will
cause us to start over from square one
+* (possibly waiting on socket timeout against primary node).
+* So, in the below, we do the following:
+* 1. Call recoverLease.
+* 2. If it returns true, break.
+* 3. If it returns false, wait a few seconds and then call it again.
+* 4. If it returns true, break.
+* 5. If it returns false, wait for what we think the datanode socket
timeout is (configurable) and then try again.
+* 6. If it returns true, break.
+* 7. If it returns false, repeat starting at step 5. above. If
HDFS-4525 is available, call it every second and we might be able to exit early.
+*/
+ private static boolean recoverLease(Path path, DistributedFileSystem
dfs) throws IOException {
Review comment:
Looks like this new method is unused. Shouldn't we replace
#waitUntilLeaseIsRevoked with it?
##
File path:
flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/fs/hdfs/HadoopRecoverableFsDataOutputStream.java
##
@@ -357,6 +364,126 @@ private static boolean waitUntilLeaseIsRevoked(final
FileSystem fs, final Path p
}
isClosed = dfs.isFileClosed(path);
}
+ // [FLINK-18592] recover lease after the lease timeout passed
but file was still not closed
+ if(!isClosed && !deadline.hasTimeLeft()){
+ recoverLease(path, dfs);
+ }
return isClosed;
}
+
+
+ /*
+* Run the dfs recover lease. recoverLease is asynchronous. It returns:
-false when it starts the lease recovery (i.e. lease recovery not *yet* done) -
true when the lease recovery has
+* succeeded or the file is closed.
+*
+* But, we have to be careful. Each time we call recoverLease, it
starts the recover lease process over from the beginning. We could put
ourselves in a situation
+* where we are doing nothing but starting a recovery, interrupting it
to start again, and so on.
+*
+* The namenode will try to recover the lease on the file's primary
node. If all is well, it should return near immediately.
+* But, as is common, it is the very primary node that has crashed and
so the namenode will be stuck waiting on a socket timeout before it will ask
another datanode to start the recovery.
+* It does not help if we call recoverLease in the meantime and in
particular, subsequent to the socket timeout, a recoverLease invocation will
cause us to start over from square one
+* (possibly waiting on socket timeout against primary node).
+* So, in the below, we do the following:
+* 1. Call recoverLease.
+* 2. If it returns true, break.
+* 3. If it returns false, wait a few seconds and then call it again.
+* 4. If it returns true, break.
+* 5. If it returns false, wait for what we think the datanode socket
timeout is (configurable) and then try again.
+* 6. If it returns true, break.
+* 7. If it returns false, repeat starting at step 5. above. If
HDFS-4525 is available, call it every second and we might be able to exit early.
+*/
+ private static boolean recoverLease(Path path, DistributedFileSystem
dfs) throws IOException {
+ LOG.info("Recover lease on dfs file " + path);
+ long startWaiting = System.currentTimeMillis();
+
[GitHub] [flink] flinkbot edited a comment on pull request #15654: [FLINK-22141] Bumps checkstyle, junit, log4j version for security
flinkbot edited a comment on pull request #15654: URL: https://github.com/apache/flink/pull/15654#issuecomment-821836292 ## CI report: Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15654: [FLINK-22141] Bumps checkstyle, junit, log4j version for security
flinkbot edited a comment on pull request #15654: URL: https://github.com/apache/flink/pull/15654#issuecomment-821836292 ## CI report: * Unknown: [CANCELED](TBD) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] brandonJY commented on pull request #15654: [FLINK-22141] Bumps checkstyle, junit, log4j version for security
brandonJY commented on pull request #15654: URL: https://github.com/apache/flink/pull/15654#issuecomment-821911681 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] brandonJY commented on pull request #15654: [FLINK-22141] Bumps checkstyle, junit, log4j version for security
brandonJY commented on pull request #15654: URL: https://github.com/apache/flink/pull/15654#issuecomment-821911588 @flinkbot run travis -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15574: [hotfix][docs] Removed reference to fold in window function
flinkbot edited a comment on pull request #15574: URL: https://github.com/apache/flink/pull/15574#issuecomment-817750003 ## CI report: * fba7c0da981277648ff3629c1f4de48d1ccdae45 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16713) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15574: [hotfix][docs] Removed reference to fold in window function
flinkbot edited a comment on pull request #15574: URL: https://github.com/apache/flink/pull/15574#issuecomment-817750003 ## CI report: * 6e8a34bd5b59ebf1afa8149edc9f3ca232c35bf2 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16388) * fba7c0da981277648ff3629c1f4de48d1ccdae45 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16713) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15574: [hotfix][docs] Removed reference to fold in window function
flinkbot edited a comment on pull request #15574: URL: https://github.com/apache/flink/pull/15574#issuecomment-817750003 ## CI report: * 6e8a34bd5b59ebf1afa8149edc9f3ca232c35bf2 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16388) * fba7c0da981277648ff3629c1f4de48d1ccdae45 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15121: The method $(String) is undefined for the type TableExample
flinkbot edited a comment on pull request #15121: URL: https://github.com/apache/flink/pull/15121#issuecomment-793480240 ## CI report: * cf26ce895a7956d258acc073817f578558e78227 UNKNOWN * 709c4110370b845f42d916a06e56c6026cf2fac8 UNKNOWN * 4ea99332e1997eebca1f3f0a9d9229b8265fe32c UNKNOWN * d39b3fb096a3c883d36d3dcc85ea93efda8edd3d Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16708) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-7883) Make savepoints atomic with respect to state and side effects
[
https://issues.apache.org/jira/browse/FLINK-7883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324335#comment-17324335
]
Olivier NOUGUIER commented on FLINK-7883:
-
Any update ?
> Make savepoints atomic with respect to state and side effects
> -
>
> Key: FLINK-7883
> URL: https://issues.apache.org/jira/browse/FLINK-7883
> Project: Flink
> Issue Type: Improvement
> Components: API / DataStream, Connectors / Kafka, Runtime / State
> Backends
>Affects Versions: 1.3.2, 1.4.0
>Reporter: Antoine Philippot
>Priority: Major
>
> For a cancel with savepoint command, the JobManager trigger the cancel call
> once the savepoint is finished, but during the savepoint execution, kafka
> source continue to poll new messages which will not be part of the savepoint
> and will be replayed on the next application start.
> A solution could be to stop fetching the source stream task before triggering
> the savepoint.
> I suggest to add an interface {{StoppableFetchingSourceFunction}} with a
> method {{stopFetching}} that existant SourceFunction implementations could
> implement.
> We can add a {{stopFetchingSource}} property in
> {{CheckpointOptions}} class to pass the desired behaviour from
> {{JobManager.handleMessage(CancelJobWithSavepoint)}} to
> {{SourceStreamTask.triggerCheckpoint}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-22331) CLI Frontend (RestClient) doesn't work on Apple M1
Robert Metzger created FLINK-22331:
--
Summary: CLI Frontend (RestClient) doesn't work on Apple M1
Key: FLINK-22331
URL: https://issues.apache.org/jira/browse/FLINK-22331
Project: Flink
Issue Type: Bug
Components: Client / Job Submission
Affects Versions: 1.12.2, 1.13.0
Reporter: Robert Metzger
Attachments: flink-muthmann-client-KlemensMac.local (1).log_native,
flink-muthmann-client-KlemensMac.local.log_rosetta
This issue was first reported by a user:
https://lists.apache.org/thread.html/r50bda40a69688de52c9d6e3489ac2641491387c20fdc1cecedceee76%40%3Cuser.flink.apache.org%3E
See attached logs.
Exception without rosetta:
{code}
org.apache.flink.client.program.ProgramInvocationException: The main method
caused an error: Failed to execute job 'Streaming WordCount'.
at
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:366)
at
org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:219)
at
org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114)
at
org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:812)
at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:246)
at
org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1054)
at
org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1132)
at
org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:28)
at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1132)
Caused by: org.apache.flink.util.FlinkException: Failed to execute job
'Streaming WordCount'.
at
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.executeAsync(StreamExecutionEnvironment.java:1918)
at
org.apache.flink.client.program.StreamContextEnvironment.executeAsync(StreamContextEnvironment.java:135)
at
org.apache.flink.client.program.StreamContextEnvironment.execute(StreamContextEnvironment.java:76)
at
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1782)
at
org.apache.flink.streaming.examples.wordcount.WordCount.main(WordCount.java:97)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:349)
... 8 more
Caused by: org.apache.flink.runtime.client.JobSubmissionException: Failed to
submit JobGraph.
at
org.apache.flink.client.program.rest.RestClusterClient.lambda$submitJob$7(RestClusterClient.java:400)
at
java.base/java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:986)
at
java.base/java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:970)
at
java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
at
java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088)
at
org.apache.flink.runtime.concurrent.FutureUtils.lambda$retryOperationWithDelay$9(FutureUtils.java:390)
at
java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
at
java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)
at
java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
at
java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088)
at
org.apache.flink.runtime.rest.RestClient$ClientHandler.exceptionCaught(RestClient.java:613)
at
org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302)
at
org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281)
at
org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273)
at
org.apache.flink.shaded.netty4.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireExceptionCaught(CombinedChannelDuplexHandler.java:424)
at
[GitHub] [flink] flinkbot edited a comment on pull request #15534: [FLINK-22104][sql-client] Fix unstable SqlClientTest.testExecuteSqlFile
flinkbot edited a comment on pull request #15534: URL: https://github.com/apache/flink/pull/15534#issuecomment-816392253 ## CI report: * f0f37cfe199dc670e8908b05a7df7ace8c2ecfd4 UNKNOWN * 556c5fe14574005af9f6937c6ab4e1a5dc888d29 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16706) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] fsk119 commented on pull request #15585: [FLINK-22065] [sql-client]Beautify the parse error exception
fsk119 commented on pull request #15585: URL: https://github.com/apache/flink/pull/15585#issuecomment-821850442 There are check style errors. Use `mvn com.diffplug.spotless:spotless-maven-plugin:apply` to fix these. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] fsk119 commented on a change in pull request #15585: [FLINK-22065] [sql-client]Beautify the parse error exception
fsk119 commented on a change in pull request #15585:
URL: https://github.com/apache/flink/pull/15585#discussion_r615272815
##
File path:
flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/exception/SqlParseException.java
##
@@ -0,0 +1,32 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.client.exception;
+
+/** Exception thrown during the parse of SQL statements. */
+public class SqlParseException extends SqlClientException {
+private static final long serialVersionUID = 1L;
+
+public SqlParseException(String message) {
+super(message);
+}
Review comment:
This is method is not used. Remove this.
##
File path:
flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/cli/CliClient.java
##
@@ -320,8 +324,8 @@ private boolean executeStatement(String statement,
ExecutionMode executionMode)
try {
final Optional operation = parseCommand(statement);
operation.ifPresent(op -> callOperation(op, executionMode));
-} catch (SqlExecutionException e) {
-printExecutionException(e);
+} catch (SqlClientException e) {
+printSqlClientException(e);
return false;
Review comment:
What about
```
catch (SqlParseException e) {
printSqlParseException(e);
}
catch (SqlExecutionException e) {
printSqlExecutionException(e);
}
```
We can remove the `instance of`.
##
File path:
flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/gateway/local/LocalExecutor.java
##
@@ -175,10 +176,10 @@ public Operation parseStatement(String sessionId, String
statement)
try {
operations = context.wrapClassLoader(() ->
parser.parse(statement));
} catch (Exception e) {
-throw new SqlExecutionException("Failed to parse statement: " +
statement, e);
+throw new SqlParseException("Failed to parse statement: " +
statement, e);
}
if (operations.isEmpty()) {
-throw new SqlExecutionException("Failed to parse statement: " +
statement);
+throw new SqlExecutionException("invalid statement: " + statement);
Review comment:
Use `SqlParseException`? I think the origin exception message is good
enough.
##
File path:
flink-table/flink-sql-client/src/main/java/org/apache/flink/table/client/gateway/Executor.java
##
@@ -69,74 +70,71 @@
* Both this method and {@link #getSessionConfigMap(String)} return the
same configuration
* set, but different return type.
*/
-ReadableConfig getSessionConfig(String sessionId) throws
SqlExecutionException;
+ReadableConfig getSessionConfig(String sessionId) throws
SqlClientException;
/**
* Reset all the properties for the given session identifier.
*
* @param sessionId to identifier the session
- * @throws SqlExecutionException if any error happen.
+ * @throws SqlClientException if any error happen.
*/
-void resetSessionProperties(String sessionId) throws SqlExecutionException;
+void resetSessionProperties(String sessionId) throws SqlClientException;
/**
* Reset given key's the session property for default value, if key is not
defined in config
* file, then remove it.
*
* @param sessionId to identifier the session
* @param key of need to reset the session property
- * @throws SqlExecutionException if any error happen.
+ * @throws SqlClientException if any error happen.
*/
-void resetSessionProperty(String sessionId, String key) throws
SqlExecutionException;
+void resetSessionProperty(String sessionId, String key) throws
SqlClientException;
/**
* Set given key's session property to the specific value.
*
* @param key of the session property
* @param value of the session property
- * @throws SqlExecutionException if any error happen.
+ * @throws SqlClientException if any error happen.
*/
-void setSessionProperty(String sessionId, String key, String value)
-
[GitHub] [flink] flinkbot edited a comment on pull request #15483: [FLINK-22092][hive] Ignore static conf file URLs in HiveConf
flinkbot edited a comment on pull request #15483: URL: https://github.com/apache/flink/pull/15483#issuecomment-812520117 ## CI report: * b91b371f5b6a9b522c2478ec56dfe48231c9c976 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16705) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15654: [FLINK-22141] Bumps checkstyle, junit, log4j version for security
flinkbot edited a comment on pull request #15654: URL: https://github.com/apache/flink/pull/15654#issuecomment-821836292 ## CI report: * ab75719fea8ed2bc179cfe18557211fd27510f3f Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16709) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15121: The method $(String) is undefined for the type TableExample
flinkbot edited a comment on pull request #15121: URL: https://github.com/apache/flink/pull/15121#issuecomment-793480240 ## CI report: * cf26ce895a7956d258acc073817f578558e78227 UNKNOWN * 709c4110370b845f42d916a06e56c6026cf2fac8 UNKNOWN * 4ea99332e1997eebca1f3f0a9d9229b8265fe32c UNKNOWN * 902a0f54b487ee39a8adb0e04c826bcc4a9dcab7 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16641) * d39b3fb096a3c883d36d3dcc85ea93efda8edd3d Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16708) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] AHeise commented on a change in pull request #15631: [FLINK-20103][tests] Add UnalignedCheckpointsStressITCase
AHeise commented on a change in pull request #15631:
URL: https://github.com/apache/flink/pull/15631#discussion_r615265639
##
File path:
flink-tests/src/test/java/org/apache/flink/test/checkpointing/UnalignedCheckpointStressITCase.java
##
@@ -0,0 +1,553 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.test.checkpointing;
+
+import org.apache.flink.api.common.functions.AbstractRichFunction;
+import org.apache.flink.api.common.functions.MapFunction;
+import org.apache.flink.api.common.restartstrategy.RestartStrategies;
+import org.apache.flink.api.common.state.CheckpointListener;
+import org.apache.flink.api.common.state.ListState;
+import org.apache.flink.api.common.state.ListStateDescriptor;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.configuration.RestOptions;
+import org.apache.flink.runtime.jobgraph.JobGraph;
+import org.apache.flink.runtime.jobgraph.SavepointRestoreSettings;
+import org.apache.flink.runtime.operators.testutils.ExpectedTestException;
+import org.apache.flink.runtime.state.FunctionInitializationContext;
+import org.apache.flink.runtime.state.FunctionSnapshotContext;
+import org.apache.flink.runtime.testutils.MiniClusterResourceConfiguration;
+import org.apache.flink.streaming.api.checkpoint.CheckpointedFunction;
+import org.apache.flink.streaming.api.datastream.DataStream;
+import org.apache.flink.streaming.api.datastream.DataStreamSource;
+import org.apache.flink.streaming.api.datastream.DataStreamUtils;
+import org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator;
+import
org.apache.flink.streaming.api.environment.CheckpointConfig.ExternalizedCheckpointCleanup;
+import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
+import org.apache.flink.streaming.api.functions.co.KeyedCoProcessFunction;
+import org.apache.flink.streaming.api.functions.source.ParallelSourceFunction;
+import
org.apache.flink.streaming.api.functions.windowing.ProcessWindowFunction;
+import org.apache.flink.streaming.api.graph.StreamGraph;
+import
org.apache.flink.streaming.api.windowing.assigners.TumblingProcessingTimeWindows;
+import org.apache.flink.streaming.api.windowing.time.Time;
+import org.apache.flink.streaming.api.windowing.windows.TimeWindow;
+import org.apache.flink.test.util.MiniClusterWithClientResource;
+import org.apache.flink.util.Collector;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FileUtils;
+import org.apache.flink.util.TestLogger;
+
+import org.junit.After;
+import org.junit.Before;
+import org.junit.ClassRule;
+import org.junit.Test;
+import org.junit.rules.TemporaryFolder;
+
+import javax.annotation.Nullable;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Optional;
+import java.util.concurrent.ThreadLocalRandom;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+import static
org.apache.flink.configuration.CheckpointingOptions.CHECKPOINTS_DIRECTORY;
+import static
org.apache.flink.configuration.CheckpointingOptions.MAX_RETAINED_CHECKPOINTS;
+import static
org.apache.flink.shaded.guava18.com.google.common.collect.Iterables.getOnlyElement;
+import static org.apache.flink.test.util.TestUtils.submitJobAndWaitForResult;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * A stress test that runs for a pre-defined amount of time, verifying data
correctness and every
+ * couple of checkpoints is triggering fail over to stress test unaligned
checkpoints.
+ */
+@SuppressWarnings("serial")
+public class UnalignedCheckpointStressITCase extends TestLogger {
+
+protected static final int CHECKPOINT_INTERVAL = 20;
+protected static final int MINIMUM_COMPLETED_CHECKPOINTS_BETWEEN_FAILURES
= 2;
+protected static final int MAXIMUM_COMPLETED_CHECKPOINTS_BETWEEN_FAILURES
= 10;
+protected static final long TEST_DURATION =
Time.minutes(1).toMilliseconds();
+protected static final int
[GitHub] [flink] AHeise commented on a change in pull request #15631: [FLINK-20103][tests] Add UnalignedCheckpointsStressITCase
AHeise commented on a change in pull request #15631:
URL: https://github.com/apache/flink/pull/15631#discussion_r615264959
##
File path:
flink-tests/src/test/java/org/apache/flink/test/checkpointing/UnalignedCheckpointStressITCase.java
##
@@ -0,0 +1,553 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.test.checkpointing;
+
+import org.apache.flink.api.common.functions.AbstractRichFunction;
+import org.apache.flink.api.common.functions.MapFunction;
+import org.apache.flink.api.common.restartstrategy.RestartStrategies;
+import org.apache.flink.api.common.state.CheckpointListener;
+import org.apache.flink.api.common.state.ListState;
+import org.apache.flink.api.common.state.ListStateDescriptor;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.configuration.RestOptions;
+import org.apache.flink.runtime.jobgraph.JobGraph;
+import org.apache.flink.runtime.jobgraph.SavepointRestoreSettings;
+import org.apache.flink.runtime.operators.testutils.ExpectedTestException;
+import org.apache.flink.runtime.state.FunctionInitializationContext;
+import org.apache.flink.runtime.state.FunctionSnapshotContext;
+import org.apache.flink.runtime.testutils.MiniClusterResourceConfiguration;
+import org.apache.flink.streaming.api.checkpoint.CheckpointedFunction;
+import org.apache.flink.streaming.api.datastream.DataStream;
+import org.apache.flink.streaming.api.datastream.DataStreamSource;
+import org.apache.flink.streaming.api.datastream.DataStreamUtils;
+import org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator;
+import
org.apache.flink.streaming.api.environment.CheckpointConfig.ExternalizedCheckpointCleanup;
+import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
+import org.apache.flink.streaming.api.functions.co.KeyedCoProcessFunction;
+import org.apache.flink.streaming.api.functions.source.ParallelSourceFunction;
+import
org.apache.flink.streaming.api.functions.windowing.ProcessWindowFunction;
+import org.apache.flink.streaming.api.graph.StreamGraph;
+import
org.apache.flink.streaming.api.windowing.assigners.TumblingProcessingTimeWindows;
+import org.apache.flink.streaming.api.windowing.time.Time;
+import org.apache.flink.streaming.api.windowing.windows.TimeWindow;
+import org.apache.flink.test.util.MiniClusterWithClientResource;
+import org.apache.flink.util.Collector;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FileUtils;
+import org.apache.flink.util.TestLogger;
+
+import org.junit.After;
+import org.junit.Before;
+import org.junit.ClassRule;
+import org.junit.Test;
+import org.junit.rules.TemporaryFolder;
+
+import javax.annotation.Nullable;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Optional;
+import java.util.concurrent.ThreadLocalRandom;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+import static
org.apache.flink.configuration.CheckpointingOptions.CHECKPOINTS_DIRECTORY;
+import static
org.apache.flink.configuration.CheckpointingOptions.MAX_RETAINED_CHECKPOINTS;
+import static
org.apache.flink.shaded.guava18.com.google.common.collect.Iterables.getOnlyElement;
+import static org.apache.flink.test.util.TestUtils.submitJobAndWaitForResult;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * A stress test that runs for a pre-defined amount of time, verifying data
correctness and every
+ * couple of checkpoints is triggering fail over to stress test unaligned
checkpoints.
+ */
+@SuppressWarnings("serial")
+public class UnalignedCheckpointStressITCase extends TestLogger {
+
+protected static final int CHECKPOINT_INTERVAL = 20;
+protected static final int MINIMUM_COMPLETED_CHECKPOINTS_BETWEEN_FAILURES
= 2;
+protected static final int MAXIMUM_COMPLETED_CHECKPOINTS_BETWEEN_FAILURES
= 10;
+protected static final long TEST_DURATION =
Time.minutes(1).toMilliseconds();
+protected static final int
[GitHub] [flink] flinkbot commented on pull request #15654: [FLINK-22141] Bumps checkstyle, junit, log4j version for security
flinkbot commented on pull request #15654: URL: https://github.com/apache/flink/pull/15654#issuecomment-821836292 ## CI report: * ab75719fea8ed2bc179cfe18557211fd27510f3f UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15653: [FLINK-22329][hive] Inject current ugi credentials into jobconf when getting file split in hive connector
flinkbot edited a comment on pull request #15653: URL: https://github.com/apache/flink/pull/15653#issuecomment-821801891 ## CI report: * 41dbfa9649149d9616337cccf4fc8f6c6f59a9c5 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16704) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Closed] (FLINK-22314) AggRecordsCombiner should combine buffered records first instead of accumulate on state directly
[
https://issues.apache.org/jira/browse/FLINK-22314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu closed FLINK-22314.
---
Resolution: Invalid
> AggRecordsCombiner should combine buffered records first instead of
> accumulate on state directly
>
>
> Key: FLINK-22314
> URL: https://issues.apache.org/jira/browse/FLINK-22314
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Runtime
>Reporter: Jark Wu
>Assignee: Jark Wu
>Priority: Major
>
> In Window TVF Aggregation, currently, the {{AggRecordsCombiner}} accumulates
> buffered records on state directly. This is not good for performance. We can
> accumulate records in memory first, and then merge the accumulator into
> state, if the aggs support {{merge()}} method. This can reduce lots of state
> accessing when having {{COUNT DISTINCT}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [flink] wuchong commented on a change in pull request #13011: [FLINK-16384][table sql/client] Support SHOW CREATE TABLE statement
wuchong commented on a change in pull request #13011:
URL: https://github.com/apache/flink/pull/13011#discussion_r465552827
##
File path:
flink-table/flink-table-common/src/main/java/org/apache/flink/table/api/constraints/UniqueConstraint.java
##
@@ -75,21 +76,26 @@ public ConstraintType getType() {
*/
@Override
public final String asSummaryString() {
- final String typeString;
- switch (getType()) {
- case PRIMARY_KEY:
- typeString = "PRIMARY KEY";
- break;
- case UNIQUE_KEY:
- typeString = "UNIQUE";
- break;
- default:
- throw new IllegalStateException("Unknown key
type: " + getType());
- }
-
+ final String typeString = getTypeString();
return String.format("CONSTRAINT %s %s (%s)", getName(),
typeString, String.join(", ", columns));
}
+ /**
+* Returns constraint's canonical summary. All constraints summary will
be formatted as
+*
+* CONSTRAINT [constraint-name] [constraint-type]
([constraint-definition]) NOT ENFORCED
+*
+* E.g CONSTRAINT pk PRIMARY KEY (`f0`, `f1`) NOT ENFORCED
+*
+*/
+ public final String asCanonicalString() {
+ final String typeString = getTypeString();
+ return String.format("CONSTRAINT %s %s (%s) NOT ENFORCED",
+ getName(),
+ typeString,
+ String.join(", ", columns.stream().map(col ->
String.format("`%s`", col)).collect(Collectors.toList(;
Review comment:
Use `EncodingUtils#escapeIdentifier` to escape identifiers.
##
File path: docs/content/docs/dev/table/sql/show.md
##
@@ -427,6 +458,17 @@ SHOW TABLES
Show all tables in the current catalog and the current database.
+
+## SHOW CREATE TABLE
+
+```sql
+SHOW CREATE TABLE
+```
+
+Show create table statement for specified table.
+
+Attention Currently `SHOW CREATE
TABLE` only supports table that is created by Flink SQL.
Review comment:
minor:
```suggestion
Attention Currently `SHOW CREATE
TABLE` only supports table that is created by Flink SQL DDL.
```
##
File path:
flink-table/flink-table-common/src/main/java/org/apache/flink/table/api/constraints/UniqueConstraint.java
##
@@ -75,21 +76,26 @@ public ConstraintType getType() {
*/
@Override
public final String asSummaryString() {
- final String typeString;
- switch (getType()) {
- case PRIMARY_KEY:
- typeString = "PRIMARY KEY";
- break;
- case UNIQUE_KEY:
- typeString = "UNIQUE";
- break;
- default:
- throw new IllegalStateException("Unknown key
type: " + getType());
- }
-
+ final String typeString = getTypeString();
return String.format("CONSTRAINT %s %s (%s)", getName(),
typeString, String.join(", ", columns));
Review comment:
I think we should also add `NOT ENFORCED` for the summary string.
Currentlly, it is not correct.
Besides, we should add `NOT ENFORCED` according the underlying `enforced`
flag, even though it is always flase for now.
##
File path:
flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/TableEnvironmentImpl.java
##
@@ -1072,6 +1086,75 @@ private TableResult buildShowResult(String columnName,
String[] objects) {
Arrays.stream(objects).map((c) -> new
String[]{c}).toArray(String[][]::new));
}
+ private TableResult buildShowCreateTableResult(CatalogBaseTable table,
ObjectIdentifier sqlIdentifier) {
Review comment:
I think this must be a `CatalogTable`, because this is `SHOW CREATE
TABLE`. We should throw exception before this if it is a view.
##
File path:
flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/TableEnvironmentImpl.java
##
@@ -1015,6 +1018,17 @@ private TableResult executeOperation(Operation
operation) {
return buildShowResult("database name",
listDatabases());
} else if (operation instanceof ShowCurrentDatabaseOperation) {
return buildShowResult("current database name", new
String[]{catalogManager.getCurrentDatabase()});
+ } else if (operation instanceof ShowCreateTableOperation) {
+ ShowCreateTableOperation showCreateTableOperation =
(ShowCreateTableOperation) operation;
+ Optional
[GitHub] [flink] flinkbot edited a comment on pull request #15534: [FLINK-22104][sql-client] Fix unstable SqlClientTest.testExecuteSqlFile
flinkbot edited a comment on pull request #15534: URL: https://github.com/apache/flink/pull/15534#issuecomment-816392253 ## CI report: * f0f37cfe199dc670e8908b05a7df7ace8c2ecfd4 UNKNOWN * 8a566552f9d186ed1736686f60f13f943dc70838 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16680) * 556c5fe14574005af9f6937c6ab4e1a5dc888d29 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16706) Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15121: The method $(String) is undefined for the type TableExample
flinkbot edited a comment on pull request #15121: URL: https://github.com/apache/flink/pull/15121#issuecomment-793480240 ## CI report: * cf26ce895a7956d258acc073817f578558e78227 UNKNOWN * 709c4110370b845f42d916a06e56c6026cf2fac8 UNKNOWN * 4ea99332e1997eebca1f3f0a9d9229b8265fe32c UNKNOWN * 902a0f54b487ee39a8adb0e04c826bcc4a9dcab7 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16641) * d39b3fb096a3c883d36d3dcc85ea93efda8edd3d UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #15654: [FLINK-22141] Bumps checkstyle, junit, log4j version for security
flinkbot commented on pull request #15654: URL: https://github.com/apache/flink/pull/15654#issuecomment-821829689 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit ab75719fea8ed2bc179cfe18557211fd27510f3f (Sat Apr 17 14:17:07 UTC 2021) **Warnings:** * **1 pom.xml files were touched**: Check for build and licensing issues. * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (FLINK-22330) Bumps checkstyle, junit, log4j version for security
Brandon Jiang created FLINK-22330: - Summary: Bumps checkstyle, junit, log4j version for security Key: FLINK-22330 URL: https://issues.apache.org/jira/browse/FLINK-22330 Project: Flink Issue Type: Improvement Affects Versions: 1.12.2 Reporter: Brandon Jiang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink] brandonJY opened a new pull request #15654: [FLINK-22141] Bumps checkstyle, junit, log4j version for security
brandonJY opened a new pull request #15654: URL: https://github.com/apache/flink/pull/15654 ## What is the purpose of the change - Bump junit from 4.12 to 4.13.1 due to GHSA-269g-pwp5-87pp https://github.com/advisories/GHSA-269g-pwp5-87pp - Bump log4j.version from 2.12.1 to 2.13.3 due to CVE-2020-9488 https://github.com/advisories/GHSA-vwqq-5vrc-xw9h - Bump Bump checkstyle from 8.14 to 8.29 due to GHSA-763g-fqq7-48wg https://github.com/advisories/GHSA-763g-fqq7-48wg ## Brief change log - Bump junit from 4.12 to 4.13.1 due to GHSA-269g-pwp5-87pp https://github.com/advisories/GHSA-269g-pwp5-87pp - Bump log4j.version from 2.12.1 to 2.13.3 due to CVE-2020-9488 https://github.com/advisories/GHSA-vwqq-5vrc-xw9h - Bump Bump checkstyle from 8.14 to 8.29 due to GHSA-763g-fqq7-48wg https://github.com/advisories/GHSA-763g-fqq7-48wg ## Verifying this change This change is already covered by existing tests, ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes ) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: ( no) - The serializers: (yes) - The runtime per-record code paths (performance sensitive): (no) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes) - The S3 file system connector: (don't know) ## Documentation - Does this pull request introduce a new feature? (no) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15534: [FLINK-22104][sql-client] Fix unstable SqlClientTest.testExecuteSqlFile
flinkbot edited a comment on pull request #15534: URL: https://github.com/apache/flink/pull/15534#issuecomment-816392253 ## CI report: * f0f37cfe199dc670e8908b05a7df7ace8c2ecfd4 UNKNOWN * 8a566552f9d186ed1736686f60f13f943dc70838 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=16680) * 556c5fe14574005af9f6937c6ab4e1a5dc888d29 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run travis` re-run the last Travis build - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink] wuchong commented on a change in pull request #15559: [FLINK-21627][table-planner-blink] The digest of TableScan & Sink should consider table hints
wuchong commented on a change in pull request #15559:
URL: https://github.com/apache/flink/pull/15559#discussion_r615256245
##
File path:
flink-table/flink-table-planner-blink/src/main/java/org/apache/calcite/rel/logical/LogicalTableScan.java
##
@@ -0,0 +1,135 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.calcite.rel.logical;
+
+import com.google.common.collect.ImmutableList;
+import org.apache.calcite.plan.Convention;
+import org.apache.calcite.plan.RelOptCluster;
+import org.apache.calcite.plan.RelOptTable;
+import org.apache.calcite.plan.RelTraitSet;
+import org.apache.calcite.rel.RelCollationTraitDef;
+import org.apache.calcite.rel.RelInput;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.RelWriter;
+import org.apache.calcite.rel.core.TableScan;
+import org.apache.calcite.rel.hint.RelHint;
+import org.apache.calcite.schema.Table;
+
+import java.util.List;
+
+/**
+ * A LogicalTableScan reads all the rows from a {@link
RelOptTable}.
+ *
+ * This class is copied from Calcite because the {@link #explainTerms}
should consider hints.
+ *
+ * If the table is a net.sf.saffron.ext.JdbcTable, then this
is literally possible.
+ * But for other kinds of tables, there may be many ways to read the data from
the table. For some
+ * kinds of table, it may not even be possible to read all of the rows unless
some narrowing
+ * constraint is applied.
+ *
+ * In the example of the net.sf.saffron.ext.ReflectSchema
schema,
+ *
+ *
+ *
+ * select from fields
+ *
+ *
+ *
+ * cannot be implemented, but
+ *
+ *
+ *
+ * select from fields as f
+ * where f.getClass().getName().equals("java.lang.String")
+ *
+ *
+ *
+ * can. It is the optimizer's responsibility to find these ways, by
applying transformation
+ * rules.
+ *
+ * CALCITE-4581, Line 106: {@link #explainTerms} method should consider
hints.
Review comment:
Let's use the new strategy to trace modification? i.e. `// BEGIN FLINK
MODIFICATION`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Closed] (FLINK-3387) Replace all ByteArrayOutputStream and DataOutputStream use in State Backends by DataOutputSerializer
[
https://issues.apache.org/jira/browse/FLINK-3387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephan Ewen closed FLINK-3387.
---
Resolution: Abandoned
Closing as abandoned due to inactivity.
> Replace all ByteArrayOutputStream and DataOutputStream use in State Backends
> by DataOutputSerializer
>
>
> Key: FLINK-3387
> URL: https://issues.apache.org/jira/browse/FLINK-3387
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / State Backends
>Affects Versions: 1.0.0
>Reporter: Stephan Ewen
>Priority: Major
>
> Currently, the {{byte[]}} to object conversion and vice versa is always done
> by creating new byte array streams and Data Input/Output streams. That is
> pretty wasteful, we can optimize this with Flink's own classes:
> - {{DataInputDeserializer}}
> - {{DataOutputSerializer}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-3387) Replace all ByteArrayOutputStream and DataOutputStream use in State Backends by DataOutputSerializer
[
https://issues.apache.org/jira/browse/FLINK-3387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephan Ewen updated FLINK-3387:
Labels: (was: stale-assigned)
> Replace all ByteArrayOutputStream and DataOutputStream use in State Backends
> by DataOutputSerializer
>
>
> Key: FLINK-3387
> URL: https://issues.apache.org/jira/browse/FLINK-3387
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / State Backends
>Affects Versions: 1.0.0
>Reporter: Stephan Ewen
>Priority: Major
>
> Currently, the {{byte[]}} to object conversion and vice versa is always done
> by creating new byte array streams and Data Input/Output streams. That is
> pretty wasteful, we can optimize this with Flink's own classes:
> - {{DataInputDeserializer}}
> - {{DataOutputSerializer}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (FLINK-3387) Replace all ByteArrayOutputStream and DataOutputStream use in State Backends by DataOutputSerializer
[
https://issues.apache.org/jira/browse/FLINK-3387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephan Ewen reassigned FLINK-3387:
---
Assignee: (was: Subhobrata Dey)
> Replace all ByteArrayOutputStream and DataOutputStream use in State Backends
> by DataOutputSerializer
>
>
> Key: FLINK-3387
> URL: https://issues.apache.org/jira/browse/FLINK-3387
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / State Backends
>Affects Versions: 1.0.0
>Reporter: Stephan Ewen
>Priority: Major
> Labels: stale-assigned
>
> Currently, the {{byte[]}} to object conversion and vice versa is always done
> by creating new byte array streams and Data Input/Output streams. That is
> pretty wasteful, we can optimize this with Flink's own classes:
> - {{DataInputDeserializer}}
> - {{DataOutputSerializer}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (FLINK-1778) Improve normalized keys in composite key case
[ https://issues.apache.org/jira/browse/FLINK-1778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen closed FLINK-1778. --- Resolution: Abandoned Closing as abandoned. We rely less on normalized keys now during sorting, but rather just on the byte string representation. > Improve normalized keys in composite key case > - > > Key: FLINK-1778 > URL: https://issues.apache.org/jira/browse/FLINK-1778 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Task >Affects Versions: 0.9 >Reporter: Stephan Ewen >Priority: Major > > Currently, if we have a key (String, long), the String will take up the > entire normalized key space, without being fully discerning anyways. Limiting > the key prefix in size and giving space to the second key field should in > most cases improve the comparison efficiency. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (FLINK-4374) GroupReduce Broken for null Date
[
https://issues.apache.org/jira/browse/FLINK-4374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephan Ewen closed FLINK-4374.
---
Resolution: Abandoned
Closing as abandoned, due to long inactivity.
> GroupReduce Broken for null Date
>
>
> Key: FLINK-4374
> URL: https://issues.apache.org/jira/browse/FLINK-4374
> Project: Flink
> Issue Type: Bug
> Components: API / DataSet
>Reporter: Stefan Richter
>Assignee: Timo Walther
>Priority: Major
> Labels: stale-assigned
>
> The GroupReduceITCase has an error that allows a problem with {{null}} Dates
> to go uncovered:
> If I set the parallelism to 1 in {{testDateNullException()}} and all keys
> actually end up on the same operator, then there is a problem in the
> de/serialization.
> It seems that {{null}} values are somehow skipped by the serialization
> process (e.g. maybe no {{null}} indicator is written), which leads to wrong
> deserializations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (FLINK-1778) Improve normalized keys in composite key case
[ https://issues.apache.org/jira/browse/FLINK-1778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen updated FLINK-1778: Labels: (was: stale-assigned) > Improve normalized keys in composite key case > - > > Key: FLINK-1778 > URL: https://issues.apache.org/jira/browse/FLINK-1778 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Task >Affects Versions: 0.9 >Reporter: Stephan Ewen >Priority: Major > > Currently, if we have a key (String, long), the String will take up the > entire normalized key space, without being fully discerning anyways. Limiting > the key prefix in size and giving space to the second key field should in > most cases improve the comparison efficiency. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (FLINK-1778) Improve normalized keys in composite key case
[ https://issues.apache.org/jira/browse/FLINK-1778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen reassigned FLINK-1778: --- Assignee: (was: Stephan Ewen) > Improve normalized keys in composite key case > - > > Key: FLINK-1778 > URL: https://issues.apache.org/jira/browse/FLINK-1778 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Task >Affects Versions: 0.9 >Reporter: Stephan Ewen >Priority: Major > Labels: stale-assigned > > Currently, if we have a key (String, long), the String will take up the > entire normalized key space, without being fully discerning anyways. Limiting > the key prefix in size and giving space to the second key field should in > most cases improve the comparison efficiency. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink-statefun] birbilis commented on pull request #227: Update java.md - fixed code typo
birbilis commented on pull request #227: URL: https://github.com/apache/flink-statefun/pull/227#issuecomment-821814938 already proposed by other, see mention above -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-statefun] birbilis closed pull request #227: Update java.md - fixed code typo
birbilis closed pull request #227: URL: https://github.com/apache/flink-statefun/pull/227 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (FLINK-9043) Introduce a friendly way to resume the job from externalized checkpoints automatically
[ https://issues.apache.org/jira/browse/FLINK-9043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324255#comment-17324255 ] Stephan Ewen commented on FLINK-9043: - I think this might be a good one to revisit now. The last commonly stored checkpoint storage (S3) is not consistent, so an variant that scans the checkpoint directory for the latest checkpoint is feasible now. For that approach we would need to resolve FLINK-22008 first. > Introduce a friendly way to resume the job from externalized checkpoints > automatically > -- > > Key: FLINK-9043 > URL: https://issues.apache.org/jira/browse/FLINK-9043 > Project: Flink > Issue Type: New Feature > Components: Runtime / Checkpointing >Reporter: Godfrey He >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > I know a flink job can reovery from checkpoint with restart strategy, but can > not recovery as spark streaming jobs when job is starting. > Every time, the submitted flink job is regarded as a new job, while , in the > spark streaming job, which can detect the checkpoint directory first, and > then recovery from the latest succeed one. However, Flink only can recovery > until the job failed first, then retry with strategy. > > So, would flink support to recover from the checkpoint directly in a new job? > h2. New description by [~sihuazhou] > Currently, it's quite a bit not friendly for users to recover job from the > externalized checkpoint, user need to find the dedicate dir for the job which > is not a easy thing when there are too many jobs. This ticket attend to > introduce a more friendly way to allow the user to use the externalized > checkpoint to do recovery. > The implementation steps are copied from the comments of [~StephanEwen]: > - We could make this an option where you pass a flag (-r) to automatically > look for the latest checkpoint in a given directory. > - If more than one jobs checkpointed there before, this operation would fail. > - We might also need a way to have jobs not create the UUID subdirectory, > otherwise the scanning for the latest checkpoint would not easily work. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink-statefun] birbilis commented on pull request #224: [hotfix][docs] Fix typos in java.md
birbilis commented on pull request #224: URL: https://github.com/apache/flink-statefun/pull/224#issuecomment-821814838 seems I suggested the same fix here - https://github.com/apache/flink-statefun/pull/227 - for the 2nd typo, closing mine -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-statefun] birbilis commented on pull request #228: Update java.md
birbilis commented on pull request #228: URL: https://github.com/apache/flink-statefun/pull/228#issuecomment-821814585 not sure what the second change at the "keep learning" shown in the diff is about, used GitHub's online editor for the patch -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-statefun] birbilis commented on pull request #227: Update java.md - fixed code typo
birbilis commented on pull request #227: URL: https://github.com/apache/flink-statefun/pull/227#issuecomment-821814574 not sure what the second change at the "keep learning" shown in the diff is about, used GitHub's online editor for the patch -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-statefun] birbilis commented on pull request #226: Update java.md
birbilis commented on pull request #226: URL: https://github.com/apache/flink-statefun/pull/226#issuecomment-821814553 not sure what the second change at the "keep learning" shown in the diff is about, used GitHub's online editor for the patch -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-9043) Introduce a friendly way to resume the job from externalized checkpoints automatically
[ https://issues.apache.org/jira/browse/FLINK-9043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen updated FLINK-9043: Labels: pull-request-available (was: pull-request-available stale-assigned) > Introduce a friendly way to resume the job from externalized checkpoints > automatically > -- > > Key: FLINK-9043 > URL: https://issues.apache.org/jira/browse/FLINK-9043 > Project: Flink > Issue Type: New Feature > Components: Runtime / Checkpointing >Reporter: Godfrey He >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > I know a flink job can reovery from checkpoint with restart strategy, but can > not recovery as spark streaming jobs when job is starting. > Every time, the submitted flink job is regarded as a new job, while , in the > spark streaming job, which can detect the checkpoint directory first, and > then recovery from the latest succeed one. However, Flink only can recovery > until the job failed first, then retry with strategy. > > So, would flink support to recover from the checkpoint directly in a new job? > h2. New description by [~sihuazhou] > Currently, it's quite a bit not friendly for users to recover job from the > externalized checkpoint, user need to find the dedicate dir for the job which > is not a easy thing when there are too many jobs. This ticket attend to > introduce a more friendly way to allow the user to use the externalized > checkpoint to do recovery. > The implementation steps are copied from the comments of [~StephanEwen]: > - We could make this an option where you pass a flag (-r) to automatically > look for the latest checkpoint in a given directory. > - If more than one jobs checkpointed there before, this operation would fail. > - We might also need a way to have jobs not create the UUID subdirectory, > otherwise the scanning for the latest checkpoint would not easily work. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (FLINK-3389) Add Pre-defined Options settings for RocksDB State backend
[ https://issues.apache.org/jira/browse/FLINK-3389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen closed FLINK-3389. --- Resolution: Done > Add Pre-defined Options settings for RocksDB State backend > -- > > Key: FLINK-3389 > URL: https://issues.apache.org/jira/browse/FLINK-3389 > Project: Flink > Issue Type: Improvement > Components: Runtime / State Backends >Affects Versions: 1.0.0 >Reporter: Stephan Ewen >Assignee: Stephan Ewen >Priority: Major > Labels: stale-assigned > > The RocksDB State Backend performance can be optimized for certain settings > (for example running on disk or SSD) with certain options. > Since it is hard to tune for users, we should add a set of predefined options > for certain settings. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (FLINK-9043) Introduce a friendly way to resume the job from externalized checkpoints automatically
[ https://issues.apache.org/jira/browse/FLINK-9043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen reassigned FLINK-9043: --- Assignee: (was: Sihua Zhou) > Introduce a friendly way to resume the job from externalized checkpoints > automatically > -- > > Key: FLINK-9043 > URL: https://issues.apache.org/jira/browse/FLINK-9043 > Project: Flink > Issue Type: New Feature > Components: Runtime / Checkpointing >Reporter: Godfrey He >Priority: Major > Labels: pull-request-available, stale-assigned > Time Spent: 10m > Remaining Estimate: 0h > > I know a flink job can reovery from checkpoint with restart strategy, but can > not recovery as spark streaming jobs when job is starting. > Every time, the submitted flink job is regarded as a new job, while , in the > spark streaming job, which can detect the checkpoint directory first, and > then recovery from the latest succeed one. However, Flink only can recovery > until the job failed first, then retry with strategy. > > So, would flink support to recover from the checkpoint directly in a new job? > h2. New description by [~sihuazhou] > Currently, it's quite a bit not friendly for users to recover job from the > externalized checkpoint, user need to find the dedicate dir for the job which > is not a easy thing when there are too many jobs. This ticket attend to > introduce a more friendly way to allow the user to use the externalized > checkpoint to do recovery. > The implementation steps are copied from the comments of [~StephanEwen]: > - We could make this an option where you pass a flag (-r) to automatically > look for the latest checkpoint in a given directory. > - If more than one jobs checkpointed there before, this operation would fail. > - We might also need a way to have jobs not create the UUID subdirectory, > otherwise the scanning for the latest checkpoint would not easily work. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [flink-statefun] birbilis opened a new pull request #228: Update java.md
birbilis opened a new pull request #228: URL: https://github.com/apache/flink-statefun/pull/228 typo in json declaration -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [flink-statefun] birbilis opened a new pull request #227: Update java.md - fixed code typo
birbilis opened a new pull request #227: URL: https://github.com/apache/flink-statefun/pull/227 fixed code snippet typo -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (FLINK-19441) Performance regression on 24.09.2020
[ https://issues.apache.org/jira/browse/FLINK-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen updated FLINK-19441: - Labels: pull-request-available (was: pull-request-available stale-assigned) > Performance regression on 24.09.2020 > > > Key: FLINK-19441 > URL: https://issues.apache.org/jira/browse/FLINK-19441 > Project: Flink > Issue Type: Bug >Reporter: Arvid Heise >Assignee: Stephan Ewen >Priority: Critical > Labels: pull-request-available > > A couple of benchmarks are showing a small performance regression on > 24.09.2020: > http://codespeed.dak8s.net:8000/timeline/?ben=globalWindow=2 > http://codespeed.dak8s.net:8000/timeline/?ben=tupleKeyBy=2 (?) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (FLINK-19441) Performance regression on 24.09.2020
[ https://issues.apache.org/jira/browse/FLINK-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen reassigned FLINK-19441: Assignee: (was: Stephan Ewen) > Performance regression on 24.09.2020 > > > Key: FLINK-19441 > URL: https://issues.apache.org/jira/browse/FLINK-19441 > Project: Flink > Issue Type: Bug >Reporter: Arvid Heise >Priority: Critical > Labels: pull-request-available > > A couple of benchmarks are showing a small performance regression on > 24.09.2020: > http://codespeed.dak8s.net:8000/timeline/?ben=globalWindow=2 > http://codespeed.dak8s.net:8000/timeline/?ben=tupleKeyBy=2 (?) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-19499) Expose Metric Groups to Split Assigners
[ https://issues.apache.org/jira/browse/FLINK-19499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen updated FLINK-19499: - Labels: (was: stale-assigned) > Expose Metric Groups to Split Assigners > --- > > Key: FLINK-19499 > URL: https://issues.apache.org/jira/browse/FLINK-19499 > Project: Flink > Issue Type: Improvement > Components: Connectors / FileSystem >Reporter: Stephan Ewen >Priority: Major > Fix For: 1.13.0 > > > Split Assigners should have access to metric groups, so they can report > metrics on assignment, like pending splits, local-, and remote assignments. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (FLINK-19499) Expose Metric Groups to Split Assigners
[ https://issues.apache.org/jira/browse/FLINK-19499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephan Ewen reassigned FLINK-19499: Assignee: (was: Stephan Ewen) > Expose Metric Groups to Split Assigners > --- > > Key: FLINK-19499 > URL: https://issues.apache.org/jira/browse/FLINK-19499 > Project: Flink > Issue Type: Improvement > Components: Connectors / FileSystem >Reporter: Stephan Ewen >Priority: Major > Labels: stale-assigned > Fix For: 1.13.0 > > > Split Assigners should have access to metric groups, so they can report > metrics on assignment, like pending splits, local-, and remote assignments. -- This message was sent by Atlassian Jira (v8.3.4#803005)
