[GitHub] [flink] flinkbot edited a comment on issue #9212: [FLINK-13338][table-api] Sql conformance is hard to config in TableConfig
flinkbot edited a comment on issue #9212: [FLINK-13338][table-api] Sql conformance is hard to config in TableConfig URL: https://github.com/apache/flink/pull/9212#issuecomment-514486972 ## CI report: * 68443427c89cca6ef5c1c39685c57c9884e101af : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120314731) * 51463ef8d90e7a3398b59c1efcf6ce713aaddb03 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120315489) * c839ffc015637729a586b0c78f61c54bad21b299 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120315878) * 39df37c7b430ed295b5b55a7b21e677aafd5413d : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120426548) * 254e1cdc74f8d2e6d68f59606ee0b7867132e00d : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120448887) * 32080403f13f07e4d9a0508ce5dcaba47e484f2d : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120929261) * 80ed8e6f9466323b5a92763d47c71e13df0dc1bf : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120931278) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13159) java.lang.ClassNotFoundException when restore job
[
https://issues.apache.org/jira/browse/FLINK-13159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894254#comment-16894254
]
Liu Bo commented on FLINK-13159:
I'm using flink 1.8.1 on yarn session with rocksdb as state backend.
got the same problem when starting flink jobs with checkpoints: bin/flink run
-s /flink/checkpoints/c2b8fa3c51393a2c6865ca13045eccad/chk-84 deploy/xxx.jar
job keep retrying and fail after about 8 seconds.
Our job runs on flink 1.7.2 with no problem for about 4 months and could
recover successfully.
Full stack trace:
java.lang.RuntimeException: Exception occurred while processing valve output
watermark:
at
org.apache.flink.streaming.runtime.io.StreamInputProcessor$ForwardingValveOutputHandler.handleWatermark(StreamInputProcessor.java:265)
at
org.apache.flink.streaming.runtime.streamstatus.StatusWatermarkValve.findAndOutputNewMinWatermarkAcrossAlignedChannels(StatusWatermarkValve.java:189)
at
org.apache.flink.streaming.runtime.streamstatus.StatusWatermarkValve.inputWatermark(StatusWatermarkValve.java:111)
at
org.apache.flink.streaming.runtime.io.StreamInputProcessor.processInput(StreamInputProcessor.java:184)
at
org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.run(OneInputStreamTask.java:105)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:300)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Cannot instantiate class.
at
org.apache.flink.api.java.typeutils.runtime.PojoSerializer.deserialize(PojoSerializer.java:384)
at
org.apache.flink.api.java.typeutils.runtime.TupleSerializer.deserialize(TupleSerializer.java:143)
at
org.apache.flink.api.java.typeutils.runtime.TupleSerializer.deserialize(TupleSerializer.java:37)
at
org.apache.flink.streaming.api.datastream.CoGroupedStreams$UnionSerializer.deserialize(CoGroupedStreams.java:581)
at
org.apache.flink.streaming.api.datastream.CoGroupedStreams$UnionSerializer.deserialize(CoGroupedStreams.java:506)
at
org.apache.flink.contrib.streaming.state.RocksDBListState.deserializeNextElement(RocksDBListState.java:144)
at
org.apache.flink.contrib.streaming.state.RocksDBListState.deserializeList(RocksDBListState.java:135)
at
org.apache.flink.contrib.streaming.state.RocksDBListState.getInternal(RocksDBListState.java:119)
at
org.apache.flink.contrib.streaming.state.RocksDBListState.get(RocksDBListState.java:111)
at
org.apache.flink.contrib.streaming.state.RocksDBListState.get(RocksDBListState.java:60)
at
org.apache.flink.streaming.runtime.operators.windowing.WindowOperator.onEventTime(WindowOperator.java:452)
at
org.apache.flink.streaming.api.operators.InternalTimerServiceImpl.advanceWatermark(InternalTimerServiceImpl.java:255)
at
org.apache.flink.streaming.api.operators.InternalTimeServiceManager.advanceWatermark(InternalTimeServiceManager.java:128)
at
org.apache.flink.streaming.api.operators.AbstractStreamOperator.processWatermark(AbstractStreamOperator.java:775)
at
org.apache.flink.streaming.runtime.io.StreamInputProcessor$ForwardingValveOutputHandler.handleWatermark(StreamInputProcessor.java:262)
... 7 more
Caused by: java.lang.ClassNotFoundException: com/xx/xx/xx
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at
org.apache.flink.api.java.typeutils.runtime.PojoSerializer.deserialize(PojoSerializer.java:382)
> java.lang.ClassNotFoundException when restore job
> -
>
> Key: FLINK-13159
> URL: https://issues.apache.org/jira/browse/FLINK-13159
> Project: Flink
> Issue Type: Bug
> Components: API / Type Serialization System
>Reporter: kring
>Priority: Critical
>
> {code:java}
> java.lang.Exception: Exception while creating StreamOperatorStateContext.
> at
> org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:195)
> at
> org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:250)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.initializeState(StreamTask.java:738)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:289)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.flink.util.FlinkException: Could not restore keyed
> state backend for WindowOperator_b398b3dd4c544ddf2d47a0cc47d332f4_(1/6) from
> any of the 1 prov
> ided restore options.
> at
> org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:135)
> at
>
[GitHub] [flink] flinkbot edited a comment on issue #9212: [FLINK-13338][table-api] Sql conformance is hard to config in TableConfig
flinkbot edited a comment on issue #9212: [FLINK-13338][table-api] Sql conformance is hard to config in TableConfig URL: https://github.com/apache/flink/pull/9212#issuecomment-514486972 ## CI report: * 68443427c89cca6ef5c1c39685c57c9884e101af : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120314731) * 51463ef8d90e7a3398b59c1efcf6ce713aaddb03 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120315489) * c839ffc015637729a586b0c78f61c54bad21b299 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120315878) * 39df37c7b430ed295b5b55a7b21e677aafd5413d : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120426548) * 254e1cdc74f8d2e6d68f59606ee0b7867132e00d : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120448887) * 32080403f13f07e4d9a0508ce5dcaba47e484f2d : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120929261) * 80ed8e6f9466323b5a92763d47c71e13df0dc1bf : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120931278) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9243: [FLINK-13335][sql-parser] Align the SQL CREATE TABLE DDL with FLIP-37
flinkbot edited a comment on issue #9243: [FLINK-13335][sql-parser] Align the SQL CREATE TABLE DDL with FLIP-37 URL: https://github.com/apache/flink/pull/9243#issuecomment-515486131 ## CI report: * ae2528ac686c28a60641e8ea295221a510043e76 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120870435) * a4c045ee56e7d8001f8393baeb841280d81eac1a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120875424) * 4315d411277e2e36471e67b461dae56e752ee31a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120927821) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13273) Allow switching planners in SQL Client
[ https://issues.apache.org/jira/browse/FLINK-13273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894235#comment-16894235 ] Tzu-Li (Gordon) Tai commented on FLINK-13273: - It seems like the decision is to include this in 1.9.0, as it is rather low effort. However, there doesn't seem to be a PR yet, so I'm not sure whether RC1 should wait on this issue. [~twalthr] can you confirm? Thanks! > Allow switching planners in SQL Client > -- > > Key: FLINK-13273 > URL: https://issues.apache.org/jira/browse/FLINK-13273 > Project: Flink > Issue Type: New Feature > Components: Table SQL / Client >Affects Versions: 1.9.0 >Reporter: Timo Walther >Assignee: Timo Walther >Priority: Blocker > Fix For: 1.9.0 > > > Once FLINK-13267 is resolved, we can also enable switching planners in the > SQL Client via a execution property. Even though this is kind of a new > feature, it had to be postponed after the feature-freeze as the relocation of > FLINK-13267 would have created many merge conflicts otherwise. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (FLINK-13444) Translate English content of FLINK-13396 into Chinese
[
https://issues.apache.org/jira/browse/FLINK-13444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu updated FLINK-13444:
Issue Type: Sub-task (was: Task)
Parent: FLINK-11529
> Translate English content of FLINK-13396 into Chinese
> -
>
> Key: FLINK-13444
> URL: https://issues.apache.org/jira/browse/FLINK-13444
> Project: Flink
> Issue Type: Sub-task
> Components: chinese-translation, Documentation
>Reporter: Jark Wu
>Priority: Major
>
> Translate new added English content of FLINK-13396 to Chinese.
> The markdown file is located in {{docs/dev/connectors/filesystem_sink.zh.md}}.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (FLINK-13444) Translate English content of FLINK-13396 into Chinese
Jark Wu created FLINK-13444:
---
Summary: Translate English content of FLINK-13396 into Chinese
Key: FLINK-13444
URL: https://issues.apache.org/jira/browse/FLINK-13444
Project: Flink
Issue Type: Task
Components: chinese-translation, Documentation
Reporter: Jark Wu
Translate new added English content of FLINK-13396 to Chinese.
The markdown file is located in {{docs/dev/connectors/filesystem_sink.zh.md}}.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [flink] flinkbot edited a comment on issue #9212: [FLINK-13338][table-api] Sql conformance is hard to config in TableConfig
flinkbot edited a comment on issue #9212: [FLINK-13338][table-api] Sql conformance is hard to config in TableConfig URL: https://github.com/apache/flink/pull/9212#issuecomment-514486972 ## CI report: * 68443427c89cca6ef5c1c39685c57c9884e101af : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120314731) * 51463ef8d90e7a3398b59c1efcf6ce713aaddb03 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120315489) * c839ffc015637729a586b0c78f61c54bad21b299 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120315878) * 39df37c7b430ed295b5b55a7b21e677aafd5413d : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120426548) * 254e1cdc74f8d2e6d68f59606ee0b7867132e00d : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120448887) * 32080403f13f07e4d9a0508ce5dcaba47e484f2d : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120929261) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13432) Set max polling interval threshold for job result polling
[ https://issues.apache.org/jira/browse/FLINK-13432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894226#comment-16894226 ] vinoyang commented on FLINK-13432: -- [~zjffdu] I think it's reasonable. cc [~aljoscha] > Set max polling interval threshold for job result polling > - > > Key: FLINK-13432 > URL: https://issues.apache.org/jira/browse/FLINK-13432 > Project: Flink > Issue Type: Improvement > Components: Command Line Client >Affects Versions: 1.9.0 >Reporter: Jeff Zhang >Priority: Major > > Currently, flink will increase polling interval by one ms after each polling. > It might be ok for most of the cases, but would be better to set a max > interval polling threshold -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [flink] flinkbot edited a comment on issue #9212: [FLINK-13338][table-api] Sql conformance is hard to config in TableConfig
flinkbot edited a comment on issue #9212: [FLINK-13338][table-api] Sql conformance is hard to config in TableConfig URL: https://github.com/apache/flink/pull/9212#issuecomment-514486972 ## CI report: * 68443427c89cca6ef5c1c39685c57c9884e101af : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120314731) * 51463ef8d90e7a3398b59c1efcf6ce713aaddb03 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120315489) * c839ffc015637729a586b0c78f61c54bad21b299 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120315878) * 39df37c7b430ed295b5b55a7b21e677aafd5413d : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120426548) * 254e1cdc74f8d2e6d68f59606ee0b7867132e00d : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120448887) * 32080403f13f07e4d9a0508ce5dcaba47e484f2d : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120929261) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] yanghua edited a comment on issue #9238: [FLINK-13387][WebUI] Fix log download for old UI
yanghua edited a comment on issue #9238: [FLINK-13387][WebUI] Fix log download for old UI URL: https://github.com/apache/flink/pull/9238#issuecomment-515641316 Hi @zentol When clicking the download button, the behavior is that it will open a new web page and show the content of the whole log file. There is a simple way to avoid this behavior, we could add a `download` attribute for `` tag which is [a standard of HTML](https://www.w3schools.com/tags/att_a_download.asp). It seems the new web UI also uses this attribute. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] yanghua commented on issue #9238: [FLINK-13387][WebUI] Fix log download for old UI
yanghua commented on issue #9238: [FLINK-13387][WebUI] Fix log download for old UI URL: https://github.com/apache/flink/pull/9238#issuecomment-515641316 Hi @zentol When clicking the download button, the behavior is open a new web page and show the content of the whole log file. There is a simple way to avoid this behavior, we could add a `download` attribute for `` tag which is [a standard of HTML](https://www.w3schools.com/tags/att_a_download.asp). It seems the new web UI also uses this attribute. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] danny0405 commented on issue #9212: [FLINK-13338][table-api] Sql conformance is hard to config in TableConfig
danny0405 commented on issue #9212: [FLINK-13338][table-api] Sql conformance is hard to config in TableConfig URL: https://github.com/apache/flink/pull/9212#issuecomment-515641356 Hi @twalthr , i have updated for your review address, please review again if you have time, thanks ~ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] danny0405 commented on a change in pull request #9212: [FLINK-13338][table-api] Sql conformance is hard to config in TableConfig
danny0405 commented on a change in pull request #9212: [FLINK-13338][table-api]
Sql conformance is hard to config in TableConfig
URL: https://github.com/apache/flink/pull/9212#discussion_r307948160
##
File path:
flink-table/flink-table-planner-blink/src/test/java/org/apache/flink/table/sqlexec/SqlToOperationConverterTest.java
##
@@ -0,0 +1,155 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.sqlexec;
+
+import org.apache.flink.table.api.DataTypes;
+import org.apache.flink.table.api.EnvironmentSettings;
+import org.apache.flink.table.api.SqlDialect;
+import org.apache.flink.table.api.internal.TableEnvironmentImpl;
+import org.apache.flink.table.catalog.CatalogTable;
+import org.apache.flink.table.delegation.Planner;
+import org.apache.flink.table.operations.CatalogSinkModifyOperation;
+import org.apache.flink.table.operations.Operation;
+import org.apache.flink.table.operations.ddl.CreateTableOperation;
+import org.apache.flink.table.planner.operations.SqlConversionException;
+import org.apache.flink.table.types.DataType;
+
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Test;
+
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import static org.junit.Assert.assertArrayEquals;
+import static org.junit.Assert.assertEquals;
+
+/** Test cases for SqlExecutableStatement. **/
+public class SqlToOperationConverterTest {
+ private static final EnvironmentSettings settings =
EnvironmentSettings.newInstance()
Review comment:
Agreed and updated, thanks.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (FLINK-13387) Can not download taskmanger & jobmanager's logs in the old UI
[ https://issues.apache.org/jira/browse/FLINK-13387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894221#comment-16894221 ] vinoyang commented on FLINK-13387: -- [~Zentol] OK, Glad to sound this message. > Can not download taskmanger & jobmanager's logs in the old UI > - > > Key: FLINK-13387 > URL: https://issues.apache.org/jira/browse/FLINK-13387 > Project: Flink > Issue Type: Bug > Components: Runtime / Web Frontend >Affects Versions: 1.9.0 >Reporter: Dawid Wysakowicz >Assignee: Chesnay Schepler >Priority: Critical > Labels: pull-request-available > Fix For: 1.9.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > It is not possible to download the taskmanager & jobmanager logs via the old > UI. > The exception is: "Unable to load requested file > /old-version/taskmanagers/1234ddfcc0b1d06d2615d5431a08c7b8/stdout." -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [flink] flinkbot edited a comment on issue #9245: [FLINK-13334][coordination] Remove legacy implementation of slot
flinkbot edited a comment on issue #9245: [FLINK-13334][coordination] Remove legacy implementation of slot URL: https://github.com/apache/flink/pull/9245#issuecomment-515507399 ## CI report: * 2c7c67a0d15016887e9c917fee0bfe34e9d0e131 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120879420) * 6024d92f8e533fcee4877714bf57fa267bfa79da : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120923892) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9243: [FLINK-13335][sql-parser] Align the SQL CREATE TABLE DDL with FLIP-37
flinkbot edited a comment on issue #9243: [FLINK-13335][sql-parser] Align the SQL CREATE TABLE DDL with FLIP-37 URL: https://github.com/apache/flink/pull/9243#issuecomment-515486131 ## CI report: * ae2528ac686c28a60641e8ea295221a510043e76 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120870435) * a4c045ee56e7d8001f8393baeb841280d81eac1a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120875424) * 4315d411277e2e36471e67b461dae56e752ee31a : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120927821) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9245: [FLINK-13334][coordination] Remove legacy implementation of slot
flinkbot edited a comment on issue #9245: [FLINK-13334][coordination] Remove legacy implementation of slot URL: https://github.com/apache/flink/pull/9245#issuecomment-515507399 ## CI report: * 2c7c67a0d15016887e9c917fee0bfe34e9d0e131 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120879420) * 6024d92f8e533fcee4877714bf57fa267bfa79da : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120923892) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-13334) Remove legacy slot implementation
[
https://issues.apache.org/jira/browse/FLINK-13334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
TisonKun updated FLINK-13334:
-
Summary: Remove legacy slot implementation (was: Remove legacy
implementation of slot)
> Remove legacy slot implementation
> -
>
> Key: FLINK-13334
> URL: https://issues.apache.org/jira/browse/FLINK-13334
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination
>Affects Versions: 1.10.0
>Reporter: TisonKun
>Assignee: TisonKun
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.10.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> cc [~till.rohrmann] [~srichter]
> From my investigation currently Flink use {{SlotSharingManager}} and
> {{SlotSharingGroupId}} for achieving slot sharing. And thus
> {{SlotSharingGroupAssignment}} {{SlotSharingGroup}} and {{SharedSlot}} are
> all legacy concept.
> Notice that the ongoing scheduler re-design touches frequently tests based on
> legacy slot/instance logic or even uses it for testing. I'd like to nudge
> this process for totally remove legacy code from our code base.
> Also I attach a patch on FLINK-12179 that remove {{Instance}}. With current
> contribution workflow your shepherds are significant :- )
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [flink] flinkbot edited a comment on issue #9210: [FLINK-12746][docs] Getting Started - DataStream Example Walkthrough
flinkbot edited a comment on issue #9210: [FLINK-12746][docs] Getting Started - DataStream Example Walkthrough URL: https://github.com/apache/flink/pull/9210#issuecomment-514437706 ## CI report: * 5eb979da047c442c0205464c92b5bd9ee3a740dc : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120299964) * d7bf53a30514664925357bd5817305a02553d0a3 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120506936) * 02cca7fb6283b84a20ee019159ccb023ccffbd82 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120769129) * 5009b10d38eef92f25bfe4ff4608f2dd121ea9c6 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120915709) * e3b272586d8f41d3800e86134730c4dc427952a6 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120916220) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9210: [FLINK-12746][docs] Getting Started - DataStream Example Walkthrough
flinkbot edited a comment on issue #9210: [FLINK-12746][docs] Getting Started - DataStream Example Walkthrough URL: https://github.com/apache/flink/pull/9210#issuecomment-514437706 ## CI report: * 5eb979da047c442c0205464c92b5bd9ee3a740dc : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120299964) * d7bf53a30514664925357bd5817305a02553d0a3 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120506936) * 02cca7fb6283b84a20ee019159ccb023ccffbd82 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120769129) * 5009b10d38eef92f25bfe4ff4608f2dd121ea9c6 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120915709) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xuefuz commented on issue #9237: [FLINK-13431][hive] NameNode HA configuration was not loaded when running HiveConnector on Yarn
xuefuz commented on issue #9237: [FLINK-13431][hive] NameNode HA configuration was not loaded when running HiveConnector on Yarn URL: https://github.com/apache/flink/pull/9237#issuecomment-515591055 Thanks for the contribution! The are a couple of high-level suggestions: 1. Please update the PR description detailing the changes introduced by this PR. 2. The PR seemingly breaks the build. 3. Consider adding some tests to cover the changes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9169: [FLINK-12998][docs] Update documentation for file systems loading as plugins
flinkbot edited a comment on issue #9169: [FLINK-12998][docs] Update documentation for file systems loading as plugins URL: https://github.com/apache/flink/pull/9169#issuecomment-512820058 ## CI report: * a7bfcd7a2605fe9299655cb26d3f829b4751ecd6 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119638448) * 8a0a1f2a8f13b1f8e55e50401fee5dbf0d8c1ed4 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119766161) * d3541a995150495d527b721053ef1a4077053908 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119795433) * d89f14eaa970b880a88238c8d66e71095534b868 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119799191) * 24b3afb25bad946bdb8c72965aba4833b25c46b1 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119806398) * 45178e4407304d4d8561c9d810d2ad67ad3e6a86 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120836774) * a46d68bc5d7932c2165a898d3e8e8b0a3bd5c1a3 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120837450) * 393f2e64a2ffcf06577af43099ee4dfe0cd097c1 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120852616) * c3a31748e4d8e8aeebf538bdf9a57baf9259fe4e : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120881311) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough
flinkbot edited a comment on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough URL: https://github.com/apache/flink/pull/8903#issuecomment-510464651 ## CI report: * b2821a6ae97fd943f3a66b672e85fbd2374126c4 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/118909729) * 0699f7e5f2240a4a1bc44c15f08e6a1df47d3b01 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119054579) * 509e634257496dd2d8d42d512901f5eb46a82c50 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119406891) * 7579df06b6a0bf799e8a9c2bcb09984bf52c8e8c : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119441302) * ccb9dc29d4755d0a6c4596e08743b38615eb276a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120480063) * 1b976f30a689d9bdbf65513f034b2954bfb91468 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120494302) * 3ccee75dd0d506b90a2019cde9045eee26a4f4d5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120749125) * e07648c718b4ea32a3f02f826ca6a337400572be : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120769160) * cf082ff54f7bd160b9e0eb316459f419defdd0b7 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120872425) * b9b37f124c7c92c4d4b0c4e2101be33d9b86babd : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120877385) * b09bc12655a6bfac8d3deb83dac24bf20b954423 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120879446) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9245: [FLINK-13334][coordination] Remove legacy implementation of slot
flinkbot edited a comment on issue #9245: [FLINK-13334][coordination] Remove legacy implementation of slot URL: https://github.com/apache/flink/pull/9245#issuecomment-515507399 ## CI report: * 2c7c67a0d15016887e9c917fee0bfe34e9d0e131 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120879420) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9243: [FLINK-13335][sql-parser] Align the SQL CREATE TABLE DDL with FLIP-37
flinkbot edited a comment on issue #9243: [FLINK-13335][sql-parser] Align the SQL CREATE TABLE DDL with FLIP-37 URL: https://github.com/apache/flink/pull/9243#issuecomment-515486131 ## CI report: * ae2528ac686c28a60641e8ea295221a510043e76 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120870435) * a4c045ee56e7d8001f8393baeb841280d81eac1a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120875424) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough
flinkbot edited a comment on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough URL: https://github.com/apache/flink/pull/8903#issuecomment-510464651 ## CI report: * b2821a6ae97fd943f3a66b672e85fbd2374126c4 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/118909729) * 0699f7e5f2240a4a1bc44c15f08e6a1df47d3b01 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119054579) * 509e634257496dd2d8d42d512901f5eb46a82c50 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119406891) * 7579df06b6a0bf799e8a9c2bcb09984bf52c8e8c : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119441302) * ccb9dc29d4755d0a6c4596e08743b38615eb276a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120480063) * 1b976f30a689d9bdbf65513f034b2954bfb91468 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120494302) * 3ccee75dd0d506b90a2019cde9045eee26a4f4d5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120749125) * e07648c718b4ea32a3f02f826ca6a337400572be : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120769160) * cf082ff54f7bd160b9e0eb316459f419defdd0b7 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120872425) * b9b37f124c7c92c4d4b0c4e2101be33d9b86babd : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120877385) * b09bc12655a6bfac8d3deb83dac24bf20b954423 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120879446) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9022: [FLINK-13127] Fix --yarnship classpath
flinkbot edited a comment on issue #9022: [FLINK-13127] Fix --yarnship classpath URL: https://github.com/apache/flink/pull/9022#issuecomment-512166531 ## CI report: * 277d92daa0e31cc2fced052a1b311cdba0f567e1 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119436815) * e1cd4a223022fd9a130dec570e87ec5d161d3173 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120448921) * 82721288e321d0febe318b3dac72b941041211fe : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120873477) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9244: [FLINK-13396] Deprecate the BucketingSink.
flinkbot edited a comment on issue #9244: [FLINK-13396] Deprecate the BucketingSink. URL: https://github.com/apache/flink/pull/9244#issuecomment-515493269 ## CI report: * 8e3b712c5c2e003747737d2fb77752a3e10a3279 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120873450) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough
flinkbot edited a comment on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough URL: https://github.com/apache/flink/pull/8903#issuecomment-510464651 ## CI report: * b2821a6ae97fd943f3a66b672e85fbd2374126c4 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/118909729) * 0699f7e5f2240a4a1bc44c15f08e6a1df47d3b01 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119054579) * 509e634257496dd2d8d42d512901f5eb46a82c50 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119406891) * 7579df06b6a0bf799e8a9c2bcb09984bf52c8e8c : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119441302) * ccb9dc29d4755d0a6c4596e08743b38615eb276a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120480063) * 1b976f30a689d9bdbf65513f034b2954bfb91468 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120494302) * 3ccee75dd0d506b90a2019cde9045eee26a4f4d5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120749125) * e07648c718b4ea32a3f02f826ca6a337400572be : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120769160) * cf082ff54f7bd160b9e0eb316459f419defdd0b7 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120872425) * b9b37f124c7c92c4d4b0c4e2101be33d9b86babd : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120877385) * b09bc12655a6bfac8d3deb83dac24bf20b954423 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120879446) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-13278) add Chinese documentation for Hive source/sink
[
https://issues.apache.org/jira/browse/FLINK-13278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xuefu Zhang updated FLINK-13278:
Priority: Major (was: Blocker)
> add Chinese documentation for Hive source/sink
> --
>
> Key: FLINK-13278
> URL: https://issues.apache.org/jira/browse/FLINK-13278
> Project: Flink
> Issue Type: Sub-task
> Components: Connectors / Hive, Documentation
>Reporter: Bowen Li
>Priority: Major
> Fix For: 1.9.0, 1.10.0
>
>
> add Chinese documentation of Hive source/sink in {{batch/connector_zh.md}}
> it's corresponding English one is FLINK-13277
> cc [~xuefuz] [~lirui] [~Terry1897]
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (FLINK-13443) Support/Map Hive INTERVAL type in Hive connector
[ https://issues.apache.org/jira/browse/FLINK-13443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894012#comment-16894012 ] Xuefu Zhang commented on FLINK-13443: - cc: [~Terry1897], [~twalthr], [~phoenixjiangnan] > Support/Map Hive INTERVAL type in Hive connector > > > Key: FLINK-13443 > URL: https://issues.apache.org/jira/browse/FLINK-13443 > Project: Flink > Issue Type: Improvement > Components: Connectors / Hive, Table SQL / API >Affects Versions: 1.9.0 >Reporter: Xuefu Zhang >Priority: Major > Fix For: 1.10.0 > > > The issue comes up when visiting FLINK-13385. We need to figure out how we > map between Flink's INTERVAL type and Hive's. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (FLINK-13443) Support/Map Hive INTERVAL type in Hive connector
Xuefu Zhang created FLINK-13443: --- Summary: Support/Map Hive INTERVAL type in Hive connector Key: FLINK-13443 URL: https://issues.apache.org/jira/browse/FLINK-13443 Project: Flink Issue Type: Improvement Components: Connectors / Hive, Table SQL / API Affects Versions: 1.9.0 Reporter: Xuefu Zhang Fix For: 1.10.0 The issue comes up when visiting FLINK-13385. We need to figure out how we map between Flink's INTERVAL type and Hive's. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (FLINK-13438) Fix Hive connector with DataTypes.DATE/TIME/TIMESTAMP support
[ https://issues.apache.org/jira/browse/FLINK-13438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated FLINK-13438: Priority: Blocker (was: Major) > Fix Hive connector with DataTypes.DATE/TIME/TIMESTAMP support > - > > Key: FLINK-13438 > URL: https://issues.apache.org/jira/browse/FLINK-13438 > Project: Flink > Issue Type: Sub-task > Components: Connectors / Hive >Reporter: Caizhi Weng >Priority: Blocker > Fix For: 1.9.0, 1.10.0 > > > Similar to JDBC connectors, Hive connectors communicate with Flink framework > using TableSchema, which contains DataType. As the time data read from and > write to Hive connectors must be java.sql.* types and the default conversion > class of our time data types are java.time.*, we have to fix Hive connector > with DataTypes.DATE/TIME/TIMESTAMP support. > But currently when reading tables from Hive, the table schema is created > using Hive's schema, so the time types in the created schema will be sql time > type not local time type. If user specifies a local time type in the table > schema when creating a table in Hive, he will get a different schema when > reading it out. This is undesired. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (FLINK-13278) add Chinese documentation for Hive source/sink
[
https://issues.apache.org/jira/browse/FLINK-13278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xuefu Zhang updated FLINK-13278:
Priority: Blocker (was: Major)
> add Chinese documentation for Hive source/sink
> --
>
> Key: FLINK-13278
> URL: https://issues.apache.org/jira/browse/FLINK-13278
> Project: Flink
> Issue Type: Sub-task
> Components: Connectors / Hive, Documentation
>Reporter: Bowen Li
>Priority: Blocker
> Fix For: 1.9.0, 1.10.0
>
>
> add Chinese documentation of Hive source/sink in {{batch/connector_zh.md}}
> it's corresponding English one is FLINK-13277
> cc [~xuefuz] [~lirui] [~Terry1897]
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (FLINK-9017) FlinkKafkaProducer011ITCase.testFlinkKafkaProducer011FailBeforeNotify times out on Travis
[
https://issues.apache.org/jira/browse/FLINK-9017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894014#comment-16894014
]
Jiangjie Qin commented on FLINK-9017:
-
>From the stack trace, the issue seems on the test harness side - for some
>reason, the broker did not respond to the request sent by the clients and the
>clients were keeping retrying and eventually timed out. If it is not
>recurring, I'd suggest to close this with not reproducible.
> FlinkKafkaProducer011ITCase.testFlinkKafkaProducer011FailBeforeNotify times
> out on Travis
> -
>
> Key: FLINK-9017
> URL: https://issues.apache.org/jira/browse/FLINK-9017
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka, Tests
>Affects Versions: 1.5.0
>Reporter: Till Rohrmann
>Assignee: Jiangjie Qin
>Priority: Critical
> Labels: test-stability
> Fix For: 1.9.0
>
>
> {{FlinkKafkaProducer011ITCase.testFlinkKafkaProducer011FailBeforeNotify}}
> times out on Travis.
> https://api.travis-ci.org/v3/job/355025366/log.txt
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [flink] zhijiangW commented on issue #9132: [FLINK-13245][network] Fix the bug of resource leak while releasing partition and view reader
zhijiangW commented on issue #9132: [FLINK-13245][network] Fix the bug of resource leak while releasing partition and view reader URL: https://github.com/apache/flink/pull/9132#issuecomment-515536932 I adjusted the PR description based on the latest modifications, and all the commits would be squashed after approval finally and the commit content would be adjusted as well. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] StephanEwen commented on issue #9170: [FLINK-13325][test] Add test to guard the fix for FLINK-13249
StephanEwen commented on issue #9170: [FLINK-13325][test] Add test to guard the fix for FLINK-13249 URL: https://github.com/apache/flink/pull/9170#issuecomment-515535529 I validated that the test indeed fails without the original fix. Will address @zhijiangW's comments and merge this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9243: [FLINK-13335][sql-parser] Align the SQL CREATE TABLE DDL with FLIP-37
flinkbot edited a comment on issue #9243: [FLINK-13335][sql-parser] Align the SQL CREATE TABLE DDL with FLIP-37 URL: https://github.com/apache/flink/pull/9243#issuecomment-515486131 ## CI report: * ae2528ac686c28a60641e8ea295221a510043e76 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120870435) * a4c045ee56e7d8001f8393baeb841280d81eac1a : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120875424) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xuefuz commented on a change in pull request #9239: [FLINK-13385]Align Hive data type mapping with FLIP-37
xuefuz commented on a change in pull request #9239: [FLINK-13385]Align Hive data type mapping with FLIP-37 URL: https://github.com/apache/flink/pull/9239#discussion_r307836551 ## File path: docs/dev/table/catalog.md ## @@ -179,10 +179,11 @@ Currently `HiveCatalog` supports most Flink data types with the following mappin | TIMESTAMP_WITHOUT_TIME_ZONE | Timestamp | | TIMESTAMP_WITH_TIME_ZONE | N/A | | TIMESTAMP_WITH_LOCAL_TIME_ZONE | N/A | -| INTERVAL | N/A | -| BINARY| binary | -| VARBINARY(p) | binary | -| ARRAY\ | list\ | +| INTERVAL | N/A*** | +| BINARY| N/A | +| VARBINARY(p) | N/A | +| BYTES | BINARY | +| ARRAY\ | ARRAY\ | | MAP | map | Review comment: Let's add a note saying that Hive map key type only allows primitive types, while Flink map key can be any data type. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9242: [FLINK-13408][runtime] Schedule StandaloneResourceManager.setFailUnfulfillableRequest whenever the leadership is acquired
flinkbot edited a comment on issue #9242: [FLINK-13408][runtime] Schedule StandaloneResourceManager.setFailUnfulfillableRequest whenever the leadership is acquired URL: https://github.com/apache/flink/pull/9242#issuecomment-515479114 ## CI report: * bc4bc3a00e1692381368af36b87d233677e8c4ac : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120867383) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] xuefuz commented on a change in pull request #9239: [FLINK-13385]Align Hive data type mapping with FLIP-37
xuefuz commented on a change in pull request #9239: [FLINK-13385]Align Hive
data type mapping with FLIP-37
URL: https://github.com/apache/flink/pull/9239#discussion_r307830806
##
File path:
flink-connectors/flink-connector-hive/src/main/java/org/apache/flink/table/catalog/hive/util/HiveTypeUtil.java
##
@@ -146,6 +143,11 @@ public static TypeInfo toHiveTypeInfo(DataType dataType) {
// Flink's primitive types that Hive 2.3.4 doesn't
support: Time, TIMESTAMP_WITH_LOCAL_TIME_ZONE
}
+ if (dataType.equals(DataTypes.BYTES())) {
+ // Hive doesn't support variable-length binary string
Review comment:
The comment doesn't seem applicable in the code context. Let's remove it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xuefuz commented on a change in pull request #9239: [FLINK-13385]Align Hive data type mapping with FLIP-37
xuefuz commented on a change in pull request #9239: [FLINK-13385]Align Hive
data type mapping with FLIP-37
URL: https://github.com/apache/flink/pull/9239#discussion_r307832232
##
File path:
flink-connectors/flink-connector-hive/src/test/java/org/apache/flink/table/catalog/hive/HiveCatalogDataTypeTest.java
##
@@ -124,8 +122,8 @@ public void testDataTypes() throws Exception {
@Test
public void testNonExactlyMatchedDataTypes() throws Exception {
Review comment:
Maybe we should change this test to a negative test case, as BINARY and
VARBINARY are not supported by hive and it's useful to capture the limitation
with a test case.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xuefuz commented on a change in pull request #9239: [FLINK-13385]Align Hive data type mapping with FLIP-37
xuefuz commented on a change in pull request #9239: [FLINK-13385]Align Hive
data type mapping with FLIP-37
URL: https://github.com/apache/flink/pull/9239#discussion_r307832232
##
File path:
flink-connectors/flink-connector-hive/src/test/java/org/apache/flink/table/catalog/hive/HiveCatalogDataTypeTest.java
##
@@ -124,8 +122,8 @@ public void testDataTypes() throws Exception {
@Test
public void testNonExactlyMatchedDataTypes() throws Exception {
Review comment:
Maybe we should change this test to a negative test case, as BINARY and
VARBINARY are not supported by hive and it's useful to capture the limitation
with a test case.
Also, we may change the method name accordingly.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (FLINK-12396) KafkaITCase.testOneSourceMultiplePartitions doesn't fail properly
[
https://issues.apache.org/jira/browse/FLINK-12396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893972#comment-16893972
]
Jiangjie Qin commented on FLINK-12396:
--
This is probably related to FLINK-13226, which has a PR to fix at this point.
> KafkaITCase.testOneSourceMultiplePartitions doesn't fail properly
> -
>
> Key: FLINK-12396
> URL: https://issues.apache.org/jira/browse/FLINK-12396
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
>Affects Versions: 1.9.0
>Reporter: Bowen Li
>Priority: Major
> Fix For: 1.9.0
>
>
> https://api.travis-ci.org/v3/job/527599974/log.txt
> In the log, we can see that KafkaITCase.testOneSourceMultiplePartitions
> failed, but it kept running and doing all the snapshot, which caused the
> built to timeout.
> {code:java}
> 05:00:38,896 INFO
> org.apache.flink.streaming.connectors.kafka.testutils.ValidatingExactlyOnceSink
> - Snapshot of counter 4800 at checkpoint 115
> 05:00:39,050 ERROR org.apache.flink.streaming.connectors.kafka.KafkaITCase
>-
>
> Test
> testOneSourceMultiplePartitions(org.apache.flink.streaming.connectors.kafka.KafkaITCase)
> failed with:
> org.junit.runners.model.TestTimedOutException: test timed out after 6
> milliseconds
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1693)
> at
> java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
> at
> java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1729)
> at
> java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
> at
> org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:621)
> at
> org.apache.flink.streaming.util.TestStreamEnvironment.execute(TestStreamEnvironment.java:79)
> at org.apache.flink.test.util.TestUtils.tryExecute(TestUtils.java:35)
> at
> org.apache.flink.streaming.connectors.kafka.KafkaConsumerTestBase.runOneSourceMultiplePartitionsExactlyOnceTest(KafkaConsumerTestBase.java:924)
> at
> org.apache.flink.streaming.connectors.kafka.KafkaITCase.testOneSourceMultiplePartitions(KafkaITCase.java:102)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
>
> 05:00:39,057 INFO org.apache.flink.streaming.connectors.kafka.KafkaITCase
>-
>
> Test
> testCancelingFullTopic(org.apache.flink.streaming.connectors.kafka.KafkaITCase)
> is running.
>
> 05:00:39,396 INFO
> org.apache.flink.streaming.connectors.kafka.testutils.ValidatingExactlyOnceSink
> - Snapshot of counter 4800 at checkpoint 116
> 05:00:39,896 INFO
> org.apache.flink.streaming.connectors.kafka.testutils.ValidatingExactlyOnceSink
> - Snapshot of counter 4800 at checkpoint 117
> 05:00:40,396 INFO
> org.apache.flink.streaming.connectors.kafka.testutils.ValidatingExactlyOnceSink
> - Snapshot of counter 4800 at checkpoint 118
> 05:00:40,896 INFO
> org.apache.flink.streaming.connectors.kafka.testutils.ValidatingExactlyOnceSink
> - Snapshot of counter 4800 at checkpoint 119
> 05:00:41,396 INFO
> org.apache.flink.streaming.connectors.kafka.testutils.ValidatingExactlyOnceSink
> - Snapshot of counter 4800 at checkpoint 120
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [flink] tillrohrmann commented on a change in pull request #9242: [FLINK-13408][runtime] Schedule StandaloneResourceManager.setFailUnfulfillableRequest whenever the leadership is acquired
tillrohrmann commented on a change in pull request #9242: [FLINK-13408][runtime] Schedule StandaloneResourceManager.setFailUnfulfillableRequest whenever the leadership is acquired URL: https://github.com/apache/flink/pull/9242#discussion_r307811139 ## File path: flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/MesosResourceManager.java ## @@ -201,7 +201,6 @@ public MesosResourceManager( final ContaineredTaskManagerParameters containeredTaskManagerParameters = taskManagerParameters.containeredParameters(); this.slotsPerWorker = updateTaskManagerConfigAndCreateWorkerSlotProfiles( flinkConfig, containeredTaskManagerParameters.taskManagerTotalMemoryMB(), containeredTaskManagerParameters.numSlots()); - setFailUnfulfillableRequest(true); Review comment: This is a good finding. Don't we have tests for Mesos which test this behavior? Given the previous code, unfulfillable requests should have failed right away, right? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #9242: [FLINK-13408][runtime] Schedule StandaloneResourceManager.setFailUnfulfillableRequest whenever the leadership is acquired
tillrohrmann commented on a change in pull request #9242:
[FLINK-13408][runtime] Schedule
StandaloneResourceManager.setFailUnfulfillableRequest whenever the leadership
is acquired
URL: https://github.com/apache/flink/pull/9242#discussion_r307822497
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/resourcemanager/StandaloneResourceManager.java
##
@@ -112,4 +103,21 @@ protected ResourceID workerStarted(ResourceID resourceID)
{
return resourceID;
}
+ @Override
+ protected void setFailUnfulfillableRequest(boolean
failUnfulfillableRequest) {
+ if (failUnfulfillableRequest) {
+ final long startupPeriodMillis =
startupPeriodTime.toMilliseconds();
+
+ if (startupPeriodMillis > 0) {
+ getRpcService().getScheduledExecutor().schedule(
Review comment:
Let's not use this construct. I think it would be better to simply use
`RpcEndpoint#scheduleRunAsync`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #9242: [FLINK-13408][runtime] Schedule StandaloneResourceManager.setFailUnfulfillableRequest whenever the leadership is acquired
tillrohrmann commented on a change in pull request #9242:
[FLINK-13408][runtime] Schedule
StandaloneResourceManager.setFailUnfulfillableRequest whenever the leadership
is acquired
URL: https://github.com/apache/flink/pull/9242#discussion_r307821474
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/resourcemanager/StandaloneResourceManager.java
##
@@ -112,4 +103,21 @@ protected ResourceID workerStarted(ResourceID resourceID)
{
return resourceID;
}
+ @Override
+ protected void setFailUnfulfillableRequest(boolean
failUnfulfillableRequest) {
Review comment:
Let's not override the method but introduce another method which says that
you start/stop the timeout. For the non standalone implementations the method
should do nothing.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Created] (FLINK-13442) Remove unnecessary notifySubpartitionConsumed method from view reader
zhijiang created FLINK-13442: Summary: Remove unnecessary notifySubpartitionConsumed method from view reader Key: FLINK-13442 URL: https://issues.apache.org/jira/browse/FLINK-13442 Project: Flink Issue Type: Improvement Components: Runtime / Network Reporter: zhijiang Assignee: zhijiang Currently the methods of `NetworkSequenceViewReader#notifySubpartitionConsumed` and ` NetworkSequenceViewReader#releaseAllResources` would be called meanwhile in netty stack during releasing resources. To make this release logic simple and clean, we could remove the redundant `notifySubpartitionConsumed` from `NetworkSequenceViewReader` side, and also remove it from `ResultSubpartitionView` side. In the implementation of `ResultSubpartitionView#releaseAllResources` it would further notify the parent subpartition of consumed state via `ResultSubpartition#notifySubpartitionConsumed` which further feedback to parent `ResultPartition` layer via `onConsumedSubpartition`. Finally `ResultPartition` could decide whether to release itself or not. E.g. for the case of `ReleaseOnConsumptionResultPartition` which is mainly used for pipelined partition, it would release partition after reference counter decreased to 0. For the case of `ResultPartition` which would be generated for blocking partition by default, it would never be released after notifying consumed. And the JM/scheduler would decide when to release partition properly. In addition, `InputChannel#notifySubpartitionConsumed` could also be removed as a result of above. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [flink] tillrohrmann commented on a change in pull request #9242: [FLINK-13408][runtime] Schedule StandaloneResourceManager.setFailUnfulfillableRequest whenever the leadership is acquired
tillrohrmann commented on a change in pull request #9242:
[FLINK-13408][runtime] Schedule
StandaloneResourceManager.setFailUnfulfillableRequest whenever the leadership
is acquired
URL: https://github.com/apache/flink/pull/9242#discussion_r307812176
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/resourcemanager/StandaloneResourceManager.java
##
@@ -112,4 +103,21 @@ protected ResourceID workerStarted(ResourceID resourceID)
{
return resourceID;
}
+ @Override
+ protected void setFailUnfulfillableRequest(boolean
failUnfulfillableRequest) {
+ if (failUnfulfillableRequest) {
+ final long startupPeriodMillis =
startupPeriodTime.toMilliseconds();
+
+ if (startupPeriodMillis > 0) {
+ getRpcService().getScheduledExecutor().schedule(
+ () -> getMainThreadExecutor().execute(
+ () ->
super.setFailUnfulfillableRequest(failUnfulfillableRequest)),
+ startupPeriodMillis,
+ TimeUnit.MILLISECONDS
+ );
+ }
+ } else {
+
super.setFailUnfulfillableRequest(failUnfulfillableRequest);
Review comment:
Let's replace `failUnfillableRequest` with `false` here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9132: [FLINK-13245][network] Fix the bug of file resource leak while canceling partition request
flinkbot edited a comment on issue #9132: [FLINK-13245][network] Fix the bug of file resource leak while canceling partition request URL: https://github.com/apache/flink/pull/9132#issuecomment-511875353 ## CI report: * 3f40e06e867f0b661c62e8ba690156b3819135d5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119438178) * fa5f2235fbc084ed93df94b6e5866991973dcf3a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119441046) * 676e1c8c02f70aa8ec55f8eb927ad4160afc8d33 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119568230) * 7d698bd5eefdd41df42d15df67dc9c35123edf1d : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120853356) * dc808588547e073ff7f0147f35c7dcf72f199aa1 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120861514) * cbb909bb76a5151628337f02489a75b754f948d5 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120865225) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13436) Add TPC-H queries as E2E tests
[ https://issues.apache.org/jira/browse/FLINK-13436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893962#comment-16893962 ] Yu Li commented on FLINK-13436: --- Sorry but just noticed your reply [~xuefuz]. FWIW, I think using Hive data could save us from checking legal stuff around TPCH and enough for the correctness/system-stability checking, thus a better idea. And we could leave TPCH for sql performance testing just like spark did in the [https://github.com/databricks/spark-sql-perf] project. I'm not the expert in this area though (smile). > Add TPC-H queries as E2E tests > -- > > Key: FLINK-13436 > URL: https://issues.apache.org/jira/browse/FLINK-13436 > Project: Flink > Issue Type: Test > Components: Table SQL / Planner, Tests >Affects Versions: 1.9.0 >Reporter: Till Rohrmann >Priority: Blocker > Fix For: 1.9.0 > > > We should add the TPC-H queries as E2E tests in order to verify the blink > planner. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [flink] flinkbot edited a comment on issue #9132: [FLINK-13245][network] Fix the bug of file resource leak while canceling partition request
flinkbot edited a comment on issue #9132: [FLINK-13245][network] Fix the bug of file resource leak while canceling partition request URL: https://github.com/apache/flink/pull/9132#issuecomment-511875353 ## CI report: * 3f40e06e867f0b661c62e8ba690156b3819135d5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119438178) * fa5f2235fbc084ed93df94b6e5866991973dcf3a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119441046) * 676e1c8c02f70aa8ec55f8eb927ad4160afc8d33 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119568230) * 7d698bd5eefdd41df42d15df67dc9c35123edf1d : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120853356) * dc808588547e073ff7f0147f35c7dcf72f199aa1 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120861514) * cbb909bb76a5151628337f02489a75b754f948d5 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120865225) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13436) Add TPC-H queries as E2E tests
[
https://issues.apache.org/jira/browse/FLINK-13436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893957#comment-16893957
]
Yu Li commented on FLINK-13436:
---
After some offline discussion with [~till.rohrmann] and [~twalthr], we all
agree that it's safer to check the whole path from submitting sql query from
cli, running on a real cluster to returning the result, so making it end-to-end
test is necessary rather than adding an IT case.
Checking the initial patch for IT case (kind of big so won't upload it here), I
think we have the below TODO items to get the job done:
# Figure out how to generate the TPCH data and use it legally (prevent any
licensing issue).
** FWIW, we may refer to the https://github.com/databricks/tpch-dbgen repo.
# Write SQL commands for TPCH query.
# Add a script to 1) start a cluster, 2) invoke the data generator, 3) submit
the SQL query from cli, and 4) verifying the result. Refer to
{{flink-end-to-end-tests/test-scripts/test_batch_allround.sh}} as a sample.
I could help on the 3rd one but need some help on the other two due to lack of
background. [~lzljs3620320] [~TsReaper]
Thanks.
> Add TPC-H queries as E2E tests
> --
>
> Key: FLINK-13436
> URL: https://issues.apache.org/jira/browse/FLINK-13436
> Project: Flink
> Issue Type: Test
> Components: Table SQL / Planner, Tests
>Affects Versions: 1.9.0
>Reporter: Till Rohrmann
>Priority: Blocker
> Fix For: 1.9.0
>
>
> We should add the TPC-H queries as E2E tests in order to verify the blink
> planner.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [flink] xintongsong commented on a change in pull request #9105: [FLINK-13241][Yarn/Mesos] Fix Yarn/MesosResourceManager setting managed memory size into wrong configuration instance.
xintongsong commented on a change in pull request #9105: [FLINK-13241][Yarn/Mesos] Fix Yarn/MesosResourceManager setting managed memory size into wrong configuration instance. URL: https://github.com/apache/flink/pull/9105#discussion_r307812100 ## File path: flink-yarn/src/main/java/org/apache/flink/yarn/YarnResourceManager.java ## @@ -184,7 +183,9 @@ public YarnResourceManager( this.defaultCpus = flinkConfig.getInteger(YarnConfigOptions.VCORES, numberOfTaskSlots); this.resource = Resource.newInstance(defaultTaskManagerMemoryMB, defaultCpus); - this.slotsPerWorker = updateTaskManagerConfigAndCreateWorkerSlotProfiles(flinkConfig, defaultTaskManagerMemoryMB, numberOfTaskSlots); + this.defaultTaskManagerParameters = ContaineredTaskManagerParameters.create(flinkConfig, defaultTaskManagerMemoryMB, numberOfTaskSlots); Review comment: Yes, the `createTaskExecutorLaunchContext ` recreates a `ContaineredTaskManagerParameters`, but with the actual container memory allocated by Yarn, which could be slightly larger than requested. Reusing the same `ContaineredTaskManagerParameters` instance prevents task managers from making use of the extra allocated memory, which could be a user aware behavior change compare to the current behavior. I think maybe we should not introduce such behavior change in this version, since we are going to change resource configuration and management anyway in the next version. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9169: [FLINK-12998][docs] Update documentation for file systems loading as plugins
flinkbot edited a comment on issue #9169: [FLINK-12998][docs] Update documentation for file systems loading as plugins URL: https://github.com/apache/flink/pull/9169#issuecomment-512820058 ## CI report: * a7bfcd7a2605fe9299655cb26d3f829b4751ecd6 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119638448) * 8a0a1f2a8f13b1f8e55e50401fee5dbf0d8c1ed4 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119766161) * d3541a995150495d527b721053ef1a4077053908 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119795433) * d89f14eaa970b880a88238c8d66e71095534b868 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119799191) * 24b3afb25bad946bdb8c72965aba4833b25c46b1 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119806398) * 45178e4407304d4d8561c9d810d2ad67ad3e6a86 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120836774) * a46d68bc5d7932c2165a898d3e8e8b0a3bd5c1a3 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120837450) * 393f2e64a2ffcf06577af43099ee4dfe0cd097c1 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120852616) * c3a31748e4d8e8aeebf538bdf9a57baf9259fe4e : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120881311) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] StephanEwen commented on issue #9132: [FLINK-13245][network] Fix the bug of file resource leak while canceling partition request
StephanEwen commented on issue #9132: [FLINK-13245][network] Fix the bug of file resource leak while canceling partition request URL: https://github.com/apache/flink/pull/9132#issuecomment-515511582 From my side, +1 to commit this This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] tillrohrmann commented on issue #9105: [FLINK-13241][Yarn/Mesos] Fix Yarn/MesosResourceManager setting managed memory size into wrong configuration instance.
tillrohrmann commented on issue #9105: [FLINK-13241][Yarn/Mesos] Fix Yarn/MesosResourceManager setting managed memory size into wrong configuration instance. URL: https://github.com/apache/flink/pull/9105#issuecomment-515509225 @xintongsong please also open a PR against the `master` branch since we need to forward port this fix. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #9105: [FLINK-13241][Yarn/Mesos] Fix Yarn/MesosResourceManager setting managed memory size into wrong configuration instance.
tillrohrmann commented on a change in pull request #9105:
[FLINK-13241][Yarn/Mesos] Fix Yarn/MesosResourceManager setting managed memory
size into wrong configuration instance.
URL: https://github.com/apache/flink/pull/9105#discussion_r307803681
##
File path:
flink-yarn/src/main/java/org/apache/flink/yarn/YarnResourceManager.java
##
@@ -325,7 +326,12 @@ Resource getContainerResource() {
@VisibleForTesting
Collection getSlotsPerWorker() {
- return slotsPerWorker;
+ return
defaultTaskManagerParameters.getWorkerSlotResourceProfiles();
Review comment:
This method is no longer needed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #9105: [FLINK-13241][Yarn/Mesos] Fix Yarn/MesosResourceManager setting managed memory size into wrong configuration instance.
tillrohrmann commented on a change in pull request #9105:
[FLINK-13241][Yarn/Mesos] Fix Yarn/MesosResourceManager setting managed memory
size into wrong configuration instance.
URL: https://github.com/apache/flink/pull/9105#discussion_r307803611
##
File path:
flink-yarn/src/main/java/org/apache/flink/yarn/YarnResourceManager.java
##
@@ -325,7 +326,12 @@ Resource getContainerResource() {
@VisibleForTesting
Collection getSlotsPerWorker() {
- return slotsPerWorker;
+ return
defaultTaskManagerParameters.getWorkerSlotResourceProfiles();
+ }
+
+ @VisibleForTesting
+ Configuration getFlinkConfig() {
Review comment:
No longer needed
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #9105: [FLINK-13241][Yarn/Mesos] Fix Yarn/MesosResourceManager setting managed memory size into wrong configuration instance.
tillrohrmann commented on a change in pull request #9105: [FLINK-13241][Yarn/Mesos] Fix Yarn/MesosResourceManager setting managed memory size into wrong configuration instance. URL: https://github.com/apache/flink/pull/9105#discussion_r307806934 ## File path: flink-yarn/src/main/java/org/apache/flink/yarn/YarnResourceManager.java ## @@ -184,7 +183,9 @@ public YarnResourceManager( this.defaultCpus = flinkConfig.getInteger(YarnConfigOptions.VCORES, numberOfTaskSlots); this.resource = Resource.newInstance(defaultTaskManagerMemoryMB, defaultCpus); - this.slotsPerWorker = updateTaskManagerConfigAndCreateWorkerSlotProfiles(flinkConfig, defaultTaskManagerMemoryMB, numberOfTaskSlots); + this.defaultTaskManagerParameters = ContaineredTaskManagerParameters.create(flinkConfig, defaultTaskManagerMemoryMB, numberOfTaskSlots); + this.flinkConfig = defaultTaskManagerParameters.getTaskManagerConfig(); Review comment: The only reason for the `flinkConfig` is the call `HistoryServerUtils.getHistoryServerURL` for which we need a Flink configuration. We could create a follow up issue to pass in a `HistoryServerConfiguration` instead which contains all configuration values. That way, we could get rid of having to pass in a `Configuration` object. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] tillrohrmann commented on a change in pull request #9105: [FLINK-13241][Yarn/Mesos] Fix Yarn/MesosResourceManager setting managed memory size into wrong configuration instance.
tillrohrmann commented on a change in pull request #9105: [FLINK-13241][Yarn/Mesos] Fix Yarn/MesosResourceManager setting managed memory size into wrong configuration instance. URL: https://github.com/apache/flink/pull/9105#discussion_r307805904 ## File path: flink-yarn/src/main/java/org/apache/flink/yarn/YarnResourceManager.java ## @@ -184,7 +183,9 @@ public YarnResourceManager( this.defaultCpus = flinkConfig.getInteger(YarnConfigOptions.VCORES, numberOfTaskSlots); this.resource = Resource.newInstance(defaultTaskManagerMemoryMB, defaultCpus); - this.slotsPerWorker = updateTaskManagerConfigAndCreateWorkerSlotProfiles(flinkConfig, defaultTaskManagerMemoryMB, numberOfTaskSlots); + this.defaultTaskManagerParameters = ContaineredTaskManagerParameters.create(flinkConfig, defaultTaskManagerMemoryMB, numberOfTaskSlots); Review comment: I think this change isn't complete yet. The method `createTaskExecutorLaunchContext` recreates a `ContaineredTaskManagerParameters` instance, for example. I would suggest that we pass in a `ContaineredTaskManagerParameters` instance to the `YarnResourceManager` and reuse this instance within this class. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13436) Add TPC-H queries as E2E tests
[ https://issues.apache.org/jira/browse/FLINK-13436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893934#comment-16893934 ] Xuefu Zhang commented on FLINK-13436: - Maybe we can use Hive data for this, which can verify both blink planner and Hive. > Add TPC-H queries as E2E tests > -- > > Key: FLINK-13436 > URL: https://issues.apache.org/jira/browse/FLINK-13436 > Project: Flink > Issue Type: Test > Components: Table SQL / Planner, Tests >Affects Versions: 1.9.0 >Reporter: Till Rohrmann >Priority: Blocker > Fix For: 1.9.0 > > > We should add the TPC-H queries as E2E tests in order to verify the blink > planner. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [flink] flinkbot commented on issue #9245: [FLINK-13334][coordination] Remove legacy implementation of slot
flinkbot commented on issue #9245: [FLINK-13334][coordination] Remove legacy implementation of slot URL: https://github.com/apache/flink/pull/9245#issuecomment-515507399 ## CI report: * 2c7c67a0d15016887e9c917fee0bfe34e9d0e131 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120879420) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13437) Add Hive SQL E2E test
[ https://issues.apache.org/jira/browse/FLINK-13437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893932#comment-16893932 ] Xuefu Zhang commented on FLINK-13437: - There seems to be a JIRA (FLINK-13303) covering this. We are still finding resources to do the work. :) > Add Hive SQL E2E test > - > > Key: FLINK-13437 > URL: https://issues.apache.org/jira/browse/FLINK-13437 > Project: Flink > Issue Type: Test > Components: Connectors / Hive, Tests >Affects Versions: 1.9.0 >Reporter: Till Rohrmann >Priority: Blocker > Fix For: 1.9.0 > > > We should add an E2E test for the Hive integration: List all tables and read > some metadata, read from an existing table, register a new table in Hive, use > a registered function, write to an existing table, write to a new table. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [flink] flinkbot edited a comment on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough
flinkbot edited a comment on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough URL: https://github.com/apache/flink/pull/8903#issuecomment-510464651 ## CI report: * b2821a6ae97fd943f3a66b672e85fbd2374126c4 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/118909729) * 0699f7e5f2240a4a1bc44c15f08e6a1df47d3b01 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119054579) * 509e634257496dd2d8d42d512901f5eb46a82c50 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119406891) * 7579df06b6a0bf799e8a9c2bcb09984bf52c8e8c : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119441302) * ccb9dc29d4755d0a6c4596e08743b38615eb276a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120480063) * 1b976f30a689d9bdbf65513f034b2954bfb91468 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120494302) * 3ccee75dd0d506b90a2019cde9045eee26a4f4d5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120749125) * e07648c718b4ea32a3f02f826ca6a337400572be : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120769160) * cf082ff54f7bd160b9e0eb316459f419defdd0b7 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120872425) * b9b37f124c7c92c4d4b0c4e2101be33d9b86babd : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120877385) * b09bc12655a6bfac8d3deb83dac24bf20b954423 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120879446) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9245: [FLINK-13334][coordination] Remove legacy implementation of slot
flinkbot commented on issue #9245: [FLINK-13334][coordination] Remove legacy implementation of slot URL: https://github.com/apache/flink/pull/9245#issuecomment-515505238 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-13334) Remove legacy implementation of slot
[
https://issues.apache.org/jira/browse/FLINK-13334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-13334:
---
Labels: pull-request-available (was: )
> Remove legacy implementation of slot
>
>
> Key: FLINK-13334
> URL: https://issues.apache.org/jira/browse/FLINK-13334
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination
>Affects Versions: 1.10.0
>Reporter: TisonKun
>Assignee: TisonKun
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.10.0
>
>
> cc [~till.rohrmann] [~srichter]
> From my investigation currently Flink use {{SlotSharingManager}} and
> {{SlotSharingGroupId}} for achieving slot sharing. And thus
> {{SlotSharingGroupAssignment}} {{SlotSharingGroup}} and {{SharedSlot}} are
> all legacy concept.
> Notice that the ongoing scheduler re-design touches frequently tests based on
> legacy slot/instance logic or even uses it for testing. I'd like to nudge
> this process for totally remove legacy code from our code base.
> Also I attach a patch on FLINK-12179 that remove {{Instance}}. With current
> contribution workflow your shepherds are significant :- )
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [flink] walterddr commented on a change in pull request #9227: [FLINK-13347] [table-planner] should handle SEMI/ANTI JoinRelType in switch case
walterddr commented on a change in pull request #9227: [FLINK-13347] [table-planner] should handle SEMI/ANTI JoinRelType in switch case URL: https://github.com/apache/flink/pull/9227#discussion_r307800633 ## File path: flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/plan/nodes/datastream/DataStreamJoinToCoProcessTranslator.scala ## @@ -145,6 +145,7 @@ class DataStreamJoinToCoProcessTranslator( genFunction.name, genFunction.code, queryConfig) + case _ => throw new TableException(s"$joinType is not supported.") Review comment: I think this one should throw `ValidationException` similar to other Translator classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (FLINK-13334) Remove legacy implementation of slot
[
https://issues.apache.org/jira/browse/FLINK-13334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893922#comment-16893922
]
TisonKun edited comment on FLINK-13334 at 7/26/19 3:43 PM:
---
Change the purpose of this JIRA into remove the whole legacy slot implement
because legacy slot sharing is used in it and I find we can do it at once.
was (Author: tison):
Change the purpose of this JIRA into remove the whole legacy slot implement
because legacy slot sharing is used in it and I find we can do it in batch.
> Remove legacy implementation of slot
>
>
> Key: FLINK-13334
> URL: https://issues.apache.org/jira/browse/FLINK-13334
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination
>Affects Versions: 1.10.0
>Reporter: TisonKun
>Assignee: TisonKun
>Priority: Major
> Fix For: 1.10.0
>
>
> cc [~till.rohrmann] [~srichter]
> From my investigation currently Flink use {{SlotSharingManager}} and
> {{SlotSharingGroupId}} for achieving slot sharing. And thus
> {{SlotSharingGroupAssignment}} {{SlotSharingGroup}} and {{SharedSlot}} are
> all legacy concept.
> Notice that the ongoing scheduler re-design touches frequently tests based on
> legacy slot/instance logic or even uses it for testing. I'd like to nudge
> this process for totally remove legacy code from our code base.
> Also I attach a patch on FLINK-12179 that remove {{Instance}}. With current
> contribution workflow your shepherds are significant :- )
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (FLINK-13334) Remove legacy implementation of slot
[
https://issues.apache.org/jira/browse/FLINK-13334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893922#comment-16893922
]
TisonKun commented on FLINK-13334:
--
Change the purpose of this JIRA into remove the whole legacy slot implement
because legacy slot sharing is used in it and I find we can do it in batch.
> Remove legacy implementation of slot
>
>
> Key: FLINK-13334
> URL: https://issues.apache.org/jira/browse/FLINK-13334
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination
>Affects Versions: 1.10.0
>Reporter: TisonKun
>Assignee: TisonKun
>Priority: Major
> Fix For: 1.10.0
>
>
> cc [~till.rohrmann] [~srichter]
> From my investigation currently Flink use {{SlotSharingManager}} and
> {{SlotSharingGroupId}} for achieving slot sharing. And thus
> {{SlotSharingGroupAssignment}} {{SlotSharingGroup}} and {{SharedSlot}} are
> all legacy concept.
> Notice that the ongoing scheduler re-design touches frequently tests based on
> legacy slot/instance logic or even uses it for testing. I'd like to nudge
> this process for totally remove legacy code from our code base.
> Also I attach a patch on FLINK-12179 that remove {{Instance}}. With current
> contribution workflow your shepherds are significant :- )
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [flink] TisonKun opened a new pull request #9245: [FLINK-13334][coordination] Remove legacy implementation of slot
TisonKun opened a new pull request #9245: [FLINK-13334][coordination] Remove legacy implementation of slot URL: https://github.com/apache/flink/pull/9245 ## What is the purpose of the change Remove legacy implementation of slot ## Verifying this change This change is a trivial rework / code cleanup without any test coverage. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no) - The serializers: (no) - The runtime per-record code paths (performance sensitive): (no) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no) - The S3 file system connector: (no) ## Documentation - Does this pull request introduce a new feature? (no) - If yes, how is the feature documented? (not applicable) cc @tillrohrmann @zentol This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] sjwiesman edited a comment on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough
sjwiesman edited a comment on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough URL: https://github.com/apache/flink/pull/8903#issuecomment-515499428 Thanks for taking a look @fhueske, I believe I have addressed all of your points. In particular, I've added descriptions and sample output after each modification to the job. I also updated the log levels to make the job output easier to see. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough
flinkbot edited a comment on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough URL: https://github.com/apache/flink/pull/8903#issuecomment-510464651 ## CI report: * b2821a6ae97fd943f3a66b672e85fbd2374126c4 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/118909729) * 0699f7e5f2240a4a1bc44c15f08e6a1df47d3b01 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119054579) * 509e634257496dd2d8d42d512901f5eb46a82c50 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119406891) * 7579df06b6a0bf799e8a9c2bcb09984bf52c8e8c : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119441302) * ccb9dc29d4755d0a6c4596e08743b38615eb276a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120480063) * 1b976f30a689d9bdbf65513f034b2954bfb91468 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120494302) * 3ccee75dd0d506b90a2019cde9045eee26a4f4d5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120749125) * e07648c718b4ea32a3f02f826ca6a337400572be : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120769160) * cf082ff54f7bd160b9e0eb316459f419defdd0b7 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120872425) * b9b37f124c7c92c4d4b0c4e2101be33d9b86babd : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120877385) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-13334) Remove legacy implementation of slot
[
https://issues.apache.org/jira/browse/FLINK-13334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
TisonKun updated FLINK-13334:
-
Summary: Remove legacy implementation of slot (was: Remove legacy
implementation of slot sharing)
> Remove legacy implementation of slot
>
>
> Key: FLINK-13334
> URL: https://issues.apache.org/jira/browse/FLINK-13334
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination
>Affects Versions: 1.10.0
>Reporter: TisonKun
>Assignee: TisonKun
>Priority: Major
> Fix For: 1.10.0
>
>
> cc [~till.rohrmann] [~srichter]
> From my investigation currently Flink use {{SlotSharingManager}} and
> {{SlotSharingGroupId}} for achieving slot sharing. And thus
> {{SlotSharingGroupAssignment}} {{SlotSharingGroup}} and {{SharedSlot}} are
> all legacy concept.
> Notice that the ongoing scheduler re-design touches frequently tests based on
> legacy slot/instance logic or even uses it for testing. I'd like to nudge
> this process for totally remove legacy code from our code base.
> Also I attach a patch on FLINK-12179 that remove {{Instance}}. With current
> contribution workflow your shepherds are significant :- )
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (FLINK-13441) Add batch E2E test which runs with fewer slots than parallelism
Till Rohrmann created FLINK-13441:
-
Summary: Add batch E2E test which runs with fewer slots than
parallelism
Key: FLINK-13441
URL: https://issues.apache.org/jira/browse/FLINK-13441
Project: Flink
Issue Type: Test
Components: API / DataSet, Tests
Reporter: Till Rohrmann
Fix For: 1.9.0
We should adapt the existing batch E2E test to use the newly introduced
{{ScheduleMode#LAZY_FROM_SOURCES_WITH_BATCH_SLOT_REQUEST}} and verify that the
job runs on a cluster with fewer slots than the job's parallelism. In order to
make this work, we need to set the shuffles to be blocking via
{{ExecutionMode#BATCH}}. As a batch job we should use the
{{DataSetAllroundTestProgram}}.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (FLINK-9900) Failed to testRestoreBehaviourWithFaultyStateHandles (org.apache.flink.test.checkpointing.ZooKeeperHighAvailabilityITCase)
[ https://issues.apache.org/jira/browse/FLINK-9900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893912#comment-16893912 ] Till Rohrmann commented on FLINK-9900: -- Another instance: https://api.travis-ci.org/v3/job/563889124/log.txt > Failed to testRestoreBehaviourWithFaultyStateHandles > (org.apache.flink.test.checkpointing.ZooKeeperHighAvailabilityITCase) > --- > > Key: FLINK-9900 > URL: https://issues.apache.org/jira/browse/FLINK-9900 > Project: Flink > Issue Type: Bug > Components: Runtime / Coordination, Tests >Affects Versions: 1.5.1, 1.6.0, 1.9.0 >Reporter: zhangminglei >Assignee: Chesnay Schepler >Priority: Critical > Labels: pull-request-available, test-stability > Fix For: 1.9.0 > > > https://api.travis-ci.org/v3/job/405843617/log.txt > Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 124.598 sec > <<< FAILURE! - in > org.apache.flink.test.checkpointing.ZooKeeperHighAvailabilityITCase > > testRestoreBehaviourWithFaultyStateHandles(org.apache.flink.test.checkpointing.ZooKeeperHighAvailabilityITCase) > Time elapsed: 120.036 sec <<< ERROR! > org.junit.runners.model.TestTimedOutException: test timed out after 12 > milliseconds > at sun.misc.Unsafe.park(Native Method) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1693) > at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323) > at > java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1729) > at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895) > at > org.apache.flink.test.checkpointing.ZooKeeperHighAvailabilityITCase.testRestoreBehaviourWithFaultyStateHandles(ZooKeeperHighAvailabilityITCase.java:244) > Results : > Tests in error: > > ZooKeeperHighAvailabilityITCase.testRestoreBehaviourWithFaultyStateHandles:244 > » TestTimedOut > Tests run: 1453, Failures: 0, Errors: 1, Skipped: 29 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [flink] sjwiesman commented on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough
sjwiesman commented on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough URL: https://github.com/apache/flink/pull/8903#issuecomment-515499428 Thanks for taking a look @fhueske, I believe I have addressed all of your points. In particular, I've added descriptions and sample output after each modification to the job. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9243: [FLINK-13335][sql-parser] Align the SQL CREATE TABLE DDL with FLIP-37
flinkbot edited a comment on issue #9243: [FLINK-13335][sql-parser] Align the SQL CREATE TABLE DDL with FLIP-37 URL: https://github.com/apache/flink/pull/9243#issuecomment-515486131 ## CI report: * ae2528ac686c28a60641e8ea295221a510043e76 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120870435) * a4c045ee56e7d8001f8393baeb841280d81eac1a : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120875424) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9241: [FLINK-13429][table-common] Fix BoundedOutOfOrderTimestamps watermark strategy
flinkbot edited a comment on issue #9241: [FLINK-13429][table-common] Fix BoundedOutOfOrderTimestamps watermark strategy URL: https://github.com/apache/flink/pull/9241#issuecomment-515451295 ## CI report: * 0e7253d2000a50c620e1f889242dd88225cac335 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120856752) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13384) The back pressure monitoring does not work for StreamSources
[
https://issues.apache.org/jira/browse/FLINK-13384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893897#comment-16893897
]
Gary Yao commented on FLINK-13384:
--
Currently, we sample the stacktrace in
[TaskStackTraceSampleableTaskAdapter|https://github.com/apache/flink/blob/bb82aacf46565968359f5ee6d2503c3293d71d6e/flink-runtime/src/main/java/org/apache/flink/runtime/taskexecutor/TaskStackTraceSampleableTaskAdapter.java#L50].
As a quick fix, I see the following options:
* We could somehow expose the {{LegacySourceFunctionThread}} instance from
{{Task}} so that we sample the correct thread in
{{TaskStackTraceSampleableTaskAdapter}}.
* Using
[ThreadGroup|https://docs.oracle.com/javase/7/docs/api/java/lang/ThreadGroup.html],
we could enumerate (all) threads in the JVM and find the correct thread to
sample from.
Maybe [~pnowojski] can chime in here.
> The back pressure monitoring does not work for StreamSources
>
>
> Key: FLINK-13384
> URL: https://issues.apache.org/jira/browse/FLINK-13384
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Metrics, Runtime / Task
>Affects Versions: 1.9.0
>Reporter: Dawid Wysakowicz
>Assignee: Gary Yao
>Priority: Blocker
> Fix For: 1.9.0
>
>
> I think it is caused by: [FLINK-12483]. The reason is that the
> {{BackPressureStatsTrackerImpl}} samples only the main thread. FLINK-12483
> introduced a separate thread for executing the sources in the mailbox model.
> It is similar to other old bug that concerned only Kafka source: [FLINK-3456]
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [flink] flinkbot edited a comment on issue #9022: [FLINK-13127] Fix --yarnship classpath
flinkbot edited a comment on issue #9022: [FLINK-13127] Fix --yarnship classpath URL: https://github.com/apache/flink/pull/9022#issuecomment-512166531 ## CI report: * 277d92daa0e31cc2fced052a1b311cdba0f567e1 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119436815) * e1cd4a223022fd9a130dec570e87ec5d161d3173 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120448921) * 82721288e321d0febe318b3dac72b941041211fe : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120873477) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9244: [FLINK-13396] Deprecate the BucketingSink.
flinkbot commented on issue #9244: [FLINK-13396] Deprecate the BucketingSink. URL: https://github.com/apache/flink/pull/9244#issuecomment-515493269 ## CI report: * 8e3b712c5c2e003747737d2fb77752a3e10a3279 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120873450) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9240: [FLINK-13440] Add test that fails job when in-flight sync savepoint is discarded.
flinkbot edited a comment on issue #9240: [FLINK-13440] Add test that fails job when in-flight sync savepoint is discarded. URL: https://github.com/apache/flink/pull/9240#issuecomment-515444385 ## CI report: * 183d83b2c0abf0804c8ce3b68c3546cd05b46741 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120854235) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-12858) Potential distributed deadlock in case of synchronous savepoint failure
[
https://issues.apache.org/jira/browse/FLINK-12858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893881#comment-16893881
]
Kostas Kloudas commented on FLINK-12858:
The test is ready and there is a PR for it here
https://github.com/apache/flink/pull/9240
> Potential distributed deadlock in case of synchronous savepoint failure
> ---
>
> Key: FLINK-12858
> URL: https://issues.apache.org/jira/browse/FLINK-12858
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing
>Affects Versions: 1.9.0
>Reporter: Alex
>Assignee: Alex
>Priority: Blocker
> Fix For: 1.9.0
>
>
> Current implementation of stop-with-savepoint (FLINK-11458) would lock the
> thread (on {{syncSavepointLatch}}) that carries
> {{StreamTask.performCheckpoint()}}. For non-source tasks, this thread is
> implied to be the task's main thread (stop-with-savepoint deliberately stops
> any activity in the task's main thread).
> Unlocking happens either when the task is cancelled or when the corresponding
> checkpoint is acknowledged.
> It's possible, that other downstream tasks of the same Flink job "soft" fail
> the checkpoint/savepoint due to various reasons (for example, due to max
> buffered bytes {{BarrierBuffer.checkSizeLimit()}}. In such case, the
> checkpoint abortion would be notified to JM . But it looks like, the
> checkpoint coordinator would handle such abortion as usual and assume that
> the Flink job continues running.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [flink] flinkbot commented on issue #9244: [FLINK-13396] Deprecate the BucketingSink.
flinkbot commented on issue #9244: [FLINK-13396] Deprecate the BucketingSink. URL: https://github.com/apache/flink/pull/9244#issuecomment-515489951 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough
flinkbot edited a comment on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough URL: https://github.com/apache/flink/pull/8903#issuecomment-510464651 ## CI report: * b2821a6ae97fd943f3a66b672e85fbd2374126c4 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/118909729) * 0699f7e5f2240a4a1bc44c15f08e6a1df47d3b01 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119054579) * 509e634257496dd2d8d42d512901f5eb46a82c50 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119406891) * 7579df06b6a0bf799e8a9c2bcb09984bf52c8e8c : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119441302) * ccb9dc29d4755d0a6c4596e08743b38615eb276a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120480063) * 1b976f30a689d9bdbf65513f034b2954bfb91468 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120494302) * 3ccee75dd0d506b90a2019cde9045eee26a4f4d5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120749125) * e07648c718b4ea32a3f02f826ca6a337400572be : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/120769160) * cf082ff54f7bd160b9e0eb316459f419defdd0b7 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120872425) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] dmvk commented on issue #9022: [FLINK-13127] Fix --yarnship classpath
dmvk commented on issue #9022: [FLINK-13127] Fix --yarnship classpath URL: https://github.com/apache/flink/pull/9022#issuecomment-515490263 @xintongsong done ;) thanks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-13396) Deprecate BucketingSink
[
https://issues.apache.org/jira/browse/FLINK-13396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-13396:
---
Labels: pull-request-available (was: )
> Deprecate BucketingSink
> ---
>
> Key: FLINK-13396
> URL: https://issues.apache.org/jira/browse/FLINK-13396
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / FileSystem
>Reporter: Aljoscha Krettek
>Assignee: Kostas Kloudas
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.9.0
>
>
> This has been superseded by the new {{StreamingFileSink}} and should be
> removed at some point in the future. For this we need to deprecate it first.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [flink] kl0u opened a new pull request #9244: [FLINK-13396] Deprecate the BucketingSink.
kl0u opened a new pull request #9244: [FLINK-13396] Deprecate the BucketingSink. URL: https://github.com/apache/flink/pull/9244 ## What is the purpose of the change It marks the `BucketingSink` as deprecated in favour of the `StreamingFileSink` and updates the documentation accordingly. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / **no**) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**) - The serializers: (yes / **no** / don't know) - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / **no** / don't know) - The S3 file system connector: (yes / **no** / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / **no**) - If yes, how is the feature documented? (not applicable / **docs** / **JavaDocs** / not documented) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] zhijiangW commented on a change in pull request #9132: [FLINK-13245][network] Fix the bug of file resource leak while canceling partition request
zhijiangW commented on a change in pull request #9132: [FLINK-13245][network]
Fix the bug of file resource leak while canceling partition request
URL: https://github.com/apache/flink/pull/9132#discussion_r307782071
##
File path:
flink-runtime/src/test/java/org/apache/flink/runtime/io/network/partition/PartitionTestUtils.java
##
@@ -158,4 +159,11 @@ public ResultPartitionID getResultPartitionID() {
true,
releaseType);
}
+
+ public static void writeBuffers(ResultPartition partition, int
numberOfBuffers, int bufferSize) throws IOException {
+ for (int i = 0; i < numberOfBuffers; i++) {
+
partition.addBufferConsumer(createFilledBufferConsumer(bufferSize, bufferSize),
0);
+ }
+ partition.finish();
+ }
Review comment:
In addition, these three tests are focusing on different classes actually,
which are `ResultPartition`, `ResultSubpartition` and `BoundedData`, so they
are not totally duplications. If we want to unify them, we have to also
refactor the relevant unit tests. I do not think it is necessary to do so now.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9243: [FLINK-13335][sql-parser] Align the SQL CREATE TABLE DDL with FLIP-37
flinkbot commented on issue #9243: [FLINK-13335][sql-parser] Align the SQL CREATE TABLE DDL with FLIP-37 URL: https://github.com/apache/flink/pull/9243#issuecomment-515486131 ## CI report: * ae2528ac686c28a60641e8ea295221a510043e76 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120870435) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9132: [FLINK-13245][network] Fix the bug of file resource leak while canceling partition request
flinkbot edited a comment on issue #9132: [FLINK-13245][network] Fix the bug of file resource leak while canceling partition request URL: https://github.com/apache/flink/pull/9132#issuecomment-511875353 ## CI report: * 3f40e06e867f0b661c62e8ba690156b3819135d5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119438178) * fa5f2235fbc084ed93df94b6e5866991973dcf3a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119441046) * 676e1c8c02f70aa8ec55f8eb927ad4160afc8d33 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/119568230) * 7d698bd5eefdd41df42d15df67dc9c35123edf1d : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/120853356) * dc808588547e073ff7f0147f35c7dcf72f199aa1 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120861514) * cbb909bb76a5151628337f02489a75b754f948d5 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/120865225) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9243: [FLINK-13335][sql-parser] Align the SQL CREATE TABLE DDL with FLIP-37
flinkbot commented on issue #9243: [FLINK-13335][sql-parser] Align the SQL CREATE TABLE DDL with FLIP-37 URL: https://github.com/apache/flink/pull/9243#issuecomment-515483799 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-13335) Align the SQL CREATE TABLE DDL with FLIP-37
[ https://issues.apache.org/jira/browse/FLINK-13335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-13335: --- Labels: pull-request-available (was: ) > Align the SQL CREATE TABLE DDL with FLIP-37 > --- > > Key: FLINK-13335 > URL: https://issues.apache.org/jira/browse/FLINK-13335 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / API >Reporter: Timo Walther >Assignee: Danny Chan >Priority: Blocker > Labels: pull-request-available > Fix For: 1.9.0 > > > At a first glance it does not seem that the newly introduced DDL is compliant > with FLIP-37. We should ensure consistent behavior esp. also for corner cases. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [flink] danny0405 opened a new pull request #9243: [FLINK-13335][sql-parser] Align the SQL CREATE TABLE DDL with FLIP-37
danny0405 opened a new pull request #9243: [FLINK-13335][sql-parser] Align the
SQL CREATE TABLE DDL with FLIP-37
URL: https://github.com/apache/flink/pull/9243
## What is the purpose of the change
Support parse all the data types defined in FLINK-37
- STRING
- BYTES
- TIME WITHOUT TIME ZONE
- TIME(3) WITHOUT TIME ZONE
- TIMESTAMP WITHOUT TIME ZONE
- TIMESTAMP(3) WITHOUT TIME ZONE
- TIMESTAMP WITH TIME ZONE (throws error)
- TIMESTAMP(3) WITH TIME ZONE (throws error)
- TIMESTAMP WITH LOCAL TIME ZONE
- TIMESTAMP(3) WITH LOCAL TIME ZONE
- INT NOT NULL ARRAY
- ROW(f0 INT NOT NULL, f1 BOOLEAN)
- ROW
- ANY('unknown.class', '') (throws error)
- INT MULTISET
- MULTISET
- INT NOT NULL ARRAY
- INT NOT NULL MULTISET
- ROW()
## Brief change log
- Add new data types in FLINK-37
## Verifying this change
See tests in FlinkDDLDataTypeTest.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? yes
- If yes, how is the feature documented? no
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] xintongsong commented on issue #9022: [FLINK-13127] Fix --yarnship classpath
xintongsong commented on issue #9022: [FLINK-13127] Fix --yarnship classpath URL: https://github.com/apache/flink/pull/9022#issuecomment-515482406 Maybe also rebase to the latest master? There is a travis test failure, which is not caused by this PR and should been fixed on master branch. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] twalthr commented on a change in pull request #9219: [FLINK-13404] [table] Port csv factories and descriptors to flink-csv
twalthr commented on a change in pull request #9219: [FLINK-13404] [table] Port csv factories and descriptors to flink-csv URL: https://github.com/apache/flink/pull/9219#discussion_r307774075 ## File path: flink-formats/flink-csv/src/main/java/org/apache/flink/table/descriptors/OldCsv.java ## @@ -0,0 +1,193 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.table.descriptors; + +import org.apache.flink.api.common.typeinfo.TypeInformation; +import org.apache.flink.table.api.TableSchema; +import org.apache.flink.table.api.ValidationException; +import org.apache.flink.table.utils.TypeStringUtils; + +import java.util.Arrays; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.Optional; +import java.util.stream.Collectors; + +import static org.apache.flink.table.descriptors.OldCsvValidator.FORMAT_COMMENT_PREFIX; +import static org.apache.flink.table.descriptors.OldCsvValidator.FORMAT_FIELDS; +import static org.apache.flink.table.descriptors.OldCsvValidator.FORMAT_FIELD_DELIMITER; +import static org.apache.flink.table.descriptors.OldCsvValidator.FORMAT_IGNORE_FIRST_LINE; +import static org.apache.flink.table.descriptors.OldCsvValidator.FORMAT_IGNORE_PARSE_ERRORS; +import static org.apache.flink.table.descriptors.OldCsvValidator.FORMAT_LINE_DELIMITER; +import static org.apache.flink.table.descriptors.OldCsvValidator.FORMAT_QUOTE_CHARACTER; +import static org.apache.flink.table.descriptors.OldCsvValidator.FORMAT_TYPE_VALUE; + +/** + * Format descriptor for comma-separated values (CSV). + * + * Note: This descriptor describes Flink's non-standard CSV table source/sink. In the future, the + * descriptor will be replaced by a proper RFC-compliant version. Use the RFC-compliant `Csv` + * format in the dedicated `flink-formats/flink-csv` module instead when writing to Kafka. Use the Review comment: update comment and link to new csv descriptor This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] twalthr commented on a change in pull request #9219: [FLINK-13404] [table] Port csv factories and descriptors to flink-csv
twalthr commented on a change in pull request #9219: [FLINK-13404] [table] Port
csv factories and descriptors to flink-csv
URL: https://github.com/apache/flink/pull/9219#discussion_r307774266
##
File path:
flink-formats/flink-csv/src/main/java/org/apache/flink/table/descriptors/OldCsvValidator.java
##
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.descriptors;
+
+/**
+ * Validator for {@link OldCsv}.
+ *
+ * @deprecated Use the RFC-compliant `Csv` format in the dedicated
Review comment:
update comment
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] twalthr commented on a change in pull request #9219: [FLINK-13404] [table] Port csv factories and descriptors to flink-csv
twalthr commented on a change in pull request #9219: [FLINK-13404] [table] Port
csv factories and descriptors to flink-csv
URL: https://github.com/apache/flink/pull/9219#discussion_r307773251
##
File path: flink-formats/flink-csv/pom.xml
##
@@ -58,31 +58,24 @@ under the License.
provided
-
-
-
org.apache.flink
- flink-table-common
+
flink-table-api-java-bridge_${scala.binary.version}
Review comment:
We cannot do this because it would introduce a Scala suffix to the module.
So far we could keep Scala out of the new format module for non-testing scopes.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
