[GitHub] [flink] flinkbot edited a comment on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese
flinkbot edited a comment on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese URL: https://github.com/apache/flink/pull/9648#issuecomment-529346361 ## CI report: * 6def0e035c5e7f442e45a00f1f7628726aef42eb : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/126406386) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-10837) Kafka 2.0 test KafkaITCase.testOneToOneSources dead locked on deleteTestTopic?
[
https://issues.apache.org/jira/browse/FLINK-10837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925526#comment-16925526
]
Yun Tang commented on FLINK-10837:
--
I come across this problem in instance
[https://api.travis-ci.com/v3/job/230376914/log.txt] although failed at
KafkaITCase.testTimestamps but stuck at KafkaTestBase.deleteTestTopic.
> Kafka 2.0 test KafkaITCase.testOneToOneSources dead locked on deleteTestTopic?
> --
>
> Key: FLINK-10837
> URL: https://issues.apache.org/jira/browse/FLINK-10837
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
>Affects Versions: 1.7.0, 1.8.0, 1.9.0
>Reporter: Piotr Nowojski
>Priority: Major
> Fix For: 1.7.0
>
>
> https://api.travis-ci.org/v3/job/452439034/log.txt
> {noformat}
> Tests in error:
>
> KafkaITCase.testOneToOneSources:97->KafkaConsumerTestBase.runOneToOneExactlyOnceTest:875->KafkaTestBase.deleteTestTopic:206->Object.wait:502->Object.wait:-2
> » TestTimedOut
> {noformat}
> {noformat}
> java.lang.InterruptedException
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:502)
> at
> org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:92)
> at
> org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:262)
> at
> org.apache.flink.streaming.connectors.kafka.KafkaTestEnvironmentImpl.deleteTestTopic(KafkaTestEnvironmentImpl.java:158)
> at
> org.apache.flink.streaming.connectors.kafka.KafkaTestBase.deleteTestTopic(KafkaTestBase.java:206)
> at
> org.apache.flink.streaming.connectors.kafka.KafkaConsumerTestBase.runOneToOneExactlyOnceTest(KafkaConsumerTestBase.java:875)
> at
> org.apache.flink.streaming.connectors.kafka.KafkaITCase.testOneToOneSources(KafkaITCase.java:97)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface
flinkbot edited a comment on issue #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface URL: https://github.com/apache/flink/pull/9604#issuecomment-527447402 ## CI report: * 66ade45fec93a56dde7609dffa0ad9dd241d0713 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125596416) * 51c44ece086b9d8fb8f420f783b3cf9af404f3b6 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/126417487) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9649: [FLINK-13994][docs-zh]Translate 'Getting Started' page to chinese
flinkbot edited a comment on issue #9649: [FLINK-13994][docs-zh]Translate 'Getting Started' page to chinese URL: https://github.com/apache/flink/pull/9649#issuecomment-529370466 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit fb2dd86ea8f17d546f026f779625813b1dd79e9a (Mon Sep 09 09:35:03 UTC 2019) ✅no warnings Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] gaofeilong198810 commented on issue #9649: [FLINK-13994][docs-zh]Translate 'Getting Started' page to chinese
gaofeilong198810 commented on issue #9649: [FLINK-13994][docs-zh]Translate 'Getting Started' page to chinese URL: https://github.com/apache/flink/pull/9649#issuecomment-529383906 I've add some description, please have a check, thx This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13958) Job class loader may not be reused after batch job recovery
[ https://issues.apache.org/jira/browse/FLINK-13958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925516#comment-16925516 ] Stephan Ewen commented on FLINK-13958: -- We had a similar problem initially with RocksDB. The strange thing is that the JVM can load a library once under a given file name only, but multiple times under different file names. So in RocksDB, we rename the library file to something random and then it can be linked multiple times. > Job class loader may not be reused after batch job recovery > --- > > Key: FLINK-13958 > URL: https://issues.apache.org/jira/browse/FLINK-13958 > Project: Flink > Issue Type: Bug > Components: Runtime / Task >Affects Versions: 1.9.0 >Reporter: David Moravek >Priority: Major > > [https://lists.apache.org/thread.html/e241be9a1a10810a1203786dff3b7386d265fbe8702815a77bad42eb@%3Cdev.flink.apache.org%3E|http://example.com] > 1) We have a per-job flink cluster > 2) We use BATCH execution mode + region failover strategy > Point 1) should imply single user code class loader per task manager (because > there is only single pipeline, that reuses class loader cached in > BlobLibraryCacheManager). We need this property, because we have UDFs that > access C libraries using JNI (I think this may be fairly common use-case when > dealing with legacy code). [JDK > internals|https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/java/lang/ClassLoader.java#L2466] > make sure that single library can be only loaded by a single class loader > per JVM. > When region recovery is triggered, vertices that need recover are first reset > back to CREATED stated and then rescheduled. In case all tasks in a task > manager are reset, this results in [cached class loader being > released|https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/execution/librarycache/BlobLibraryCacheManager.java#L338]. > This unfortunately causes job failure, because we try to reload a native > library in a newly created class loader. > I believe the correct approach would be not to release cached class loader if > the job is recovering, even though there are no tasks currently registered > with TM. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator
flinkbot edited a comment on issue #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator URL: https://github.com/apache/flink/pull/9564#issuecomment-526484775 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 16bac68a7fd82876d0ad2d56c8e7c199dc816d04 (Mon Sep 09 09:29:58 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-12958).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] 1u0 commented on a change in pull request #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator
1u0 commented on a change in pull request #9564:
[FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox
for timer triggers, checkpoints and AsyncWaitOperator
URL: https://github.com/apache/flink/pull/9564#discussion_r322145045
##
File path:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/async/AsyncWaitOperatorFactory.java
##
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.streaming.api.operators.async;
+
+import org.apache.flink.streaming.api.datastream.AsyncDataStream;
+import org.apache.flink.streaming.api.functions.async.AsyncFunction;
+import org.apache.flink.streaming.api.graph.StreamConfig;
+import org.apache.flink.streaming.api.operators.ChainingStrategy;
+import org.apache.flink.streaming.api.operators.OneInputStreamOperatorFactory;
+import org.apache.flink.streaming.api.operators.Output;
+import org.apache.flink.streaming.api.operators.StreamOperator;
+import org.apache.flink.streaming.api.operators.YieldingOperatorFactory;
+import org.apache.flink.streaming.runtime.tasks.StreamTask;
+import
org.apache.flink.streaming.runtime.tasks.mailbox.execution.MailboxExecutor;
+
+/**
+ * The factory of {@link AsyncWaitOperator}.
+ *
+ * @param The output type of the operator
+ */
+public class AsyncWaitOperatorFactory implements
OneInputStreamOperatorFactory, YieldingOperatorFactory {
+ private final AsyncFunction asyncFunction;
+ private final long timeout;
+ private final int capacity;
+ private final AsyncDataStream.OutputMode outputMode;
+ private MailboxExecutor mailboxExecutor;
+ private ChainingStrategy strategy = ChainingStrategy.HEAD;
+
+ public AsyncWaitOperatorFactory(
+ AsyncFunction asyncFunction,
+ long timeout,
+ int capacity,
+ AsyncDataStream.OutputMode outputMode) {
+ this.asyncFunction = asyncFunction;
+ this.timeout = timeout;
+ this.capacity = capacity;
+ this.outputMode = outputMode;
+ }
+
+ @Override public void setMailboxExecutor(MailboxExecutor
mailboxExecutor) {
+ this.mailboxExecutor = mailboxExecutor;
+ }
+
+ @Override public StreamOperator createStreamOperator(StreamTask
containingTask, StreamConfig config,
+ Output output) {
+ AsyncWaitOperator asyncWaitOperator = new AsyncWaitOperator(
+ asyncFunction,
+ timeout,
+ capacity,
+ outputMode,
+ mailboxExecutor);
+ asyncWaitOperator.setup(containingTask, config, output);
Review comment:
The created ticket:
[FLINK-14011](https://issues.apache.org/jira/browse/FLINK-14011).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9649: [FLINK-13994][docs-zh]Translate 'Getting Started' page
flinkbot edited a comment on issue #9649: [FLINK-13994][docs-zh]Translate 'Getting Started' page URL: https://github.com/apache/flink/pull/9649#issuecomment-529372783 ## CI report: * fb2dd86ea8f17d546f026f779625813b1dd79e9a : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/126415066) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface
flinkbot edited a comment on issue #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface URL: https://github.com/apache/flink/pull/9604#issuecomment-527447402 ## CI report: * 66ade45fec93a56dde7609dffa0ad9dd241d0713 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/125596416) * 51c44ece086b9d8fb8f420f783b3cf9af404f3b6 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-14011) Make some fields final and initialize them during construction in AsyncWaitOperator
[
https://issues.apache.org/jira/browse/FLINK-14011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925509#comment-16925509
]
Alex commented on FLINK-14011:
--
CC [~pnowojski].
> Make some fields final and initialize them during construction in
> AsyncWaitOperator
> ---
>
> Key: FLINK-14011
> URL: https://issues.apache.org/jira/browse/FLINK-14011
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Task
>Reporter: Alex
>Priority: Minor
>
> This is a small follow up ticket after the FLINK-12958.
> With the changes introduced there, the {{AsyncWaitOperator}} is created by
> {{AsyncWaitOperatorFactory}}, so some fields that initialized in the
> {{setup}} method can be setup in the constructor instead.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Created] (FLINK-14011) Make some fields final and initialize them during construction in AsyncWaitOperator
Alex created FLINK-14011:
Summary: Make some fields final and initialize them during
construction in AsyncWaitOperator
Key: FLINK-14011
URL: https://issues.apache.org/jira/browse/FLINK-14011
Project: Flink
Issue Type: Improvement
Components: Runtime / Task
Reporter: Alex
This is a small follow up ticket after the FLINK-12958.
With the changes introduced there, the {{AsyncWaitOperator}} is created by
{{AsyncWaitOperatorFactory}}, so some fields that initialized in the {{setup}}
method can be setup in the constructor instead.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface
flinkbot edited a comment on issue #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface URL: https://github.com/apache/flink/pull/9604#issuecomment-527441976 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 51c44ece086b9d8fb8f420f783b3cf9af404f3b6 (Mon Sep 09 09:10:35 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❗ 3. Needs [attention] from. - Needs attention by @aljoscha [PMC] * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (FLINK-13953) Facilitate enabling new Scheduler in MiniCluster Tests
[
https://issues.apache.org/jira/browse/FLINK-13953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925457#comment-16925457
]
Zhu Zhu edited comment on FLINK-13953 at 9/9/19 9:08 AM:
-
Hi [~gjy], I did some investigation on how to get different MiniCluster test
methods to run multiple times for different schedulers, so that we can do all
the tests in one maven tests pass. (the
[prototype|https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]
requires one more tests pass to run tests for schedulerNG tests)
1. The parameterized tests mechanism of JUnit 4.12(currently used) only
supports the same parameters for all tests in the same class.
2. The parameterized tests mechanism of JUnit 5 can support to execute a single
test method multiple times with different parameters. However, I'm not sure if
anything would break if we'd like to upgrade it.
3. I also found a lib [JUnitParams|https://github.com/Pragmatists/JUnitParams]
which supports method level test parameters for JUnit 4.12. However, the
project has no new commit for almost 1 year. So it might have been out of
maintenance.
One concern to use parameterized tests is that we may need to change the
existing MiniCluster tests to generate and handle scheduler parameters, even
though the tests target for verifying nothing related with schedulers. And
later anyone developing an IT case needs to tag and parameterize it as well.
This is a bit weird and is hard to guarantee as some IT cases are out of the
control of runtime.
Considering a long term solution to verify different schedulers, I think the
[prototype|https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]
from [~till.rohrmann] would be better.
With it we can run MiniCluster tests for different schedulers with different
profiles.
For per-commit tests maybe we can test the default scheduler("legacy" currently
and "ng" later) only. And other schedulers can be verified with other scheduled
travis tests or in manual ways.
A category named "MiniClusterTest" (rather than "AlsoRunWithSchedulerNG") can
be used to tag the all MiniCluster tests so that it still works when we switch
the default scheduler to "ng".
What do you think?
was (Author: zhuzh):
Hi [~gjy], I did some investigation on how to get different MiniCluster test
methods to run multiple times for different schedulers, so that we can do all
the tests in one maven tests pass. (the
[prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]]
requires one more tests pass to run tests for schedulerNG tests)
1. The parameterized tests mechanism of JUnit 4.12(currently used) only
supports the same parameters for all tests in the same class.
2. The parameterized tests mechanism of JUnit 5 can support to execute a single
test method multiple times with different parameters. However, I'm not sure if
anything would break if we'd like to upgrade it.
3. I also found a lib
[JUnitParams|[https://github.com/Pragmatists/JUnitParams]] which supports
method level test parameters for JUnit 4.12. However, the project has no new
commit for almost 1 year. So it might have been out of maintenance.
One concern to use parameterized tests is that we may need to change the
existing MiniCluster tests to generate and handle scheduler parameters, even
though the tests target for verifying nothing related with schedulers. And
later anyone developing an IT case needs to tag and parameterize it as well.
This is a bit weird and is hard to guarantee as some IT cases are out of the
control of runtime.
Considering a long term solution to verify different schedulers, I think the
[prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]]
from [~till.rohrmann] would be better.
With it we can run MiniCluster tests for different schedulers with different
profiles.
For per-commit tests maybe we can test the default scheduler("legacy" currently
and "ng" later) only. And other schedulers can be verified with other scheduled
travis tests or in manual ways.
A category named "MiniClusterTest" (rather than "AlsoRunWithSchedulerNG") can
be used to tag the all MiniCluster tests so that it still works when we switch
the default scheduler to "ng".
What do you think?
> Facilitate enabling new Scheduler in MiniCluster Tests
> --
>
> Key: FLINK-13953
> URL: https://issues.apache.org/jira/browse/FLINK-13953
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination, Tests
>Reporter: Gary Yao
>Assignee: Zhu Zhu
>Priority: Major
>
> Currently, tests using the {{MiniCluster}} use the legacy scheduler by
> default. Once the new scheduler is implemented, we should run tests with the
> new
[GitHub] [flink] flinkbot commented on issue #9649: [FLINK-13994][docs]Translage 'Getting Started' page
flinkbot commented on issue #9649: [FLINK-13994][docs]Translage 'Getting Started' page URL: https://github.com/apache/flink/pull/9649#issuecomment-529372783 ## CI report: * fb2dd86ea8f17d546f026f779625813b1dd79e9a : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese
flinkbot edited a comment on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese URL: https://github.com/apache/flink/pull/9648#issuecomment-529343948 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 6def0e035c5e7f442e45a00f1f7628726aef42eb (Mon Sep 09 09:04:28 UTC 2019) ✅no warnings Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (FLINK-13953) Facilitate enabling new Scheduler in MiniCluster Tests
[
https://issues.apache.org/jira/browse/FLINK-13953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925457#comment-16925457
]
Zhu Zhu edited comment on FLINK-13953 at 9/9/19 9:04 AM:
-
Hi [~gjy], I did some investigation on how to get different MiniCluster test
methods to run multiple times for different schedulers, so that we can do all
the tests in one maven tests pass. (the
[prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]]
requires one more tests pass to run tests for schedulerNG tests)
1. The parameterized tests mechanism of JUnit 4.12(currently used) only
supports the same parameters for all tests in the same class.
2. The parameterized tests mechanism of JUnit 5 can support to execute a single
test method multiple times with different parameters. However, I'm not sure if
anything would break if we'd like to upgrade it.
3. I also found a lib
[JUnitParams|[https://github.com/Pragmatists/JUnitParams]] which supports
method level test parameters for JUnit 4.12. However, the project has no new
commit for almost 1 year. So it might have been out of maintenance.
One concern to use parameterized tests is that we may need to change the
existing MiniCluster tests to generate and handle scheduler parameters, even
though the tests target for verifying nothing related with schedulers. And
later anyone developing an IT case needs to tag and parameterize it as well.
This is a bit weird and is hard to guarantee as some IT cases are out of the
control of runtime.
Considering a long term solution to verify different schedulers, I think the
[prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]]
from [~till.rohrmann] would be better.
With it we can run MiniCluster tests for different schedulers with different
profiles.
For per-commit tests maybe we can test the default scheduler("legacy" currently
and "ng" later) only. And other schedulers can be verified with other scheduled
travis tests or in manual ways.
A category named "MiniClusterTest" (rather than "AlsoRunWithSchedulerNG") can
be used to tag the all MiniCluster tests so that it still works when we switch
the default scheduler to "ng".
What do you think?
was (Author: zhuzh):
Hi [~gjy], I did some investigation on how to get different MiniCluster test
methods to run multiple times for different schedulers, so that we can do all
the tests in one maven tests pass. (the
[prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]]
requires one more tests pass to run tests for schedulerNG tests)
1. The parameterized tests mechanism of JUnit 4.12(currently used) only
supports the same parameters for all tests in the same class.
2. The parameterized tests mechanism of JUnit 5 can support to execute a single
test method multiple times with different parameters. However, I'm not sure if
anything would broke if we'd like to upgrade it.
3. I also found a lib
[JUnitParams|[https://github.com/Pragmatists/JUnitParams]] which supports
method level test parameters for JUnit 4.12. However, the project has no new
commit for almost 1 year. So I'm not sure if it's a good idea to use it as it
might have been out of maintenance.
One concern to use parameterized tests is that we may need to change the
existing MiniCluster tests to generate, parse and handle the parameters, even
though the tests target for verifying nothing related with schedulers. And
later anyone developing an IT case needs to tag and parameterize it as well.
This is a bit weird and is hard to guarantee as some IT cases are out of the
control of runtime.
Considering a long term solution to verify different schedulers, I think the
[prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]]
from [~till.rohrmann] would be better.
With it we can run MiniCluster tests for different schedulers with different
profiles.
For per-commit tests maybe we can test the default scheduler("legacy" currently
and "ng" later) only. And other schedulers can be verified with other scheduled
travis tests and in manual ways.
A category named "MiniClusterTest" (rather than "AlsoRunWithSchedulerNG") can
be used to tag the all MiniCluster tests so that it still works when we switch
the default scheduler to "ng".
What do you think?
> Facilitate enabling new Scheduler in MiniCluster Tests
> --
>
> Key: FLINK-13953
> URL: https://issues.apache.org/jira/browse/FLINK-13953
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination, Tests
>Reporter: Gary Yao
>Assignee: Zhu Zhu
>Priority: Major
>
> Currently, tests using the {{MiniCluster}} use the legacy scheduler by
> default. Once the new
[jira] [Commented] (FLINK-13994) Translate "Getting Started" overview to Chinese
[ https://issues.apache.org/jira/browse/FLINK-13994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925491#comment-16925491 ] gaofeilong commented on FLINK-13994: I have finished the translation and make a new PR on github. > Translate "Getting Started" overview to Chinese > --- > > Key: FLINK-13994 > URL: https://issues.apache.org/jira/browse/FLINK-13994 > Project: Flink > Issue Type: Task > Components: chinese-translation, Documentation >Reporter: Fabian Hueske >Assignee: gaofeilong >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The "Getting Started" overview page needs to be translated to Chinese: > https://github.com/apache/flink/blob/master/docs/getting-started/index.zh.md -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] wuchong commented on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese
wuchong commented on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese URL: https://github.com/apache/flink/pull/9648#issuecomment-529370517 No @lord-i-will . You can rebase your branch based on the latest master. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9649: [FLINK-13994][docs]Translage 'Getting Started' page
flinkbot commented on issue #9649: [FLINK-13994][docs]Translage 'Getting Started' page URL: https://github.com/apache/flink/pull/9649#issuecomment-529370466 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit fb2dd86ea8f17d546f026f779625813b1dd79e9a (Mon Sep 09 09:02:13 UTC 2019) ✅no warnings Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-13994) Translate "Getting Started" overview to Chinese
[ https://issues.apache.org/jira/browse/FLINK-13994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-13994: --- Labels: pull-request-available (was: ) > Translate "Getting Started" overview to Chinese > --- > > Key: FLINK-13994 > URL: https://issues.apache.org/jira/browse/FLINK-13994 > Project: Flink > Issue Type: Task > Components: chinese-translation, Documentation >Reporter: Fabian Hueske >Assignee: gaofeilong >Priority: Major > Labels: pull-request-available > > The "Getting Started" overview page needs to be translated to Chinese: > https://github.com/apache/flink/blob/master/docs/getting-started/index.zh.md -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese
flinkbot edited a comment on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese URL: https://github.com/apache/flink/pull/9648#issuecomment-529343948 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 6def0e035c5e7f442e45a00f1f7628726aef42eb (Mon Sep 09 08:59:23 UTC 2019) ✅no warnings Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] gaofeilong198810 opened a new pull request #9649: [FLINK-13994][docs]Translage 'Getting Started' page
gaofeilong198810 opened a new pull request #9649: [FLINK-13994][docs]Translage 'Getting Started' page URL: https://github.com/apache/flink/pull/9649 ## What is the purpose of the change *(For example: This pull request makes task deployment go through the blob server, rather than through RPC. That way we avoid re-transferring them on each deployment (during recovery).)* ## Brief change log *(for example:)* - *The TaskInfo is stored in the blob store on job creation time as a persistent artifact* - *Deployments RPC transmits only the blob storage reference* - *TaskManagers retrieve the TaskInfo from the blob cache* ## Verifying this change *(Please pick either of the following options)* This change is a trivial rework / code cleanup without any test coverage. *(or)* This change is already covered by existing tests, such as *(please describe tests)*. *(or)* This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (100MB)* - *Extended integration test for recovery after master (JobManager) failure* - *Added test that validates that TaskInfo is transferred only once across recoveries* - *Manually verified the change by running a 4 node cluser with 2 JobManagers and 4 TaskManagers, a stateful streaming program, and killing one JobManager and two TaskManagers during the execution, verifying that recovery happens correctly.* ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / no) - The serializers: (yes / no / don't know) - The runtime per-record code paths (performance sensitive): (yes / no / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / no / don't know) - The S3 file system connector: (yes / no / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9647: [FLINK-14005][hive] Support Hive version 2.2.0
flinkbot edited a comment on issue #9647: [FLINK-14005][hive] Support Hive version 2.2.0 URL: https://github.com/apache/flink/pull/9647#issuecomment-529282630 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit cab04a32f620acef83f46bc111512b748b4acce8 (Mon Sep 09 08:58:22 UTC 2019) **Warnings:** * **1 pom.xml files were touched**: Check for build and licensing issues. * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] lord-i-will commented on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese
lord-i-will commented on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese URL: https://github.com/apache/flink/pull/9648#issuecomment-529368509 Now i have to close this PR? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] lord-i-will edited a comment on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese
lord-i-will edited a comment on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese URL: https://github.com/apache/flink/pull/9648#issuecomment-529368509 Now i have to close this PR? @wuchong This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9644: [FLINK-13931][hive] Support Hive version 2.0.x
flinkbot edited a comment on issue #9644: [FLINK-13931][hive] Support Hive version 2.0.x URL: https://github.com/apache/flink/pull/9644#issuecomment-529009260 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 1472dbd78a06bc5897aa4922adfb66e82cd01f40 (Mon Sep 09 08:56:20 UTC 2019) **Warnings:** * **1 pom.xml files were touched**: Check for build and licensing issues. * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13872) Translate Operations Playground to Chinese
[ https://issues.apache.org/jira/browse/FLINK-13872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925480#comment-16925480 ] Jark Wu commented on FLINK-13872: - Let's move the discussion to the pull request. > Translate Operations Playground to Chinese > -- > > Key: FLINK-13872 > URL: https://issues.apache.org/jira/browse/FLINK-13872 > Project: Flink > Issue Type: Task > Components: chinese-translation, Documentation >Affects Versions: 1.9.0 >Reporter: Fabian Hueske >Assignee: Lord i Will >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The [Operations > Playground|https://ci.apache.org/projects/flink/flink-docs-release-1.9/getting-started/docker-playgrounds/flink_operations_playground.html] > is a quick and convenient way to learn about Flink's operational features > (job submission, failure recovery, job updates, scaling, metrics). > We should translate it to Chinese as well. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese
flinkbot edited a comment on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese URL: https://github.com/apache/flink/pull/9648#issuecomment-529343948 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 6def0e035c5e7f442e45a00f1f7628726aef42eb (Mon Sep 09 08:55:18 UTC 2019) ✅no warnings Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] wuchong edited a comment on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese
wuchong edited a comment on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese URL: https://github.com/apache/flink/pull/9648#issuecomment-529366699 Hi @lord-i-will , thanks for the contribution. There are some conflicts between your branch and master. Please rebase your branch to current master.  This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] wuchong commented on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese
wuchong commented on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese URL: https://github.com/apache/flink/pull/9648#issuecomment-529366699 Hi @lord-i-will , thanks for the contribution. There are some conflicts between your branch and master. Please rebase your branch to current master. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface
flinkbot edited a comment on issue #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface URL: https://github.com/apache/flink/pull/9604#issuecomment-527441976 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 66ade45fec93a56dde7609dffa0ad9dd241d0713 (Mon Sep 09 08:54:17 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❗ 3. Needs [attention] from. - Needs attention by @aljoscha [PMC] * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] Myasuka commented on issue #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface
Myasuka commented on issue #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface URL: https://github.com/apache/flink/pull/9604#issuecomment-529366170 It seems command `run travis` not taking effect @zentol . To rerun the CI, I have to close-reopen this PR. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-14009) Cron jobs broken due to verifying incorrect NOTICE-binary file
[
https://issues.apache.org/jira/browse/FLINK-14009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925479#comment-16925479
]
Jark Wu commented on FLINK-14009:
-
FLINK-13968 is fixed in 1.8.3, so I changed this issue to 1.8.3.
> Cron jobs broken due to verifying incorrect NOTICE-binary file
> --
>
> Key: FLINK-14009
> URL: https://issues.apache.org/jira/browse/FLINK-14009
> Project: Flink
> Issue Type: Bug
> Components: Build System
>Affects Versions: 1.10.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Blocker
> Fix For: 1.9.1, 1.8.3
>
>
> With FLINK-13968 we introduced an automatic {{NOTICE-binary}} file check.
> However, since we don't use the correct {{NOTICE-binary}} file (FLINK-14008)
> for Scala 2.12 it fails currently our cron jobs.
> I suggest to only enable the automatic {{NOTICE-binary}} files for Scala 2.11
> until FLINK-14008 has been fixed.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface
flinkbot edited a comment on issue #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface URL: https://github.com/apache/flink/pull/9604#issuecomment-527441976 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 66ade45fec93a56dde7609dffa0ad9dd241d0713 (Mon Sep 09 08:52:14 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❗ 3. Needs [attention] from. - Needs attention by @aljoscha [PMC] * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-14009) Cron jobs broken due to verifying incorrect NOTICE-binary file
[
https://issues.apache.org/jira/browse/FLINK-14009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu updated FLINK-14009:
Fix Version/s: (was: 1.8.2)
1.8.3
> Cron jobs broken due to verifying incorrect NOTICE-binary file
> --
>
> Key: FLINK-14009
> URL: https://issues.apache.org/jira/browse/FLINK-14009
> Project: Flink
> Issue Type: Bug
> Components: Build System
>Affects Versions: 1.10.0
>Reporter: Till Rohrmann
>Assignee: Till Rohrmann
>Priority: Blocker
> Fix For: 1.9.1, 1.8.3
>
>
> With FLINK-13968 we introduced an automatic {{NOTICE-binary}} file check.
> However, since we don't use the correct {{NOTICE-binary}} file (FLINK-14008)
> for Scala 2.12 it fails currently our cron jobs.
> I suggest to only enable the automatic {{NOTICE-binary}} files for Scala 2.11
> until FLINK-14008 has been fixed.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9643: [FLINK-13961][client] Remove obsolete abstraction JobExecutor(Service)
flinkbot edited a comment on issue #9643: [FLINK-13961][client] Remove obsolete abstraction JobExecutor(Service) URL: https://github.com/apache/flink/pull/9643#issuecomment-528865483 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 0f85333e5d275aedc4560241bf9a268d887bb5e7 (Mon Sep 09 08:50:13 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] Myasuka closed pull request #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface
Myasuka closed pull request #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface URL: https://github.com/apache/flink/pull/9604 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] Myasuka opened a new pull request #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface
Myasuka opened a new pull request #9604: [FLINK-13845][docs] Drop all the content of removed Checkpointed interface URL: https://github.com/apache/flink/pull/9604 ## What is the purpose of the change Drop all the content of removed `Checkpointed` interface. ## Brief change log ## Verifying this change This change is a trivial rework without any test coverage. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): **no** - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: **no** - The serializers: **no** - The runtime per-record code paths (performance sensitive): **no** - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: **no** - The S3 file system connector: **no** ## Documentation - Does this pull request introduce a new feature? **no** - If yes, how is the feature documented? **not applicable** This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (FLINK-13994) Translate "Getting Started" overview to Chinese
[
https://issues.apache.org/jira/browse/FLINK-13994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925475#comment-16925475
]
Jark Wu edited comment on FLINK-13994 at 9/9/19 8:49 AM:
-
Hi [~gaofeilong],
1. you don't need to translate the table (but the "title" and "nav-title" can
be translated).
2. Yes. The link should be changed to Chinese version.
If the link is started from "./", then nothing needs to change, e.g.
"(./docker-playgrounds/flink-operations-playground.html)"
If the link is started from "{{ site.baseurl }}", then you should add "/zh"
after that, e.g. "({{ site.baseurl }}/dev/event_timestamps_watermarks.html)"
needs to be changed to "({{ site.baseurl
}}/zh/dev/event_timestamps_watermarks.html)".
was (Author: jark):
Hi [~gaofeilong],
1. you don't need to translate the table (but the title can be translated).
2. Yes. The link should be changed to Chinese version.
If the link is started from "./", then nothing needs to change, e.g.
"(./docker-playgrounds/flink-operations-playground.html)"
If the link is started from "{{ site.baseurl }}", then you should add "/zh"
after that, e.g. "({{ site.baseurl }}/dev/event_timestamps_watermarks.html)"
needs to be changed to "({{ site.baseurl
}}/zh/dev/event_timestamps_watermarks.html)".
> Translate "Getting Started" overview to Chinese
> ---
>
> Key: FLINK-13994
> URL: https://issues.apache.org/jira/browse/FLINK-13994
> Project: Flink
> Issue Type: Task
> Components: chinese-translation, Documentation
>Reporter: Fabian Hueske
>Assignee: gaofeilong
>Priority: Major
>
> The "Getting Started" overview page needs to be translated to Chinese:
> https://github.com/apache/flink/blob/master/docs/getting-started/index.zh.md
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (FLINK-13994) Translate "Getting Started" overview to Chinese
[
https://issues.apache.org/jira/browse/FLINK-13994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925475#comment-16925475
]
Jark Wu commented on FLINK-13994:
-
Hi [~gaofeilong],
1. you don't need to translate the table (but the title can be translated).
2. Yes. The link should be changed to Chinese version.
If the link is started from "./", then nothing needs to change, e.g.
"(./docker-playgrounds/flink-operations-playground.html)"
If the link is started from "{{ site.baseurl }}", then you should add "/zh"
after that, e.g. "({{ site.baseurl }}/dev/event_timestamps_watermarks.html)"
needs to be changed to "({{ site.baseurl
}}/zh/dev/event_timestamps_watermarks.html)".
> Translate "Getting Started" overview to Chinese
> ---
>
> Key: FLINK-13994
> URL: https://issues.apache.org/jira/browse/FLINK-13994
> Project: Flink
> Issue Type: Task
> Components: chinese-translation, Documentation
>Reporter: Fabian Hueske
>Assignee: gaofeilong
>Priority: Major
>
> The "Getting Started" overview page needs to be translated to Chinese:
> https://github.com/apache/flink/blob/master/docs/getting-started/index.zh.md
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [flink] TisonKun commented on issue #9643: [FLINK-13961][client] Remove obsolete abstraction JobExecutor(Service)
TisonKun commented on issue #9643: [FLINK-13961][client] Remove obsolete abstraction JobExecutor(Service) URL: https://github.com/apache/flink/pull/9643#issuecomment-529364151 travis fails on NOTICE-binary checker. I don't think it is relevant to this pull request. Instead, it looks like an instance of [FLINK-14009](https://issues.apache.org/jira/browse/FLINK-14009). @tillrohrmann is it relevant? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9644: [FLINK-13931][hive] Support Hive version 2.0.x
flinkbot edited a comment on issue #9644: [FLINK-13931][hive] Support Hive version 2.0.x URL: https://github.com/apache/flink/pull/9644#issuecomment-529009260 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 1472dbd78a06bc5897aa4922adfb66e82cd01f40 (Mon Sep 09 08:48:10 UTC 2019) **Warnings:** * **1 pom.xml files were touched**: Check for build and licensing issues. * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] lirui-apache commented on a change in pull request #9644: [FLINK-13931][hive] Support Hive version 2.0.x
lirui-apache commented on a change in pull request #9644: [FLINK-13931][hive]
Support Hive version 2.0.x
URL: https://github.com/apache/flink/pull/9644#discussion_r322126762
##
File path:
flink-connectors/flink-connector-hive/src/test/java/org/apache/flink/connectors/hive/TableEnvHiveConnectorTest.java
##
@@ -114,6 +114,10 @@ public void testGetNonExistingFunction() throws Exception
{
public void testDifferentFormats() throws Exception {
String[] formats = new String[]{"orc", "parquet",
"sequencefile", "csv"};
for (String format : formats) {
+ if (format.equals("orc") &&
hmsClient.getHiveVersion().startsWith("2.0")) {
+ // Ignore orc test for Hive version 2.0.x for
now due to FLINK-13998
Review comment:
Do we plan to test against 2.0.1 as well? If so, we also need to skip
parquet and sequencefile tests for 2.0.1.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (FLINK-13994) Translate "Getting Started" overview to Chinese
[ https://issues.apache.org/jira/browse/FLINK-13994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925469#comment-16925469 ] gaofeilong commented on FLINK-13994: hi [~jark], I have two question when i translating this page:[https://github.com/apache/flink/blob/master/docs/getting-started/index.zh.md:|https://github.com/apache/flink/blob/master/docs/getting-started/index.zh.md] # should I translate the table on the top of the page? # the link in this page is relative path, should I change the link in this page to zh's link? > Translate "Getting Started" overview to Chinese > --- > > Key: FLINK-13994 > URL: https://issues.apache.org/jira/browse/FLINK-13994 > Project: Flink > Issue Type: Task > Components: chinese-translation, Documentation >Reporter: Fabian Hueske >Assignee: gaofeilong >Priority: Major > > The "Getting Started" overview page needs to be translated to Chinese: > https://github.com/apache/flink/blob/master/docs/getting-started/index.zh.md -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (FLINK-14010) Dispatcher & JobManagers don't give up leadership when AM is shut down
[
https://issues.apache.org/jira/browse/FLINK-14010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925460#comment-16925460
]
TisonKun commented on FLINK-14010:
--
CC [~StephanEwen] [~till.rohrmann] [~xiaogang.shi]
Here comes a high-level problem, do we explicitly constrain Dispatcher,
ResourceManager and JobManagers run on one process?
1. the usage of reference to {{JobManagerGateway}} in Dispatcher already infers
that we require this.
2. back to the design of FLIP-6, we have a global singleton of Dispatcher, and
for each job, launch a JobManager and ResourceManager. The implementation
diverges quite a lot. Could you please provide any background?
3. if we explicitly constrain as above, we actually need not to start leader
election services per components, actually, we can use the abstraction and
layout as below:
- start a leader election service per dispatcher-resource-manager component, in
cluster entrypoint level. It will participant the election and all metadata
commits are delegate to this service.
- all cluster level components that need to publish their address, such as
Dispatcher, ResourceManager and WebMonitor publish their address via this
leader election service.
- Actors can be started as {{PermanentlyFencedRpcEndpoint}} and thus we survive
from handling a lot of mutable shared state among leadership epoch.
Specifically, cluster entrypoint acts as DispatcherRunner and so on, like
JobManagerRunner to JobMaster. See also [this
branch|https://github.com/tillrohrmann/flink/commits/removeSuspendFromJobMaster].
- back to this issue, cluster entrypoint({{YARNClusterEntrypoint}} maybe)
reacts to AMRM request and thus all components can be required to shutdown
properly.
> Dispatcher & JobManagers don't give up leadership when AM is shut down
> --
>
> Key: FLINK-14010
> URL: https://issues.apache.org/jira/browse/FLINK-14010
> Project: Flink
> Issue Type: Bug
> Components: Deployment / YARN, Runtime / Coordination
>Affects Versions: 1.7.2, 1.8.1, 1.9.0, 1.10.0
>Reporter: TisonKun
>Priority: Critical
>
> In YARN deployment scenario, YARN RM possibly launches a new AM for the job
> even if the previous AM does not terminated, for example, when AMRM heartbeat
> timeout. This is a common case that RM will send a shutdown request to the
> previous AM and expect the AM shutdown properly.
> However, currently in {{YARNResourceManager}}, we handle this request in
> {{onShutdownRequest}} which simply close the {{YARNResourceManager}} *but not
> Dispatcher and JobManagers*. Thus, Dispatcher and JobManager launched in new
> AM cannot be granted leadership properly. Visually,
> on previous AM: Dispatcher leader, JM leaders
> on new AM: ResourceManager leader
> since on client side or in per-job mode, JobManager address and port are
> configured as the new AM, the whole cluster goes into an unrecoverable
> inconsistent status: client all queries the dispatcher on new AM who is now
> the leader. Briefly, Dispatcher and JobManagers on previous AM do not give up
> their leadership properly.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Comment Edited] (FLINK-13953) Facilitate enabling new Scheduler in MiniCluster Tests
[
https://issues.apache.org/jira/browse/FLINK-13953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925457#comment-16925457
]
Zhu Zhu edited comment on FLINK-13953 at 9/9/19 8:30 AM:
-
Hi [~gjy], I did some investigation on how to get different MiniCluster test
methods to run multiple times for different schedulers, so that we can do all
the tests in one maven tests pass. (the
[prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]]
requires one more tests pass to run tests for schedulerNG tests)
1. The parameterized tests mechanism of JUnit 4.12(currently used) only
supports the same parameters for all tests in the same class.
2. The parameterized tests mechanism of JUnit 5 can support to execute a single
test method multiple times with different parameters. However, I'm not sure if
anything would broke if we'd like to upgrade it.
3. I also found a lib
[JUnitParams|[https://github.com/Pragmatists/JUnitParams]] which supports
method level test parameters for JUnit 4.12. However, the project has no new
commit for almost 1 year. So I'm not sure if it's a good idea to use it as it
might have been out of maintenance.
One concern to use parameterized tests is that we may need to change the
existing MiniCluster tests to generate, parse and handle the parameters, even
though the tests target for verifying nothing related with schedulers. And
later anyone developing an IT case needs to tag and parameterize it as well.
This is a bit weird and is hard to guarantee as some IT cases are out of the
control of runtime.
Considering a long term solution to verify different schedulers, I think the
[prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]]
from [~till.rohrmann] would be better.
With it we can run MiniCluster tests for different schedulers with different
profiles.
For per-commit tests maybe we can test the default scheduler("legacy" currently
and "ng" later) only. And other schedulers can be verified with other scheduled
travis tests and in manual ways.
A category named "MiniClusterTest" (rather than "AlsoRunWithSchedulerNG") can
be used to tag the all MiniCluster tests so that it still works when we switch
the default scheduler to "ng".
What do you think?
was (Author: zhuzh):
Hi [~gjy], I did some investigation on how to get different MiniCluster test
methods to run multiple times for different schedulers, so that we can do all
the tests in one maven tests pass. (the
[prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]]
requires one more tests pass to run tests for schedulerNG tests)
1. The parameterized tests mechanism of JUnit 4.12(currently used) only
supports the same parameters for all tests in the same class.
2. The parameterized tests mechanism of JUnit 5 can support to execute a single
test method multiple times with different parameters. However, I'm not sure if
anything would broke if we'd like to upgrade it.
3. I also found a lib
[JUnitParams|[https://github.com/Pragmatists/JUnitParams]] which supports
method level test parameters for JUnit 4.12. However, the project has no new
commit for almost 1 year. So I'm not sure if it's a good idea to use it as it
might have been out of maintenance.
One concern to use parameterized tests is that we may need to change the
existing MiniCluster tests to generate, parse and handle the parameters, even
though the tests target for verifying nothing related with schedulers. And
later anyone developing an IT case needs to tag and parameterize it as well.
This is a bit weird and is hard to guarantee as some IT cases are out of the
control of runtime.
Considering a long term solution to verify different schedulers, I think the
[prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]]
from [~till.rohrmann] would be better.
With it we can run MiniCluster tests for different schedulers with different
profiles.
For per-commit tests only the default scheduler("legacy" currently and "ng"
later) is used. And scheduled travis builds can be added for other schedulers.
A category named "MiniClusterTest" (rather than "AlsoRunWithSchedulerNG") can
be used to tag the all MiniCluster tests so that it still works when we switch
the default scheduler to "ng".
What do you think?
> Facilitate enabling new Scheduler in MiniCluster Tests
> --
>
> Key: FLINK-13953
> URL: https://issues.apache.org/jira/browse/FLINK-13953
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination, Tests
>Reporter: Gary Yao
>Assignee: Zhu Zhu
>Priority: Major
>
> Currently, tests using the {{MiniCluster}} use the legacy scheduler by
> default. Once the
[jira] [Comment Edited] (FLINK-13953) Facilitate enabling new Scheduler in MiniCluster Tests
[
https://issues.apache.org/jira/browse/FLINK-13953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925457#comment-16925457
]
Zhu Zhu edited comment on FLINK-13953 at 9/9/19 8:28 AM:
-
Hi [~gjy], I did some investigation on how to get different MiniCluster test
methods to run multiple times for different schedulers, so that we can do all
the tests in one maven tests pass. (the
[prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]]
requires one more tests pass to run tests for schedulerNG tests)
1. The parameterized tests mechanism of JUnit 4.12(currently used) only
supports the same parameters for all tests in the same class.
2. The parameterized tests mechanism of JUnit 5 can support to execute a single
test method multiple times with different parameters. However, I'm not sure if
anything would broke if we'd like to upgrade it.
3. I also found a lib
[JUnitParams|[https://github.com/Pragmatists/JUnitParams]] which supports
method level test parameters for JUnit 4.12. However, the project has no new
commit for almost 1 year. So I'm not sure if it's a good idea to use it as it
might have been out of maintenance.
One concern to use parameterized tests is that we may need to change the
existing MiniCluster tests to generate, parse and handle the parameters, even
though the tests target for verifying nothing related with schedulers. And
later anyone developing an IT case needs to tag and parameterize it as well.
This is a bit weird and is hard to guarantee as some IT cases are out of the
control of runtime.
Considering a long term solution to verify different schedulers, I think the
[prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]]
from [~till.rohrmann] would be better.
With it we can run MiniCluster tests for different schedulers with different
profiles.
For per-commit tests only the default scheduler("legacy" currently and "ng"
later) is used. And scheduled travis builds can be added for other schedulers.
A category named "MiniClusterTest" (rather than "AlsoRunWithSchedulerNG") can
be used to tag the all MiniCluster tests so that it still works when we switch
the default scheduler to "ng".
What do you think?
was (Author: zhuzh):
Hi [~gjy], I did some investigation on how to get different MiniCluster test
methods to run multiple times for different schedulers, so that we can do all
the tests in one maven tests pass. (the
[prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]]
requires one more tests pass to run tests for schedulerNG tests)
1. The parameterized tests mechanism of JUnit 4.12(currently used) only
supports the same parameters for all tests in the same class.
2. The parameterized tests mechanism of JUnit 5 can support to execute a single
test method multiple times with different parameters. However, I'm not sure if
anything would broke if we'd like to upgrade it.
3. I also found a lib
[JUnitParams|[https://github.com/Pragmatists/JUnitParams]] which supports
method level test parameters for JUnit 4.12. However, the project has no new
commit for almost 1 year. So I'm not sure if it's a good idea to use it as it
might have been out of maintenance.
One concern to use parameterized tests is that we may need to change the
existing MiniCluster tests to generate, parse and handle the parameters, even
though the tests target for verifying nothing related with schedulers. And
later anyone developing an IT case needs to tag and parameterize it as well.
This is a bit weird and is hard to guarantee as some IT cases are out of the
control of runtime.
Considering a long term solution to verify different schedulers, I think the
[prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]]
from [~till.rohrmann] would be better.
With it we can run MiniCluster tests for different schedulers with different
profiles.
For per-commit tests only the default scheduler("legacy" currently and "ng"
later) is used. And scheduled travis builds can be added for other schedulers.
A category named "MiniClusterTest" (rather than "AlsoRunWithSchedulerNG") can
be used to tag the all MiniCluster tests so that it still works when we switch
the default scheduler to "ng".
What do you think?
> Facilitate enabling new Scheduler in MiniCluster Tests
> --
>
> Key: FLINK-13953
> URL: https://issues.apache.org/jira/browse/FLINK-13953
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination, Tests
>Reporter: Gary Yao
>Assignee: Zhu Zhu
>Priority: Major
>
> Currently, tests using the {{MiniCluster}} use the legacy scheduler by
> default. Once the new scheduler is implemented, we
[jira] [Commented] (FLINK-13953) Facilitate enabling new Scheduler in MiniCluster Tests
[
https://issues.apache.org/jira/browse/FLINK-13953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925457#comment-16925457
]
Zhu Zhu commented on FLINK-13953:
-
Hi [~gjy], I did some investigation on how to get different MiniCluster test
methods to run multiple times for different schedulers, so that we can do all
the tests in one maven tests pass. (the
[prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]]
requires one more tests pass to run tests for schedulerNG tests)
1. The parameterized tests mechanism of JUnit 4.12(currently used) only
supports the same parameters for all tests in the same class.
2. The parameterized tests mechanism of JUnit 5 can support to execute a single
test method multiple times with different parameters. However, I'm not sure if
anything would broke if we'd like to upgrade it.
3. I also found a lib
[JUnitParams|[https://github.com/Pragmatists/JUnitParams]] which supports
method level test parameters for JUnit 4.12. However, the project has no new
commit for almost 1 year. So I'm not sure if it's a good idea to use it as it
might have been out of maintenance.
One concern to use parameterized tests is that we may need to change the
existing MiniCluster tests to generate, parse and handle the parameters, even
though the tests target for verifying nothing related with schedulers. And
later anyone developing an IT case needs to tag and parameterize it as well.
This is a bit weird and is hard to guarantee as some IT cases are out of the
control of runtime.
Considering a long term solution to verify different schedulers, I think the
[prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]]
from [~till.rohrmann] would be better.
With it we can run MiniCluster tests for different schedulers with different
profiles.
For per-commit tests only the default scheduler("legacy" currently and "ng"
later) is used. And scheduled travis builds can be added for other schedulers.
A category named "MiniClusterTest" (rather than "AlsoRunWithSchedulerNG") can
be used to tag the all MiniCluster tests so that it still works when we switch
the default scheduler to "ng".
What do you think?
> Facilitate enabling new Scheduler in MiniCluster Tests
> --
>
> Key: FLINK-13953
> URL: https://issues.apache.org/jira/browse/FLINK-13953
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination, Tests
>Reporter: Gary Yao
>Assignee: Zhu Zhu
>Priority: Major
>
> Currently, tests using the {{MiniCluster}} use the legacy scheduler by
> default. Once the new scheduler is implemented, we should run tests with the
> new scheduler enabled. However, it is not expected that all tests will pass
> immediately. Therefore, it should be possible to enable the new scheduler for
> a subset of tests.
> *Acceptance Criteria*
> * Subset of tests using {{MiniCluster}} can be run on a per-commit basis (on
> Travis) against new scheduler and also legacy scheduler
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator
flinkbot edited a comment on issue #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator URL: https://github.com/apache/flink/pull/9564#issuecomment-526484775 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 16bac68a7fd82876d0ad2d56c8e7c199dc816d04 (Mon Sep 09 08:20:41 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-12958).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] pnowojski commented on a change in pull request #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperat
pnowojski commented on a change in pull request #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator URL: https://github.com/apache/flink/pull/9564#discussion_r322080057 ## File path: flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/tasks/StreamTaskTest.java ## @@ -1380,6 +1398,54 @@ protected void performDefaultAction(DefaultActionContext context) throws Excepti } } + /** +* A task that checks that some StreamTask methods are called only in the main task's thread. Review comment: I would rephrase this comment and move it above `testThreadInvariants`. Otherwise it's quite difficult to understand what this test is suppose to do. Also could you copy paste this also as a commit message? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] pnowojski commented on a change in pull request #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperat
pnowojski commented on a change in pull request #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperator URL: https://github.com/apache/flink/pull/9564#discussion_r322087304 ## File path: flink-runtime/src/main/java/org/apache/flink/runtime/jobgraph/tasks/AbstractInvokable.java ## @@ -261,10 +261,9 @@ public void abortCheckpointOnBarrier(long checkpointId, Throwable cause) throws * Invoked when a checkpoint has been completed, i.e., when the checkpoint coordinator has received Review comment: @1u0 ? (You have resolved the comment, but haven't responded and haven't updated the commit message) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] pnowojski commented on a change in pull request #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperat
pnowojski commented on a change in pull request #9564:
[FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox
for timer triggers, checkpoints and AsyncWaitOperator
URL: https://github.com/apache/flink/pull/9564#discussion_r322079010
##
File path:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/StreamTask.java
##
@@ -428,6 +429,10 @@ public final void invoke() throws Exception {
// See FLINK-7430
isRunning = false;
}
+ MailboxExecutor mainMailboxExecutor =
mailboxProcessor.getMainMailboxExecutor();
Review comment:
Was this addressed @1u0?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] pnowojski commented on a change in pull request #9564: [FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox for timer triggers, checkpoints and AsyncWaitOperat
pnowojski commented on a change in pull request #9564:
[FLINK-12481][FLINK-12482][FLINK-12958] Streaming runtime: integrate mailbox
for timer triggers, checkpoints and AsyncWaitOperator
URL: https://github.com/apache/flink/pull/9564#discussion_r322111247
##
File path:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/async/Emitter.java
##
@@ -80,9 +88,14 @@ public void run() {
try {
while (running) {
LOG.debug("Wait for next completed async stream
element result.");
- AsyncResult streamElementEntry =
streamElementQueue.peekBlockingly();
-
- output(streamElementEntry);
+ AsyncResult asyncResult =
streamElementQueue.peekBlockingly();
+ executor.submit(() -> {
+ try {
+ output(asyncResult);
+ } catch (InterruptedException e) {
Review comment:
Why do we need this `InterruptedException`? I see that it's coming (and it
is the same in master branch) from `streamElementQueue.poll()`, however I don't
understand it:
1. We shouldn't be blocking neither in the mailbox thread (this version) nor
under the checkpoint lock (master version), so this already looks suspicious.
2. `poll` method that blocks is already "mildly" surprising contract on it's
own
3. From the code it looks like this `poll` call should never block and
changing it to a proper poll (returning `Optional` or `null`) and adding a
check state that the element is present, would work just as well but would be
better/cleaner.
@AHeise I remember that you were also rising some objections about the
`Emitter` code and that it could be simplified and you wanted to work on that
later. Did you have this in mind as well?
Am I missing something? If not can you @AHeise create a JIRA ticket for
cleaning up those things?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese
flinkbot edited a comment on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese URL: https://github.com/apache/flink/pull/9648#issuecomment-529346361 ## CI report: * 6def0e035c5e7f442e45a00f1f7628726aef42eb : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/126406386) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (FLINK-14010) Dispatcher & JobManagers don't give up leadership when AM is shut down
TisonKun created FLINK-14010:
Summary: Dispatcher & JobManagers don't give up leadership when AM
is shut down
Key: FLINK-14010
URL: https://issues.apache.org/jira/browse/FLINK-14010
Project: Flink
Issue Type: Bug
Components: Deployment / YARN, Runtime / Coordination
Affects Versions: 1.9.0, 1.8.1, 1.7.2, 1.10.0
Reporter: TisonKun
In YARN deployment scenario, YARN RM possibly launches a new AM for the job
even if the previous AM does not terminated, for example, when AMRM heartbeat
timeout. This is a common case that RM will send a shutdown request to the
previous AM and expect the AM shutdown properly.
However, currently in {{YARNResourceManager}}, we handle this request in
{{onShutdownRequest}} which simply close the {{YARNResourceManager}} *but not
Dispatcher and JobManagers*. Thus, Dispatcher and JobManager launched in new AM
cannot be granted leadership properly. Visually,
on previous AM: Dispatcher leader, JM leaders
on new AM: ResourceManager leader
since on client side or in per-job mode, JobManager address and port are
configured as the new AM, the whole cluster goes into an unrecoverable
inconsistent status: client all queries the dispatcher on new AM who is now the
leader. Briefly, Dispatcher and JobManagers on previous AM do not give up their
leadership properly.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [flink] flinkbot commented on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese
flinkbot commented on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese URL: https://github.com/apache/flink/pull/9648#issuecomment-529346361 ## CI report: * 6def0e035c5e7f442e45a00f1f7628726aef42eb : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (FLINK-13872) Translate Operations Playground to Chinese
[ https://issues.apache.org/jira/browse/FLINK-13872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925443#comment-16925443 ] Lord i Will edited comment on FLINK-13872 at 9/9/19 7:59 AM: - [~jark] [~fhueske] it seems that there's a conflict. But i have rebased on the master. Don't know why~ was (Author: a64011144): [~jark] [~fhueske] it seems that there've a conflict. But i have rebased on the master. Don't know why~ > Translate Operations Playground to Chinese > -- > > Key: FLINK-13872 > URL: https://issues.apache.org/jira/browse/FLINK-13872 > Project: Flink > Issue Type: Task > Components: chinese-translation, Documentation >Affects Versions: 1.9.0 >Reporter: Fabian Hueske >Assignee: Lord i Will >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The [Operations > Playground|https://ci.apache.org/projects/flink/flink-docs-release-1.9/getting-started/docker-playgrounds/flink_operations_playground.html] > is a quick and convenient way to learn about Flink's operational features > (job submission, failure recovery, job updates, scaling, metrics). > We should translate it to Chinese as well. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (FLINK-13872) Translate Operations Playground to Chinese
[ https://issues.apache.org/jira/browse/FLINK-13872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925443#comment-16925443 ] Lord i Will commented on FLINK-13872: - [~jark] [~fhueske] it seems that there've a conflict. But i have rebased on the master. Don't know why~ > Translate Operations Playground to Chinese > -- > > Key: FLINK-13872 > URL: https://issues.apache.org/jira/browse/FLINK-13872 > Project: Flink > Issue Type: Task > Components: chinese-translation, Documentation >Affects Versions: 1.9.0 >Reporter: Fabian Hueske >Assignee: Lord i Will >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The [Operations > Playground|https://ci.apache.org/projects/flink/flink-docs-release-1.9/getting-started/docker-playgrounds/flink_operations_playground.html] > is a quick and convenient way to learn about Flink's operational features > (job submission, failure recovery, job updates, scaling, metrics). > We should translate it to Chinese as well. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (FLINK-13619) Update FlinkKafkaProducerMigrationOperatorTest to restore from 1.9 savepoint
[
https://issues.apache.org/jira/browse/FLINK-13619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925439#comment-16925439
]
vinoyang commented on FLINK-13619:
--
[~pnowojski] So how to generate the binary file for this migration test? Or how
to verify this migration test?
> Update FlinkKafkaProducerMigrationOperatorTest to restore from 1.9 savepoint
>
>
> Key: FLINK-13619
> URL: https://issues.apache.org/jira/browse/FLINK-13619
> Project: Flink
> Issue Type: Sub-task
> Components: Connectors / Kafka, Tests
>Affects Versions: 1.10.0
>Reporter: Till Rohrmann
>Assignee: vinoyang
>Priority: Blocker
> Fix For: 1.10.0
>
>
> Update {{FlinkKafkaProducerMigrationOperatorTest}} to restore from 1.9
> savepoint.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [flink] flinkbot commented on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese
flinkbot commented on issue #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese URL: https://github.com/apache/flink/pull/9648#issuecomment-529343948 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 6def0e035c5e7f442e45a00f1f7628726aef42eb (Mon Sep 09 07:55:45 UTC 2019) ✅no warnings Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-13872) Translate Operations Playground to Chinese
[ https://issues.apache.org/jira/browse/FLINK-13872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-13872: --- Labels: pull-request-available (was: ) > Translate Operations Playground to Chinese > -- > > Key: FLINK-13872 > URL: https://issues.apache.org/jira/browse/FLINK-13872 > Project: Flink > Issue Type: Task > Components: chinese-translation, Documentation >Affects Versions: 1.9.0 >Reporter: Fabian Hueske >Assignee: Lord i Will >Priority: Major > Labels: pull-request-available > > The [Operations > Playground|https://ci.apache.org/projects/flink/flink-docs-release-1.9/getting-started/docker-playgrounds/flink_operations_playground.html] > is a quick and convenient way to learn about Flink's operational features > (job submission, failure recovery, job updates, scaling, metrics). > We should translate it to Chinese as well. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] lord-i-will opened a new pull request #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese
lord-i-will opened a new pull request #9648: [FLINK-13872] [docs-zh] Translate Operations Playground to Chinese URL: https://github.com/apache/flink/pull/9648 Please have a check, thx. @wuchong @fhueske This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (FLINK-14009) Cron jobs broken due to verifying incorrect NOTICE-binary file
Till Rohrmann created FLINK-14009:
-
Summary: Cron jobs broken due to verifying incorrect NOTICE-binary
file
Key: FLINK-14009
URL: https://issues.apache.org/jira/browse/FLINK-14009
Project: Flink
Issue Type: Bug
Components: Build System
Affects Versions: 1.10.0
Reporter: Till Rohrmann
Assignee: Till Rohrmann
Fix For: 1.8.2, 1.9.1
With FLINK-13968 we introduced an automatic {{NOTICE-binary}} file check.
However, since we don't use the correct {{NOTICE-binary}} file (FLINK-14008)
for Scala 2.12 it fails currently our cron jobs.
I suggest to only enable the automatic {{NOTICE-binary}} files for Scala 2.11
until FLINK-14008 has been fixed.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Created] (FLINK-14008) Flink Scala 2.12 binary distribution contains incorrect NOTICE-binary file
Till Rohrmann created FLINK-14008:
-
Summary: Flink Scala 2.12 binary distribution contains incorrect
NOTICE-binary file
Key: FLINK-14008
URL: https://issues.apache.org/jira/browse/FLINK-14008
Project: Flink
Issue Type: Bug
Components: Build System
Affects Versions: 1.9.0, 1.8.1, 1.10.0
Reporter: Till Rohrmann
Fix For: 1.10.0
The Flink Scala {{2.12}} binary distribution contains an incorrect
NOTICE-binary file. The problem is that we don't update the Scala version of
the Scala dependencies listed in the {{NOTICE-binary}} file.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (FLINK-13968) Add travis check for the correctness of the binary licensing
[ https://issues.apache.org/jira/browse/FLINK-13968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925424#comment-16925424 ] Till Rohrmann commented on FLINK-13968: --- Failing cron job: https://api.travis-ci.org/v3/job/582421469/log.txt > Add travis check for the correctness of the binary licensing > > > Key: FLINK-13968 > URL: https://issues.apache.org/jira/browse/FLINK-13968 > Project: Flink > Issue Type: Improvement > Components: Travis >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Major > Labels: pull-request-available > Fix For: 1.10.0, 1.9.1, 1.8.3 > > Time Spent: 20m > Remaining Estimate: 0h > > Since the binary licensing must be updated manually whenever the packaging of > flink-dist or contained jars is modified we should add an automatic check to > ensure the licensing is updated appropriately. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (FLINK-13968) Add travis check for the correctness of the binary licensing
[ https://issues.apache.org/jira/browse/FLINK-13968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925423#comment-16925423 ] Till Rohrmann commented on FLINK-13968: --- I think this change breaks our cron jobs, because the NOTICE-binary files isn't correct when building with Scala 2.12. > Add travis check for the correctness of the binary licensing > > > Key: FLINK-13968 > URL: https://issues.apache.org/jira/browse/FLINK-13968 > Project: Flink > Issue Type: Improvement > Components: Travis >Reporter: Chesnay Schepler >Assignee: Chesnay Schepler >Priority: Major > Labels: pull-request-available > Fix For: 1.10.0, 1.9.1, 1.8.3 > > Time Spent: 20m > Remaining Estimate: 0h > > Since the binary licensing must be updated manually whenever the packaging of > flink-dist or contained jars is modified we should add an automatic check to > ensure the licensing is updated appropriately. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9432: [FLINK-13653][sql-client] ResultStore should avoid using RowTypeInfo …
flinkbot edited a comment on issue #9432: [FLINK-13653][sql-client] ResultStore should avoid using RowTypeInfo … URL: https://github.com/apache/flink/pull/9432#issuecomment-521081788 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 9f196705610a4b48715dcff37610dee4438fc010 (Mon Sep 09 07:22:34 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❗ 3. Needs [attention] from. - Needs attention by @docete * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9647: [FLINK-14005][hive] Support Hive version 2.2.0
flinkbot edited a comment on issue #9647: [FLINK-14005][hive] Support Hive version 2.2.0 URL: https://github.com/apache/flink/pull/9647#issuecomment-529282630 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit cab04a32f620acef83f46bc111512b748b4acce8 (Mon Sep 09 07:07:18 UTC 2019) **Warnings:** * **1 pom.xml files were touched**: Check for build and licensing issues. * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-14002) FlinkKafkaProducer constructor that takes KafkaSerializationSchema shouldnt take default topic
[ https://issues.apache.org/jira/browse/FLINK-14002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925398#comment-16925398 ] Gyula Fora commented on FLINK-14002: Actually the default topic is used in case of KafkaContextAware serialization schemas which might cause further problems. > FlinkKafkaProducer constructor that takes KafkaSerializationSchema shouldnt > take default topic > -- > > Key: FLINK-14002 > URL: https://issues.apache.org/jira/browse/FLINK-14002 > Project: Flink > Issue Type: Improvement > Components: Connectors / Kafka >Reporter: Gyula Fora >Priority: Major > > When the KafkaSerializationSchema is used the user has the to provide the > topic always when they create the ProducerRecord. > The defaultTopic specified in the constructor (and enforced not to be null) > will always be ignored, this is very misleading. > We should depracate these constructors and create new ones without > defaultTopic. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (FLINK-14003) Add access to customized state other than built in state types
[ https://issues.apache.org/jira/browse/FLINK-14003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925394#comment-16925394 ] Shimin Yang commented on FLINK-14003: - [~carp84], sounds good. > Add access to customized state other than built in state types > -- > > Key: FLINK-14003 > URL: https://issues.apache.org/jira/browse/FLINK-14003 > Project: Flink > Issue Type: Wish > Components: Runtime / State Backends >Reporter: Shimin Yang >Priority: Minor > > Currently the keyed state backend supports states like value, list, map, > although they are quite useful, but it may not fits all user cases. > User can develop their customized state backend based on different data store > as a plugin, but they do not have access to customized state if the type is > not one of those built-in types. > I propose to add getPartitionedState method to RuntimeContext and > KeyedStateStore interface to allow user to access their customized state, and > it will not affect current code path. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9634: [hotfix][runtime] Streaming runtime 1.9 improvements for debuggability
flinkbot edited a comment on issue #9634: [hotfix][runtime] Streaming runtime 1.9 improvements for debuggability URL: https://github.com/apache/flink/pull/9634#issuecomment-528758790 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 313d9b3295124b77fee967c76f63f5aef9bd3db8 (Mon Sep 09 06:14:24 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] pnowojski commented on issue #9634: [hotfix][runtime] Streaming runtime 1.9 improvements for debuggability
pnowojski commented on issue #9634: [hotfix][runtime] Streaming runtime 1.9 improvements for debuggability URL: https://github.com/apache/flink/pull/9634#issuecomment-529314668 @zhijiangW sorry for misunderstanding, but I wrote that we can backport the `perfomDefaultAction` rename if it's not too much hassle (no conflicts etc.). As @1u0 has already done it, it's better to merge both commits, as it will make 1.9 and 1.10 branches a little bit more similar, so it will be easier to backport future changes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
